May 09 14:04:10.874955 np0000173183 systemd[1]: Started devstack@neutron-ovn-maintenance-worker.service - Devstack devstack@neutron-ovn-maintenance-worker.service.
May 09 14:04:11.190644 np0000173183 neutron-ovn-maintenance-worker[104836]: /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:41: EventletDeprecationWarning:
May 09 14:04:11.190644 np0000173183 neutron-ovn-maintenance-worker[104836]: Eventlet is deprecated. It is currently being maintained in bugfix mode, and
May 09 14:04:11.190644 np0000173183 neutron-ovn-maintenance-worker[104836]: we strongly recommend against using it for new projects.
May 09 14:04:11.190644 np0000173183 neutron-ovn-maintenance-worker[104836]: If you are already using Eventlet, we recommend migrating to a different
May 09 14:04:11.190644 np0000173183 neutron-ovn-maintenance-worker[104836]: framework.  For more detail see
May 09 14:04:11.190644 np0000173183 neutron-ovn-maintenance-worker[104836]: https://eventlet.readthedocs.io/en/latest/asyncio/migration.html
May 09 14:04:11.190644 np0000173183 neutron-ovn-maintenance-worker[104836]:   import eventlet
May 09 14:04:12.180058 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.common.config [-] Logging enabled!
May 09 14:04:12.180211 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-ovn-maintenance-worker version 28.1.0.dev201
May 09 14:04:12.180429 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-ovn-maintenance-worker --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini {{(pid=104836) setup_logging /opt/stack/neutron/neutron/common/config.py:124}}
May 09 14:04:12.180631 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.server.ovn_maintenance [-] OVN maintenance process starting...
May 09 14:04:12.180697 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "plugin-directory" by "neutron_lib.plugins.directory._create_plugin_directory" {{(pid=104836) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:506}}
May 09 14:04:12.180815 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG oslo_concurrency.lockutils [-] Lock "plugin-directory" acquired by "neutron_lib.plugins.directory._create_plugin_directory" :: waited 0.000s {{(pid=104836) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:519}}
May 09 14:04:12.180878 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG oslo_concurrency.lockutils [-] Lock "plugin-directory" "released" by "neutron_lib.plugins.directory._create_plugin_directory" :: held 0.000s {{(pid=104836) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:538}}
May 09 14:04:12.180940 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "manager" by "neutron.manager.NeutronManager._create_instance" {{(pid=104836) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:506}}
May 09 14:04:12.180996 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG oslo_concurrency.lockutils [-] Lock "manager" acquired by "neutron.manager.NeutronManager._create_instance" :: waited 0.000s {{(pid=104836) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:519}}
May 09 14:04:12.181069 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.manager [-] Loading core plugin: ml2
May 09 14:04:12.435701 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.securitygroup.SecurityGroup'>> rbac-policy before_create 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:12.436324 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.securitygroup.SecurityGroup'>> rbac-policy before_update 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:12.436324 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.securitygroup.SecurityGroup'>> rbac-policy before_delete 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:12.436482 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.address_group.AddressGroup'>> rbac-policy before_create 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:12.436557 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.address_group.AddressGroup'>> rbac-policy before_update 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:12.436557 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.address_group.AddressGroup'>> rbac-policy before_delete 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:12.469862 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.network.Network'>> rbac-policy before_create 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:12.470011 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.network.Network'>> rbac-policy before_update 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:12.470011 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.network.Network'>> rbac-policy before_delete 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:12.562765 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.qos.policy.QosPolicy'>> rbac-policy before_create 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:12.562924 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.qos.policy.QosPolicy'>> rbac-policy before_update 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:12.562924 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.qos.policy.QosPolicy'>> rbac-policy before_delete 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:12.583118 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.subnetpool.SubnetPool'>> rbac-policy before_create 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:12.583118 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.subnetpool.SubnetPool'>> rbac-policy before_update 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:12.583118 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.subnetpool.SubnetPool'>> rbac-policy before_delete 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:12.583118 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.address_scope.AddressScope'>> rbac-policy before_create 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:12.583118 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.address_scope.AddressScope'>> rbac-policy before_update 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:12.583495 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.address_scope.AddressScope'>> rbac-policy before_delete 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:12.664960 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <function _update_segment_host_mapping_for_agent at 0x735c988ce660> agent after_create 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:12.665105 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <function _update_segment_host_mapping_for_agent at 0x735c988ce660> agent after_update 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:12.665105 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <function _add_segment_host_mapping_for_segment at 0x735c988ce700> segment precommit_create 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:12.665105 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <function _delete_segments_for_network at 0x735c988ce7a0> network precommit_delete 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:12.665382 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <function _prevent_segment_delete_with_port_bound at 0x735c988cf9c0> segment before_delete 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:12.714356 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method External_net_db_mixin._before_network_delete_handler of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x735c998e1160>> network before_delete 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:12.714526 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <function DVRDbMixin._delete_mac_associated_with_agent at 0x735c988be0c0> agent before_delete 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:12.714526 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SecurityGroupDbMixin._ensure_default_security_group_handler of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x735c998e1160>> network before_create 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:12.714526 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SecurityGroupDbMixin._ensure_default_security_group_handler of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x735c998e1160>> port before_create 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:12.714807 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SecurityGroupDbMixin._ensure_default_security_group_handler of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x735c998e1160>> port before_update 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:12.714807 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._handle_segment_change of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x735c998e1160>> segment precommit_create 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:12.714807 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._handle_segment_change of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x735c998e1160>> segment precommit_delete 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:12.714807 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._handle_segment_change of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x735c998e1160>> segment after_create 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:12.715070 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._handle_segment_change of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x735c998e1160>> segment after_delete 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:12.715070 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._network_delete_after_delete_handler of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x735c998e1160>> network after_delete 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:12.715070 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._network_delete_precommit_handler of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x735c998e1160>> network precommit_delete 0, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:12.715070 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._port_provisioned of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x735c998e1160>> port provisioning_complete 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:12.715070 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method External_net_db_mixin._process_ext_policy_create of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x735c998e1160>> rbac-policy before_create 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:12.715502 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method External_net_db_mixin._process_ext_policy_delete of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x735c998e1160>> rbac-policy after_delete 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:12.715502 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._retry_binding_revived_agents of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x735c998e1160>> agent after_update 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:12.715502 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._subnet_delete_after_delete_handler of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x735c998e1160>> subnet after_delete 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:12.715502 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._subnet_delete_precommit_handler of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x735c998e1160>> subnet precommit_delete 0, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:12.715502 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method External_net_db_mixin._validate_ext_not_in_use_by_project of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x735c998e1160>> rbac-policy before_update 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:12.715794 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method External_net_db_mixin._validate_ext_not_in_use_by_project of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x735c998e1160>> rbac-policy before_delete 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:12.715794 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method DhcpAgentSchedulerDbMixin.auto_schedule_new_network_segments of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x735c998e1160>> segment_host_mapping after_create 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:12.715794 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin.delete_agent_notified of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x735c998e1160>> agent after_delete 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:12.715794 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NeutronDbPluginV2.validate_network_rbac_policy_change of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x735c998e1160>> rbac-policy before_create 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:12.715794 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NeutronDbPluginV2.validate_network_rbac_policy_change of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x735c998e1160>> rbac-policy before_update 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:12.716076 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NeutronDbPluginV2.validate_network_rbac_policy_change of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x735c998e1160>> rbac-policy before_delete 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:12.716076 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: network configured {{(pid=104836) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}}
May 09 14:04:12.716196 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: port configured {{(pid=104836) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}}
May 09 14:04:12.716196 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: subnet configured {{(pid=104836) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}}
May 09 14:04:12.716313 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: subnetpool configured {{(pid=104836) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}}
May 09 14:04:12.716313 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: security_group configured {{(pid=104836) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}}
May 09 14:04:12.716313 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: security_group_rule configured {{(pid=104836) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}}
May 09 14:04:12.716523 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.plugins.ml2.managers [-] Configured type driver names: ['local', 'flat', 'vlan', 'geneve']
May 09 14:04:12.717160 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.plugins.ml2.drivers.type_flat [-] Allowable flat physical_network names: ['public']
May 09 14:04:12.717736 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.plugins.ml2.drivers.type_local [-] ML2 LocalTypeDriver initialization complete
May 09 14:04:12.718201 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.plugins.ml2.drivers.type_vlan [-] Network VLAN ranges: OrderedDict({'public': [(1, 4094)]})
May 09 14:04:12.718278 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.plugins.ml2.managers [-] Loaded type driver names: ['flat', 'geneve', 'local', 'vlan']
May 09 14:04:12.718425 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.plugins.ml2.managers [-] Registered types: dict_keys(['flat', 'geneve', 'local', 'vlan'])
May 09 14:04:12.718493 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.plugins.ml2.managers [-] Project network_types: ['geneve']
May 09 14:04:12.718610 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.plugins.ml2.managers [-] Configured extension driver names: ['port_security', 'qos']
May 09 14:04:12.719110 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.plugins.ml2.managers [-] Loaded extension driver names: ['port_security', 'qos']
May 09 14:04:12.719210 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.plugins.ml2.managers [-] Registered extension drivers: ['port_security', 'qos']
May 09 14:04:12.719299 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.plugins.ml2.managers [-] Configured mechanism driver names: ['ovn']
May 09 14:04:12.755065 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.plugins.ml2.managers [-] Loaded mechanism driver names: ['ovn']
May 09 14:04:12.755065 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.plugins.ml2.managers [-] Registered mechanism drivers: ['ovn']
May 09 14:04:12.755270 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.plugins.ml2.managers [-] No mechanism drivers provide segment reachability information for agent scheduling.
May 09 14:04:12.755734 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Notifier._send_nova_notification of <neutron.notifiers.nova.Notifier object at 0x735c98698f50>> floatingip before_response 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:12.755839 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Notifier._send_nova_notification of <neutron.notifiers.nova.Notifier object at 0x735c98698f50>> port before_response 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:12.773204 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.plugins.ml2.managers [-] Initializing driver for type 'flat'
May 09 14:04:12.773204 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.plugins.ml2.drivers.type_flat [-] ML2 FlatTypeDriver initialization complete
May 09 14:04:12.773473 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.plugins.ml2.managers [-] Initializing driver for type 'geneve'
May 09 14:04:12.773473 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.plugins.ml2.drivers.type_tunnel [-] geneve ID ranges: [(1, 65536)]
May 09 14:04:12.817743 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO dbcounter [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Registered counter for database neutron
May 09 14:04:12.832755 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG oslo_db.sqlalchemy.engines [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=104836) _check_effective_sql_mode /opt/stack/data/venv/lib/python3.12/site-packages/oslo_db/sqlalchemy/engines.py:325}}
May 09 14:04:13.217385 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG dbcounter [-] [104836] Writer thread running {{(pid=104836) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:102}}
May 09 14:04:13.410559 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.plugins.ml2.managers [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Initializing driver for type 'local'
May 09 14:04:13.410770 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.plugins.ml2.managers [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Initializing driver for type 'vlan'
May 09 14:04:13.517894 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.plugins.ml2.drivers.type_vlan [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] VlanTypeDriver initialization complete
May 09 14:04:13.518598 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.plugins.ml2.managers [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Initializing extension driver 'port_security'
May 09 14:04:13.518598 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.plugins.ml2.extensions.port_security [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] PortSecurityExtensionDriver initialization complete
May 09 14:04:13.518598 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.plugins.ml2.managers [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Initializing extension driver 'qos'
May 09 14:04:13.518598 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.plugins.ml2.extensions.qos [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] QosExtensionDriver initialization complete {{(pid=104836) initialize /opt/stack/neutron/neutron/plugins/ml2/extensions/qos.py:31}}
May 09 14:04:13.518598 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.plugins.ml2.managers [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Initializing mechanism driver 'ovn'
May 09 14:04:13.518598 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Starting OVNMechanismDriver
May 09 14:04:13.519668 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Subscribe: <bound method OVNMechanismDriver.pre_fork_initialize of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x735c986096a0>> process before_spawn 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.519762 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Subscribe: <bound method OVNMechanismDriver.post_fork_initialize of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x735c986096a0>> process after_init 55550000, True {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.519873 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Subscribe: <bound method OVNMechanismDriver._add_segment_host_mapping_for_segment of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x735c986096a0>> segment after_create 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.519985 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Subscribe: <bound method OVNMechanismDriver.create_segment_provnet_port of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x735c986096a0>> segment after_create 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.520050 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Subscribe: <bound method OVNMechanismDriver.delete_segment_provnet_port of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x735c986096a0>> segment after_delete 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.520112 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Subscribe: <bound method OVNMechanismDriver._validate_allowed_address_pairs of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x735c986096a0>> allowed_address_pair before_create 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.520183 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Subscribe: <bound method OVNMechanismDriver._create_security_group_precommit of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x735c986096a0>> security_group precommit_create 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.521535 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Subscribe: <bound method OVNMechanismDriver._update_security_group of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x735c986096a0>> security_group after_update 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.521535 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Subscribe: <bound method OVNMechanismDriver._create_security_group of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x735c986096a0>> security_group after_create 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.521535 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Subscribe: <bound method OVNMechanismDriver._delete_security_group_precommit of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x735c986096a0>> security_group precommit_delete 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.521535 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Subscribe: <bound method OVNMechanismDriver._delete_security_group of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x735c986096a0>> security_group after_delete 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.521847 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Subscribe: <bound method OVNMechanismDriver._create_sg_rule_precommit of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x735c986096a0>> security_group_rule precommit_create 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.521847 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Subscribe: <bound method OVNMechanismDriver._process_sg_rule_notification of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x735c986096a0>> security_group_rule after_create 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.521847 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Subscribe: <bound method OVNMechanismDriver._process_sg_rule_notification of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x735c986096a0>> security_group_rule before_delete 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.521847 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Subscribe: <bound method OVNMechanismDriver._process_ag_notification of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x735c986096a0>> address_group after_create 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.522101 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Subscribe: <bound method OVNMechanismDriver._process_ag_notification of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x735c986096a0>> address_group after_update 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.522101 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Subscribe: <bound method OVNMechanismDriver._process_ag_notification of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x735c986096a0>> address_group after_delete 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.522101 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Subscribe: <bound method DriverBase._register of <neutron.services.qos.drivers.ovn.driver.OVNQosDriver object at 0x735c9828f410>> qos_plugin after_init 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.522101 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Subscribe: <bound method OVNTrunkDriver.register of <neutron.services.trunk.drivers.ovn.trunk_driver.OVNTrunkDriver object at 0x735c9852e180>> trunk_plugin after_init 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.522510 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Subscribe: <bound method DriverBase._register of <neutron.services.logapi.drivers.ovn.driver.OVNDriver object at 0x735c9823b080>> logging-plugin after_init 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.522510 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.services.logapi.drivers.ovn.driver [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] OVN logging driver registered
May 09 14:04:13.523143 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.plugins.ml2.plugin [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] neutron.plugins.ml2.plugin.Ml2Plugin method _start_rpc_notifiers called with arguments () {} {{(pid=104836) wrapper /opt/stack/data/venv/lib/python3.12/site-packages/oslo_log/helpers.py:77}}
May 09 14:04:13.523361 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x735c96f1e480>> port after_create 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.523448 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x735c96f1e480>> port after_update 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.523535 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x735c96f1e480>> port after_delete 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.523598 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x735c96f1e540>> subnet after_create 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.523689 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x735c96f1e540>> subnet after_update 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.523764 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x735c96f1e540>> subnet after_delete 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.523845 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x735c96f1e5a0>> network after_create 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.523928 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x735c96f1e5a0>> network after_update 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.524005 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x735c96f1e5a0>> network after_delete 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.524071 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x735c96f1e5d0>> security_group after_create 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.524163 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x735c96f1e5d0>> security_group after_update 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.524234 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x735c96f1e5d0>> security_group after_delete 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.524310 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x735c96f1e570>> security_group_rule after_create 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.524387 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x735c96f1e570>> security_group_rule after_update 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.524481 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x735c96f1e570>> security_group_rule after_delete 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.524577 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x735c96f1e660>> address_group after_create 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.524625 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x735c96f1e660>> address_group after_update 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.524997 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x735c96f1e660>> address_group after_delete 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.524997 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.plugins.ml2.ovo_rpc [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] ML2 OVO RPC backend initialized. {{(pid=104836) __init__ /opt/stack/neutron/neutron/plugins/ml2/ovo_rpc.py:117}}
May 09 14:04:13.525398 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.quota [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loaded quota_driver: <neutron.db.quota.driver_nolock.DbQuotaNoLockDriver object at 0x735c96f1e690>.
May 09 14:04:13.525514 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.plugins.ml2.plugin [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Modular L2 Plugin initialization complete
May 09 14:04:13.525619 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading services supported by the core plugin {{(pid=104836) _load_services_from_core_plugin /opt/stack/neutron/neutron/manager.py:170}}
May 09 14:04:13.525673 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.plugins.ml2.managers [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Got port-security extension from driver 'port_security'
May 09 14:04:13.525917 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading service plugins: ['ovn-router', 'auto_allocate', 'tag', 'timestamp', 'network_ip_availability', 'flavors', 'revisions'] {{(pid=104836) _load_service_plugins /opt/stack/neutron/neutron/manager.py:195}}
May 09 14:04:13.525987 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading Plugin: ovn-router
May 09 14:04:13.534116 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Subscribe: <bound method ExtraGatewaysDbOnlyMixin._delete_router_remove_external_gateways of <neutron.services.ovn_l3.plugin.OVNL3RouterPlugin object at 0x735c96f1cda0>> router before_delete 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.534231 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Subscribe: <function L3RpcNotifierMixin._notify_gateway_port_ip_changed at 0x735c98723ec0> port after_update 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.534314 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Subscribe: <function L3RpcNotifierMixin._notify_routers_callback at 0x735c98728040> port after_delete 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.534392 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Subscribe: <function L3RpcNotifierMixin._notify_subnet_gateway_ip_update at 0x735c98723f60> subnet after_update 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.534564 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Subscribe: <function L3RpcNotifierMixin._notify_subnetpool_address_scope_update at 0x735c98723e20> subnetpool_address_scope after_update 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.534564 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Subscribe: <function OVNL3RouterPlugin._port_update at 0x735c96ac3920> port before_update 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.534654 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Subscribe: <function OVNL3RouterPlugin._port_update at 0x735c96ac3920> port after_update 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.534806 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Subscribe: <bound method L3_NAT_dbonly_mixin._precommit_delete_port_callback of <neutron.services.ovn_l3.plugin.OVNL3RouterPlugin object at 0x735c96f1cda0>> port precommit_delete 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.534887 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Subscribe: <function L3_NAT_dbonly_mixin._prevent_internal_ip_change_for_fip at 0x735c987176a0> port before_update 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.534964 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Subscribe: <function L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback at 0x735c98717600> port before_delete 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.535043 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Subscribe: <bound method RouterAvailabilityZoneMixin._process_az_request of <neutron.services.ovn_l3.plugin.OVNL3RouterPlugin object at 0x735c96f1cda0>> router precommit_create 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.535128 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Subscribe: <bound method ExtraGatewaysDbOnlyMixin._process_bfd_ecmp_create of <neutron.services.ovn_l3.plugin.OVNL3RouterPlugin object at 0x735c96f1cda0>> router precommit_create 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.535231 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Subscribe: <bound method ExtraGatewaysDbOnlyMixin._process_bfd_ecmp_update of <neutron.services.ovn_l3.plugin.OVNL3RouterPlugin object at 0x735c96f1cda0>> router precommit_update 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.535285 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Subscribe: <function OVNL3RouterPlugin._subnet_update at 0x735c96ac3880> subnet after_update 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.535870 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.quota.resource_registry [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Tracking information for resource: router configured {{(pid=104836) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}}
May 09 14:04:13.535946 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.quota.resource_registry [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Tracking information for resource: floatingip configured {{(pid=104836) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}}
May 09 14:04:13.536018 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.services.ovn_l3.plugin [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Starting OVNL3RouterPlugin
May 09 14:04:13.536194 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Subscribe: <bound method OVNPortForwarding.register of <neutron.services.portforwarding.drivers.ovn.driver.OVNPortForwarding object at 0x735c96f1c860>> port_forwarding_plugin after_init 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.536367 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Subscribe: <bound method DriverController._check_router_request of <neutron.services.ovn_l3.service_providers.driver_controller.DriverController object at 0x735c96f1ce60>> router before_create 55550100, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.536470 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Subscribe: <bound method DriverController._clear_router_provider of <neutron.services.ovn_l3.service_providers.driver_controller.DriverController object at 0x735c96f1ce60>> router precommit_delete 55550100, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.536552 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Subscribe: <bound method DriverController._set_router_provider of <neutron.services.ovn_l3.service_providers.driver_controller.DriverController object at 0x735c96f1ce60>> router precommit_create 55550100, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.536630 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Subscribe: <bound method DriverController._update_router_provider of <neutron.services.ovn_l3.service_providers.driver_controller.DriverController object at 0x735c96f1ce60>> router precommit_update 55550100, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.539086 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.services.provider_configuration [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Service providers = [] {{(pid=104836) parse_service_provider_opt /opt/stack/neutron/neutron/services/provider_configuration.py:170}}
May 09 14:04:13.539168 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.db.servicetype_db [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Adding provider configuration for service L3_ROUTER_NAT {{(pid=104836) add_provider_configuration /opt/stack/neutron/neutron/db/servicetype_db.py:45}}
May 09 14:04:13.540245 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Subscribe: <bound method OVN_L3_HA_db_mixin._precommit_router_create of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x735c96b00470>> router precommit_create 55549900, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.540370 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Subscribe: <bound method OvnDriver._process_add_router_interface of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x735c96b00470>> router_interface after_create 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.540370 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Subscribe: <bound method OvnDriver._process_floatingip_create of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x735c96b00470>> floatingip after_create 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.540534 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Subscribe: <bound method OvnDriver._process_floatingip_create_update_precommit of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x735c96b00470>> floatingip precommit_create 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.540534 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Subscribe: <bound method OvnDriver._process_floatingip_create_update_precommit of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x735c96b00470>> floatingip precommit_update 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.540646 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Subscribe: <bound method OvnDriver._process_floatingip_delete of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x735c96b00470>> floatingip after_delete 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.540708 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Subscribe: <bound method OvnDriver._process_floatingip_status_update of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x735c96b00470>> floatingip after_status_update 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.540776 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Subscribe: <bound method OvnDriver._process_floatingip_update of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x735c96b00470>> floatingip after_update 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.540836 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Subscribe: <bound method OvnDriver._process_portforwarding_create of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x735c96b00470>> port_forwarding after_create 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.540895 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Subscribe: <bound method OvnDriver._process_remove_router_interface of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x735c96b00470>> router_interface after_delete 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.540954 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Subscribe: <bound method OvnDriver._process_router_create of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x735c96b00470>> router after_create 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.541049 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Subscribe: <bound method OvnDriver._process_router_create_precommit of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x735c96b00470>> router precommit_create 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.541116 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Subscribe: <bound method OvnDriver._process_router_delete of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x735c96b00470>> router after_delete 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.541194 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Subscribe: <bound method OvnDriver._process_router_update of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x735c96b00470>> router after_update 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.541286 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.services.service_base [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loaded 'neutron.services.ovn_l3.service_providers.ovn.OvnDriver' provider for service L3_ROUTER_NAT {{(pid=104836) load_drivers /opt/stack/neutron/neutron/services/service_base.py:49}}
May 09 14:04:13.541378 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Subscribe: <bound method OVNL3RouterPlugin._post_fork_initialize of <neutron.services.ovn_l3.plugin.OVNL3RouterPlugin object at 0x735c96f1cda0>> process after_init 55550001, True {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.541498 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Successfully loaded L3_ROUTER_NAT plugin. Description: L3 Router Service Plugin for basic L3 forwarding using OVN {{(pid=104836) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:239}}
May 09 14:04:13.541578 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading Plugin: auto_allocate
May 09 14:04:13.542261 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Subscribe: <function _ensure_external_network_default_value_callback at 0x735c96a9dbc0> network precommit_update 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.542366 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron_lib.callbacks.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Subscribe: <function _ensure_external_network_default_value_callback at 0x735c96a9dbc0> network precommit_create 55550000, False {{(pid=104836) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.542534 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Successfully loaded auto-allocated-topology plugin. Description: Auto Allocated Topology - aka get me a network. {{(pid=104836) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:239}}
May 09 14:04:13.542610 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading Plugin: tag
May 09 14:04:13.544917 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Successfully loaded TAG plugin. Description: Tag support {{(pid=104836) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:239}}
May 09 14:04:13.545014 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading Plugin: timestamp
May 09 14:04:13.545962 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Successfully loaded timestamp plugin. Description: Adds timestamps to Neutron resources with standard attributes {{(pid=104836) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:239}}
May 09 14:04:13.546049 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading Plugin: network_ip_availability
May 09 14:04:13.547164 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Successfully loaded network-ip-availability plugin. Description: Provides IP availability data for each network and subnet. {{(pid=104836) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:239}}
May 09 14:04:13.547250 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading Plugin: flavors
May 09 14:04:13.547989 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Successfully loaded FLAVORS plugin. Description: Neutron Flavors and Service Profiles manager plugin {{(pid=104836) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:239}}
May 09 14:04:13.548087 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading Plugin: revisions
May 09 14:04:13.548829 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.manager [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Successfully loaded revision_plugin plugin. Description: Adds revision numbers to resources. {{(pid=104836) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:239}}
May 09 14:04:13.548936 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG oslo_concurrency.lockutils [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Lock "manager" "released" by "neutron.manager.NeutronManager._create_instance" :: held 1.368s {{(pid=104836) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:538}}
May 09 14:04:13.549094 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] get_extension_paths = OrderedDict({'/opt/stack/neutron/neutron/extensions': 1}) {{(pid=104836) get_extensions_path /opt/stack/neutron/neutron/api/extensions.py:640}}
May 09 14:04:13.549161 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Initializing extension manager.
May 09 14:04:13.549387 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: __init__.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.549462 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: __pycache__ {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.549555 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: address_group.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.549850 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Address group" alias="address-group" description="Support address group" updated="2020-07-15T10:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.550041 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loaded extension: address-group
May 09 14:04:13.550107 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: address_scope.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.550357 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Address scope" alias="address-scope" description="Address scopes extension." updated="2015-07-26T10:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.550527 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loaded extension: address-scope
May 09 14:04:13.550583 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: admin_state_down_before_update.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.550784 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Enforce Router's Admin State Down Before Update Extension" alias="router-admin-state-down-before-update" description="Ensure that the admin state of a router is DOWN (admin_state_up=False) before updating the distributed attribute" updated="2019-07-02T15:56:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.550917 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Extension router-admin-state-down-before-update not supported by any of loaded plugins
May 09 14:04:13.550988 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: agent.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.551222 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="agent" alias="agent" description="The agent management extension." updated="2013-02-03T10:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.551359 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loaded extension: agent
May 09 14:04:13.551444 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: agent_resources_synced.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.551628 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Agent's Resource View Synced to Placement" alias="agent-resources-synced" description="Stores success/failure of last sync to Placement" updated="2018-12-19T00:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.551764 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Extension agent-resources-synced not supported by any of loaded plugins
May 09 14:04:13.551834 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: agent_sort_key.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.552012 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Agents Sort Key" alias="agent_sort_key" description="Enabling the keyword sort_key for sorting in agents." updated="2013-02-03T10:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.552148 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Extension agent_sort_key not supported by any of loaded plugins
May 09 14:04:13.552204 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: allowedaddresspairs.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.552761 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Allowed Address Pairs" alias="allowed-address-pairs" description="Provides allowed address pairs" updated="2013-07-23T10:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.552905 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loaded extension: allowed-address-pairs
May 09 14:04:13.552968 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: auto_allocated_topology.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.553167 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Auto Allocated Topology Services" alias="auto-allocated-topology" description="Auto Allocated Topology Services." updated="2016-01-01T00:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.553310 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loaded extension: auto-allocated-topology
May 09 14:04:13.553373 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: availability_zone.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.553620 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Availability Zone" alias="availability_zone" description="The availability zone extension." updated="2015-01-01T10:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.553771 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loaded extension: availability_zone
May 09 14:04:13.553842 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: availability_zone_filter.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.554030 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Availability Zone Filter Extension" alias="availability_zone_filter" description="Add filter parameters to AvailabilityZone resource" updated="2018-06-22T10:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.554159 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Extension availability_zone_filter not supported by any of loaded plugins
May 09 14:04:13.554229 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: data_plane_status.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.554430 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Port data plane status extension" alias="data-plane-status" description="Expose status of underlying data plane" updated="2017-01-24T10:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.554554 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Extension data-plane-status not supported by any of loaded plugins
May 09 14:04:13.554625 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: default_subnetpools.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.554813 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Default Subnetpools" alias="default-subnetpools" description="Provides ability to mark and use a subnetpool as the default." updated="2016-02-18T18:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.554970 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loaded extension: default-subnetpools
May 09 14:04:13.555038 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: dhcpagentscheduler.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.555330 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="DHCP Agent Scheduler" alias="dhcp_agent_scheduler" description="Schedule networks among dhcp agents" updated="2013-02-07T10:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.555494 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loaded extension: dhcp_agent_scheduler
May 09 14:04:13.555543 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: dns.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.555743 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="DNS Integration" alias="dns-integration" description="Provides integration with DNS." updated="2015-08-15T18:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.555877 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loaded extension: dns-integration
May 09 14:04:13.555930 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: dns_domain_ports.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.556119 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="dns_domain for ports" alias="dns-domain-ports" description="Allows the DNS domain to be specified for a network port." updated="2017-04-24T10:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.556247 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loaded extension: dns-domain-ports
May 09 14:04:13.556319 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: dns_integration_domain_keywords.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.556518 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="DNS domain names with keywords allowed" alias="dns-integration-domain-keywords" description="Allows to use keywords like <project_id>, <project_name>, <user_id> and <user_name> as DNS domain name" updated="2020-06-15T18:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.556654 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loaded extension: dns-integration-domain-keywords
May 09 14:04:13.556714 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: dvr.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.556915 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Distributed Virtual Router" alias="dvr" description="Enables configuration of Distributed Virtual Routers." updated="2014-06-1T10:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.557047 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Extension dvr not supported by any of loaded plugins
May 09 14:04:13.557115 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: ecmp_routes.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.557288 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Equal Cost Multipath Routing" alias="ecmp_routes" description="Allows traffic to reach thesame destination via multiple different links " updated="2020-11-26T18:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.557440 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Extension ecmp_routes not supported by any of loaded plugins
May 09 14:04:13.557511 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: empty_string_filtering.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.557734 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Empty String Filtering Extension" alias="empty-string-filtering" description="Allow filtering by attributes with empty string value" updated="2018-05-01T10:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.557854 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loaded extension: empty-string-filtering
May 09 14:04:13.557923 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: expose_l3_conntrack_helper.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.558096 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Expose CT target rules for conntrack helper" alias="expose-l3-conntrack-helper" description="Expose allow adding CT target rules for conntrack helper" updated="2019-04-04T10:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.558228 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Extension expose-l3-conntrack-helper not supported by any of loaded plugins
May 09 14:04:13.558298 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: expose_port_forwarding_in_fip.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.558490 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Expose Floating IP port forwardings" alias="expose-port-forwarding-in-fip" description="Expose port forwarding fields in floating IP response" updated="2018-06-28T16:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.558621 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Extension expose-port-forwarding-in-fip not supported by any of loaded plugins
May 09 14:04:13.558691 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: external_net.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.558865 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Neutron external network" alias="external-net" description="Adds external network attribute to network resource." updated="2013-01-14T10:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.558995 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loaded extension: external-net
May 09 14:04:13.559049 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: extra_dhcp_opt.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.559233 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Neutron Extra DHCP options" alias="extra_dhcp_opt" description="Extra options configuration for DHCP. For example PXE boot options to DHCP clients can be specified (e.g. tftp-server, server-ip-address, bootfile-name)" updated="2013-03-17T12:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.559364 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loaded extension: extra_dhcp_opt
May 09 14:04:13.559436 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: extraroute.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.559618 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Neutron Extra Route" alias="extraroute" description="Extra routes configuration for L3 router" updated="2013-02-01T10:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.559753 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loaded extension: extraroute
May 09 14:04:13.559815 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: extraroute_atomic.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.559998 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Atomically add/remove extra routes" alias="extraroute-atomic" description="Edit extra routes of a router on server side by atomically adding/removing extra routes" updated="2019-07-10T00:00:00+00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.560130 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loaded extension: extraroute-atomic
May 09 14:04:13.560198 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: filter_validation.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.560382 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Filter parameters validation" alias="filter-validation" description="Provides validation on filter parameters." updated="2018-07-04T10:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.560532 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loaded extension: filter-validation
May 09 14:04:13.560601 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: fip_pf_description.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.560778 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Floating IP Port Forwarding new attribute description" alias="floating-ip-port-forwarding-description" description="Add a description field to Port Forwarding rules" updated="2019-11-01T10:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.560905 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Extension floating-ip-port-forwarding-description not supported by any of loaded plugins
May 09 14:04:13.560972 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: fip_pf_detail.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.561143 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Floating IP port forwarding detail" alias="floating-ip-port-forwarding-detail" description="Allow list floating ip return more port forwarding data, include'id' and 'internal_port_id'" updated="2022-05-06T10:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.561273 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loaded extension: floating-ip-port-forwarding-detail
May 09 14:04:13.561339 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: fip_pf_port_range.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.561538 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Floating IP Port Forwarding support port ranges" alias="floating-ip-port-forwarding-port-ranges" description="Add support to port ranges" updated="2020-07-01T10:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.561656 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Extension floating-ip-port-forwarding-port-ranges not supported by any of loaded plugins
May 09 14:04:13.561752 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: fip_port_details.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.561929 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Floating IP Port Details Extension" alias="fip-port-details" description="Add port_details attribute to Floating IP resource" updated="2018-04-09T10:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.562057 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loaded extension: fip-port-details
May 09 14:04:13.562111 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: flavors.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.562426 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Neutron Service Flavors" alias="flavors" description="Flavor specification for Neutron advanced services." updated="2015-09-17T10:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.562581 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loaded extension: flavors
May 09 14:04:13.562656 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: floating_ip_port_forwarding.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.562903 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Floating IP Port Forwarding" alias="floating-ip-port-forwarding" description="Allow port forwarding from floating IP port to an internal port" updated="2018-01-19T10:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.563037 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Extension floating-ip-port-forwarding not supported by any of loaded plugins
May 09 14:04:13.563088 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: floatingip_pools.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.563308 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Floating IP Pools Extension" alias="floatingip-pools" description="Provides a floating IP pools API." updated="2018-03-21T10:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.563453 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loaded extension: floatingip-pools
May 09 14:04:13.563523 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: ip_allocation.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.563707 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="IP Allocation" alias="ip_allocation" description="IP allocation extension." updated="2016-06-10T23:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.563834 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Extension ip_allocation not supported by any of loaded plugins
May 09 14:04:13.563909 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: ip_substring_port_filtering.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.564074 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="IP address substring filtering" alias="ip-substring-filtering" description="Provides IP address substring filtering when listing ports" updated="2017-11-28T09:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.564206 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Extension ip-substring-filtering not supported by any of loaded plugins
May 09 14:04:13.564272 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: l2_adjacency.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.564461 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="L2 Adjacency" alias="l2_adjacency" description="Display L2 Adjacency for Neutron Networks." updated="2016-04-12T16:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.564586 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Extension l2_adjacency not supported by any of loaded plugins
May 09 14:04:13.564654 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: l3.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.564976 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Neutron L3 Router" alias="router" description="Router abstraction for basic L3 forwarding between L2 Neutron networks and access to external networks via a NAT gateway." updated="2012-07-20T10:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.565115 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loaded extension: router
May 09 14:04:13.565180 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: l3_conntrack_helper.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.565657 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="L3 Conntrack helper" alias="l3-conntrack-helper" description="Allow adding CT target rules for conntrack helper" updated="2019-04-04T10:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.565822 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Extension l3-conntrack-helper not supported by any of loaded plugins
May 09 14:04:13.565896 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: l3_enable_default_route_bfd.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.566084 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Enable Default Route BFD" alias="enable-default-route-bfd" description="Enables configurable BFD behavior for default routes" updated="2023-02-27T00:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.566218 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loaded extension: enable-default-route-bfd
May 09 14:04:13.566289 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: l3_enable_default_route_ecmp.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.566488 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Enable Default Route ECMP" alias="enable-default-route-ecmp" description="Enables configurable ECMP behavior for default routes" updated="2023-02-27T00:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.566615 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loaded extension: enable-default-route-ecmp
May 09 14:04:13.566682 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: l3_ext_gw_mode.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.566863 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Neutron L3 Configurable external gateway mode" alias="ext-gw-mode" description="Extension of the router abstraction for specifying whether SNAT should occur on the external gateway" updated="2013-03-28T10:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.566987 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loaded extension: ext-gw-mode
May 09 14:04:13.567054 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: l3_ext_ha_mode.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.567228 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="HA Router extension" alias="l3-ha" description="Adds HA capability to routers." updated="2014-04-26T00:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.567359 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loaded extension: l3-ha
May 09 14:04:13.567446 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: l3_ext_ndp_proxy.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.567619 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Router support to enable NDP proxy" alias="l3-ext-ndp-proxy" description="Router extension to enable ndp proxy" updated="2020-08-25T00:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.567750 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Extension l3-ext-ndp-proxy not supported by any of loaded plugins
May 09 14:04:13.567819 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: l3_extra_gws.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.567993 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Neutron L3 External Gateway Multihoming" alias="external-gateway-multihoming" description="Allow multiple external gateway ports per router" updated="2023-01-18T00:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.568124 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loaded extension: external-gateway-multihoming
May 09 14:04:13.568192 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: l3_flavors.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.568364 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Router Flavor Extension" alias="l3-flavors" description="Flavor support for routers." updated="2016-05-17T00:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.568514 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loaded extension: l3-flavors
May 09 14:04:13.568582 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: l3_ndp_proxy.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.568814 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="name" alias="l3-ndp-proxy" description="description" updated="2021-06-29T10:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.568941 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Extension l3-ndp-proxy not supported by any of loaded plugins
May 09 14:04:13.569007 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: l3_port_ip_change_not_allowed.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.569180 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Prevent L3 router ports IP address change extension" alias="l3-port-ip-change-not-allowed" description="Prevent change of IP address for some L3 router ports" updated="2018-10-09T10:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.569309 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Extension l3-port-ip-change-not-allowed not supported by any of loaded plugins
May 09 14:04:13.569374 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: l3agentscheduler.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.569816 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="L3 Agent Scheduler" alias="l3_agent_scheduler" description="Schedule routers among l3 agents" updated="2013-02-07T10:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.569952 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Extension l3_agent_scheduler not supported by any of loaded plugins
May 09 14:04:13.570015 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: local_ip.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.570264 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Local IP" alias="local_ip" description="Support Node Local Virtual IP" updated="2021-07-26T10:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.570398 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Extension local_ip not supported by any of loaded plugins
May 09 14:04:13.570479 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: logging.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.570713 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Logging API Extension" alias="logging" description="Provides a logging API for resources such as security group and firewall." updated="2017-01-01T10:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.570844 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Extension logging not supported by any of loaded plugins
May 09 14:04:13.570916 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: metering.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.571269 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Neutron Metering" alias="metering" description="Neutron Metering extension." updated="2013-06-12T10:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.571414 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Extension metering not supported by any of loaded plugins
May 09 14:04:13.571533 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: metering_source_and_destination_fields.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.571724 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Neutron Metering with source and destination filtering" alias="metering_source_and_destination_fields" description="Neutron Metering extension that enables the use of source and destination IP prefixes to create metering label rules." updated="2020-08-21T08:12:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.571862 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Extension metering_source_and_destination_fields not supported by any of loaded plugins
May 09 14:04:13.571932 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: multiprovidernet.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.572112 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Multi Provider Network" alias="multi-provider" description="Expose mapping of virtual networks to multiple physical networks" updated="2013-06-27T10:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.572251 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loaded extension: multi-provider
May 09 14:04:13.572304 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: netmtu.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.572495 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Network MTU" alias="net-mtu" description="Provides MTU attribute for a network resource." updated="2015-03-25T10:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.572626 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loaded extension: net-mtu
May 09 14:04:13.572693 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: netmtu_writable.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.572867 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Network MTU (writable)" alias="net-mtu-writable" description="Provides a writable MTU attribute for a network resource." updated="2017-07-12T00:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.573001 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loaded extension: net-mtu-writable
May 09 14:04:13.573069 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: network_availability_zone.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.573262 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Network Availability Zone" alias="network_availability_zone" description="Availability zone support for network." updated="2015-01-01T10:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.573396 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loaded extension: network_availability_zone
May 09 14:04:13.573483 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: network_ha.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.573673 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Network HA creation flag" alias="network_ha" description="Network high availability creation flag." updated="2023-04-27T10:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.573819 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Extension network_ha not supported by any of loaded plugins
May 09 14:04:13.573889 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: network_ip_availability.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.574078 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Network IP Availability" alias="network-ip-availability" description="Provides IP availability data for each network and subnet." updated="2015-09-24T00:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.574207 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loaded extension: network-ip-availability
May 09 14:04:13.574275 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: network_ip_availability_details.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.574489 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Network IP Availability Details Extension" alias="network-ip-availability-details" description="Add ip_availability_details attribute to network-ip-availabilities resource." updated="2025-10-31T00:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.574620 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loaded extension: network-ip-availability-details
May 09 14:04:13.574688 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: network_segment_range.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.574936 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Neutron Network Segment Range" alias="network-segment-range" description="Provides support for the network segment range management" updated="2018-11-29T00:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.575071 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Extension network-segment-range not supported by any of loaded plugins
May 09 14:04:13.575137 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: pagination.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.575318 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Pagination support" alias="pagination" description="Extension that indicates that pagination is enabled." updated="2016-06-12T00:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.575387 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loaded extension: pagination
May 09 14:04:13.575471 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: port_device_profile.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.575650 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Port device profile" alias="port-device-profile" description="Expose the port device profile (Cyborg)" updated="2020-12-17T10:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.575779 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loaded extension: port-device-profile
May 09 14:04:13.575846 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: port_hardware_offload_type.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.576024 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Port hardware offload" alias="port-hardware-offload-type" description="Expose port hardware offload extension" updated="2023-05-09T10:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.576150 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Extension port-hardware-offload-type not supported by any of loaded plugins
May 09 14:04:13.576219 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: port_hint_ovs_tx_steering.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.576390 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Port hint: Open vSwitch Tx steering" alias="port-hint-ovs-tx-steering" description="Enable Port hint: Open vSwitch Tx steering" updated="2023-01-01T00:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.576537 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Extension port-hint-ovs-tx-steering not supported by any of loaded plugins
May 09 14:04:13.576605 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: port_hints.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.576777 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Port hints" alias="port-hints" description="Backend specific port hints to allow backend specific tuning" updated="2023-01-01T00:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.576911 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Extension port-hints not supported by any of loaded plugins
May 09 14:04:13.576980 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: port_mac_address_override.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.577150 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Neutron Port MAC address override" alias="port-mac-override" description="Allow overriding the MAC address of a direct-physical Port via the active binding profile" updated="2022-03-18T10:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.577279 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Extension port-mac-override not supported by any of loaded plugins
May 09 14:04:13.577346 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: port_mac_address_regenerate.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.577531 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Neutron Port MAC address regenerate" alias="port-mac-address-regenerate" description="Network port MAC address regenerate" updated="2018-05-03T10:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.577684 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loaded extension: port-mac-address-regenerate
May 09 14:04:13.577752 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: port_numa_affinity_policy.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.577924 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Port NUMA affinity policy" alias="port-numa-affinity-policy" description="Expose the port NUMA affinity policy" updated="2020-07-08T10:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.578058 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loaded extension: port-numa-affinity-policy
May 09 14:04:13.578125 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: port_numa_affinity_policy_socket.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.578293 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Port NUMA affinity policy "socket"" alias="port-numa-affinity-policy-socket" description="Adds "socket" to the supported port NUMA affinity policies" updated="2024-02-09T10:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.578441 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loaded extension: port-numa-affinity-policy-socket
May 09 14:04:13.578511 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: port_resource_request.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.578680 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Port Resource Request" alias="port-resource-request" description="Expose resource request to Port" updated="2018-05-08T10:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.578814 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Extension port-resource-request not supported by any of loaded plugins
May 09 14:04:13.578883 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: port_resource_request_groups.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.579081 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Port Resource Request Groups" alias="port-resource-request-groups" description="Support requesting multiple groups of resources and traits from the same RP subtree in resource_request" updated="2021-08-02T10:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.579195 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Extension port-resource-request-groups not supported by any of loaded plugins
May 09 14:04:13.579261 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: port_trusted_vif.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.579447 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Port trusted vif" alias="port-trusted-vif" description="Expose port 'trusted' attribute in the API" updated="2024-07-10T10:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.579578 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Extension port-trusted-vif not supported by any of loaded plugins
May 09 14:04:13.579645 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: portbindings.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.579820 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Port Binding" alias="binding" description="Expose port bindings of a virtual port to external application" updated="2014-02-03T10:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.579950 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loaded extension: binding
May 09 14:04:13.580019 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: portbindings_extended.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.580215 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Port Bindings Extended" alias="binding-extended" description="Expose port bindings of a virtual port to external application" updated="2017-07-17T10:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.580344 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loaded extension: binding-extended
May 09 14:04:13.580426 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: portsecurity.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.580598 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Port Security" alias="port-security" description="Provides port security" updated="2012-07-23T10:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.580729 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loaded extension: port-security
May 09 14:04:13.580797 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: project_id.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.580970 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="project_id field enabled" alias="project-id" description="Extension that indicates that project_id field is enabled." updated="2016-09-09T09:09:09-09:09" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.581043 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loaded extension: project-id
May 09 14:04:13.581109 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: providernet.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.581276 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Provider Network" alias="provider" description="Expose mapping of virtual networks to physical networks" updated="2012-09-07T10:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.581415 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loaded extension: provider
May 09 14:04:13.581491 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: qinq.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.581694 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="QinQ networks" alias="qinq" description="Expose network 'qinq' attribute in the API. This allows to configure network to allow vlan in vlan configuration using 0x8a88 ethertype (QinQ)." updated="2024-12-02T10:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.581826 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loaded extension: qinq
May 09 14:04:13.581901 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: qos.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.582561 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Quality of Service" alias="qos" description="The Quality of Service extension." updated="2015-06-08T10:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.582707 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Extension qos not supported by any of loaded plugins
May 09 14:04:13.582887 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: qos_bw_limit_direction.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.582983 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Direction for QoS bandwidth limit rule" alias="qos-bw-limit-direction" description="Allow to configure QoS bandwidth limit rule with specific direction: ingress or egress" updated="2017-04-10T10:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.583115 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Extension qos-bw-limit-direction not supported by any of loaded plugins
May 09 14:04:13.583184 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: qos_bw_minimum_ingress.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.583362 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Ingress direction for QoS minimum bandwidth rule" alias="qos-bw-minimum-ingress" description="Allow to configure QoS minumum bandwidth rule with ingress direction." updated="2018-07-09T10:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.583520 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Extension qos-bw-minimum-ingress not supported by any of loaded plugins
May 09 14:04:13.583583 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: qos_default.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.583756 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="QoS default policy" alias="qos-default" description="Expose the QoS default policy per project" updated="2017-041-06T10:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.583889 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Extension qos-default not supported by any of loaded plugins
May 09 14:04:13.583957 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: qos_fip.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.584131 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Floating IP QoS" alias="qos-fip" description="The floating IP Quality of Service extension" updated="2017-07-20T00:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.584259 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Extension qos-fip not supported by any of loaded plugins
May 09 14:04:13.584326 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: qos_fip_network_policy.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.584517 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="QoS floating IP network policy ID" alias="qos-fip-network-policy" description="Adds a the QoS network ID to the floating IP definition" updated="2021-11-15T10:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.584645 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Extension qos-fip-network-policy not supported by any of loaded plugins
May 09 14:04:13.584714 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: qos_gateway_ip.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.584886 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Router gateway IP QoS" alias="qos-gateway-ip" description="The Router gateway IP Quality of Service extension" updated="2018-02-24T00:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.585016 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Extension qos-gateway-ip not supported by any of loaded plugins
May 09 14:04:13.585083 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: qos_port_network_policy.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.585255 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="QoS port network policy ID" alias="qos-port-network-policy" description="Adds a the QoS network ID to the port definition" updated="2019-11-01T10:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.585382 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Extension qos-port-network-policy not supported by any of loaded plugins
May 09 14:04:13.585467 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: qos_pps_minimum_rule.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.585683 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="QoS minimum packet rate rule" alias="qos-pps-minimum" description="Allow to configure QoS minimum packet rate rule." updated="2021-07-14T10:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.585811 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Extension qos-pps-minimum not supported by any of loaded plugins
May 09 14:04:13.585880 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: qos_pps_minimum_rule_alias.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.586061 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="QoS minimum packet rate rule alias" alias="qos-pps-minimum-rule-alias" description="API to enable GET, PUT and DELETE operations on QoS minimum packet rate rule without specifying policy ID" updated="2021-10-22T10:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.586192 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Extension qos-pps-minimum-rule-alias not supported by any of loaded plugins
May 09 14:04:13.586258 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: qos_pps_rule.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.586462 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="QoS Rule Type Packet per Second Extension" alias="qos-pps" description="Add QoS Rule Type Packet per Second" updated="2021-05-12T10:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.586592 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Extension qos-pps not supported by any of loaded plugins
May 09 14:04:13.586660 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: qos_rule_type_details.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.586831 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Details of QoS rule types" alias="qos-rule-type-details" description="Expose details about QoS rule types supported by loaded backend drivers" updated="2017-06-22T10:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.586962 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Extension qos-rule-type-details not supported by any of loaded plugins
May 09 14:04:13.587030 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: qos_rule_type_filter.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.587202 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Allow to filter the list of QoS rule types" alias="qos-rule-type-filter" description="Allows to filter the QoS rule type list adding two new flags. "all_rules" prints all implemented QoS rule types."all_supported" prints all supported QoS rule types by the loaded mechanism drivers" updated="2022-02-02T10:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.587334 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Extension qos-rule-type-filter not supported by any of loaded plugins
May 09 14:04:13.587429 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: qos_rules_alias.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.587598 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Quality of Service rules alias API" alias="qos-rules-alias" description="API to enable GET, PUT and DELETE operations on QoS policy rules without specifying policy ID" updated="2018-10-07T10:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.587727 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Extension qos-rules-alias not supported by any of loaded plugins
May 09 14:04:13.587795 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: quota_check_limit.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.587967 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Quota engine limit check" alias="quota-check-limit" description="Support for checking the resource usage before applying a new quota limit" updated="2021-09-08T16:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.588097 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loaded extension: quota-check-limit
May 09 14:04:13.588165 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: quota_check_limit_default.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.588354 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Quota engine limit check by default" alias="quota-check-limit-default" description="By default, the Neutron quota engine checks the resource usage before applying a new quota limit" updated="2024-08-21T16:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.588507 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loaded extension: quota-check-limit-default
May 09 14:04:13.588575 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: quotasv2.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.588919 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Quota management support" alias="quotas" description="Expose functions for quotas management per project" updated="2012-07-29T10:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.589123 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loaded extension: quotas
May 09 14:04:13.589200 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: quotasv2_detail.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.589473 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Quota details management support" alias="quota_details" description="Expose functions for quotas usage statistics per project" updated="2017-02-10T10:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.589551 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loaded extension: quota_details
May 09 14:04:13.589684 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: rbac.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.589994 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="RBAC Policies" alias="rbac-policies" description="Allows creation and modification of policies that control project access to resources." updated="2015-06-17T12:15:12-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.590138 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loaded extension: rbac-policies
May 09 14:04:13.590210 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: rbac_address_group.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.590394 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Add address_group type to RBAC" alias="rbac-address-group" description="Add address_group type to network RBAC" updated="2021-01-20T00:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.590550 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Extension rbac-address-group not supported by any of loaded plugins
May 09 14:04:13.590613 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: rbac_address_scope.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.590796 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Add address_scope type to RBAC" alias="rbac-address-scope" description="Add address_scope type to RBAC" updated="2020-02-12T00:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.590926 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loaded extension: rbac-address-scope
May 09 14:04:13.590993 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: rbac_security_groups.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.591167 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Add security_group type to network RBAC" alias="rbac-security-groups" description="Add security_group type to network RBAC" updated="2019-02-14T00:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.591301 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loaded extension: rbac-security-groups
May 09 14:04:13.591372 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: rbac_subnetpool.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.591563 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Add subnetpool type to RBAC" alias="rbac-subnetpool" description="Add subnetpool type to RBAC" updated="2020-02-05T00:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.591698 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Extension rbac-subnetpool not supported by any of loaded plugins
May 09 14:04:13.591767 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: revisionifmatch.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.591939 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="If-Match constraints based on revision_number" alias="revision-if-match" description="Extension indicating that If-Match based on revision_number is supported." updated="2016-12-11T00:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.592072 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loaded extension: revision-if-match
May 09 14:04:13.592142 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: revisions.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.592327 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Resource revision numbers" alias="standard-attr-revisions" description="This extension will display the revision number of neutron resources." updated="2016-04-11T10:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.592475 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loaded extension: standard-attr-revisions
May 09 14:04:13.592543 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: router_availability_zone.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.592738 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Router Availability Zone" alias="router_availability_zone" description="Availability zone support for router." updated="2015-01-01T10:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.592875 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loaded extension: router_availability_zone
May 09 14:04:13.592943 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: router_enable_snat.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.593118 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Exposes the router "enable_snat" read-only configuration flag" alias="router-enable-snat" description="Exposes the router "enable_snat" read-only configuration flag" updated="2025-09-18T18:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.593251 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loaded extension: router-enable-snat
May 09 14:04:13.593317 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: routerservicetype.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.593506 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Router Service Type" alias="router-service-type" description="Provides router service type" updated="2013-01-29T00:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.593658 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Extension router-service-type not supported by any of loaded plugins
May 09 14:04:13.593721 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: security_groups_default_rules.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.594046 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Default rules for security groups" alias="security-groups-default-rules" description="Configure set of security group rules used as default rules for every new security group" updated="2022-12-19T10:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.594183 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loaded extension: security-groups-default-rules
May 09 14:04:13.594258 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: security_groups_normalized_cidr.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.594448 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Normalized CIDR field for security group rules" alias="security-groups-normalized-cidr" description="Add new field with normalized remote_ip_prefix cidr in SG rule" updated="2020-07-28T10:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.594585 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loaded extension: security-groups-normalized-cidr
May 09 14:04:13.594648 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: security_groups_port_filtering.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.594829 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Port filtering on security groups" alias="port-security-groups-filtering" description="Provides security groups filtering when listing ports" updated="2018-01-09T09:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.594955 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Extension port-security-groups-filtering not supported by any of loaded plugins
May 09 14:04:13.595024 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: security_groups_remote_address_group.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.595196 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Remote address group id field for security group rules" alias="security-groups-remote-address-group" description="Add new field of remote address group id in SG rules" updated="2020-08-25T10:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.595329 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loaded extension: security-groups-remote-address-group
May 09 14:04:13.595395 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: security_groups_rules_belongs_to_default_sg.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.595583 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Security group rule belongs to the project's default security group" alias="security-groups-rules-belongs-to-default-sg" description="Flag to determine if the security group rule belongs to the project's default security group" updated="2023-05-23T10:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.595712 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loaded extension: security-groups-rules-belongs-to-default-sg
May 09 14:04:13.595786 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: security_groups_shared_filtering.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.595958 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Security group filtering on the shared field" alias="security-groups-shared-filtering" description="Support filtering security groups on the shared field" updated="2021-10-05T09:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.596088 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loaded extension: security-groups-shared-filtering
May 09 14:04:13.596156 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: securitygroup.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.597480 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="security-group" alias="security-group" description="The security groups extension." updated="2012-10-05T10:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.597589 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loaded extension: security-group
May 09 14:04:13.597691 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: segment.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.597936 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Segment" alias="segment" description="Segments extension." updated="2016-02-24T17:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.598076 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Extension segment not supported by any of loaded plugins
May 09 14:04:13.598141 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: segments_peer_subnet_host_routes.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.598328 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Segments peer-subnet host routes" alias="segments-peer-subnet-host-routes" description="Add host routes to subnets on a routed network (segments)" updated="2018-06-12T10:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.598471 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Extension segments-peer-subnet-host-routes not supported by any of loaded plugins
May 09 14:04:13.598540 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: servicetype.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.598727 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Neutron Service Type Management" alias="service-type" description="API for retrieving service providers for Neutron advanced services" updated="2013-01-20T00:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.598863 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loaded extension: service-type
May 09 14:04:13.598931 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: sorting.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.599114 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Sorting support" alias="sorting" description="Extension that indicates that sorting is enabled." updated="2016-06-12T00:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.599186 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loaded extension: sorting
May 09 14:04:13.599254 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: standard_attr_segment.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.599441 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Standard Attribute Segment Extension" alias="standard-attr-segment" description="Add standard attributes to Segment resource" updated="2018-04-09T10:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.599574 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Extension standard-attr-segment not supported by any of loaded plugins
May 09 14:04:13.599641 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: standardattrdescription.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.599832 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="standard-attr-description" alias="standard-attr-description" description="Extension to add descriptions to standard attributes" updated="2016-02-10T10:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.599965 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loaded extension: standard-attr-description
May 09 14:04:13.600156 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: stateful_security_group.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.600342 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Stateful security group" alias="stateful-security-group" description="Indicates if the security group is stateful or not" updated="2019-11-26T09:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.600491 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loaded extension: stateful-security-group
May 09 14:04:13.600559 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: stdattrs_common.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.600714 np0000173183 neutron-ovn-maintenance-worker[104836]: WARNING neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Did not find expected name "Stdattrs_common" in /opt/stack/neutron/neutron/extensions/stdattrs_common.py
May 09 14:04:13.600784 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: subnet_dns_publish_fixed_ip.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.600957 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Subnet DNS publish fixed IP" alias="subnet-dns-publish-fixed-ip" description="Support choosing to publish DNS records for IPs from a subnet" updated="2019-05-24T10:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.601093 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loaded extension: subnet-dns-publish-fixed-ip
May 09 14:04:13.601162 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: subnet_external_network.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.601338 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Subnet belongs to an external network" alias="subnet-external-network" description="Informs if the subnet belongs to an external network" updated="2024-02-05T18:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.601485 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loaded extension: subnet-external-network
May 09 14:04:13.601553 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: subnet_onboard.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.601747 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Subnet Onboard" alias="subnet_onboard" description="Provides support for onboarding subnets into subnet pools" updated="2018-12-18T09:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.601984 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Extension subnet_onboard not supported by any of loaded plugins
May 09 14:04:13.602061 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: subnet_segmentid_writable.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.602239 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Subnet SegmentID (writable)" alias="subnet-segmentid-writable" description="Provides a writable segment_id attribute for a subnet resource." updated="2018-03-12T00:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.602376 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Extension subnet-segmentid-writable not supported by any of loaded plugins
May 09 14:04:13.602462 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: subnet_service_types.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.602638 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Subnet service types" alias="subnet-service-types" description="Provides ability to set the subnet service_types field" updated="2016-03-15T18:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.602772 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loaded extension: subnet-service-types
May 09 14:04:13.602841 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: subnetallocation.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.603027 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Subnet Allocation" alias="subnet_allocation" description="Enables allocation of subnets from a subnet pool" updated="2015-03-30T10:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.603160 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loaded extension: subnet_allocation
May 09 14:04:13.603229 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: subnetpool_prefix_ops.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.603453 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Subnet Pool Prefix Operations" alias="subnetpool-prefix-ops" description="Provides support for adjusting the prefix list of subnet pools" updated="2019-02-08T10:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.603577 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loaded extension: subnetpool-prefix-ops
May 09 14:04:13.603646 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: tag_creation.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.603821 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Tag creation extension" alias="tag-creation" description="Allow to create multiple tags for a resource" updated="2024-23-07T10:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.603960 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loaded extension: tag-creation
May 09 14:04:13.604028 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: tag_ports_during_bulk_creation.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.604202 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Tag Ports During Bulk Creation" alias="tag-ports-during-bulk-creation" description="Allow to tag ports during bulk creation" updated="2019-12-29T19:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.604352 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Extension tag-ports-during-bulk-creation not supported by any of loaded plugins
May 09 14:04:13.604430 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: tagging.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.604955 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Tag support for resources with standard attribute: port, subnet, subnetpool, network, security_group, router, floatingip, policy, trunk, network_segment_range" alias="standard-attr-tag" description="Enables to set tag on resources with standard attribute." updated="2017-01-01T00:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.605090 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loaded extension: standard-attr-tag
May 09 14:04:13.605146 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: timestamp.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.605352 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Resource timestamps" alias="standard-attr-timestamp" description="Adds created_at and updated_at fields to all Neutron resources that have Neutron standard attributes." updated="2016-09-12T10:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.605503 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loaded extension: standard-attr-timestamp
May 09 14:04:13.605567 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: trunk.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.605770 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Trunk Extension" alias="trunk" description="Provides support for trunk ports" updated="2016-01-01T10:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.605915 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Extension trunk not supported by any of loaded plugins
May 09 14:04:13.605987 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: trunk_details.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.606160 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Trunk port details" alias="trunk-details" description="Expose trunk port details" updated="2016-01-01T10:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.606285 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Extension trunk-details not supported by any of loaded plugins
May 09 14:04:13.606349 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: uplink_status_propagation.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.606545 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Uplink status propagation extension" alias="uplink-status-propagation" description="Provides ability to propagate port uplink status." updated="2018-05-31T18:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.606675 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Extension uplink-status-propagation not supported by any of loaded plugins
May 09 14:04:13.606743 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: uplink_status_propagation_updatable.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.606922 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Uplink status propagation updatable extension" alias="uplink-status-propagation-updatable" description="Provides ability to update the port uplink status." updated="2024-09-03T18:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.607052 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Extension uplink-status-propagation-updatable not supported by any of loaded plugins
May 09 14:04:13.607121 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loading extension file: vlantransparent.py {{(pid=104836) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 14:04:13.607306 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Ext name="Vlantransparent" alias="vlan-transparent" description="Provides Vlan Transparent Networks" updated="2015-03-23T09:00:00-00:00" {{(pid=104836) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 14:04:13.607459 np0000173183 neutron-ovn-maintenance-worker[104836]: INFO neutron.api.extensions [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Loaded extension: vlan-transparent
May 09 14:04:13.609572 np0000173183 neutron-ovn-maintenance-worker[104836]: WARNING oslo_service.backend._threading.service [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Service MaintenanceWorker is not picklable with spawn; falling back to fork. Please make the service spawn-safe to avoid this fallback.
May 09 14:04:13.609882 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Full set of CONF: {{(pid=104836) _load_service_manager_options /opt/stack/data/venv/lib/python3.12/site-packages/cotyledon/oslo_config_glue.py:59}}
May 09 14:04:13.609950 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] ******************************************************************************** {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2979}}
May 09 14:04:13.610008 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Configuration options gathered from: {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2980}}
May 09 14:04:13.610080 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] command line args: ['--config-file', '/etc/neutron/neutron.conf', '--config-file', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2981}}
May 09 14:04:13.610177 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] config files: ['/etc/neutron/neutron.conf', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2982}}
May 09 14:04:13.610346 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] ================================================================================ {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2987}}
May 09 14:04:13.610346 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] agent_down_time                = 75 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.610437 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] allow_automatic_dhcp_failover  = True {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.610524 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] allow_bulk                     = True {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.610676 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] allowed_conntrack_helpers      = [{'amanda': 'tcp'}, {'ftp': 'tcp'}, {'h323': 'udp'}, {'h323': 'tcp'}, {'irc': 'tcp'}, {'netbios-ns': 'udp'}, {'pptp': 'tcp'}, {'sane': 'tcp'}, {'sip': 'udp'}, {'sip': 'tcp'}, {'snmp': 'udp'}, {'tftp': 'udp'}] {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.610759 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] api_extensions_path            =  {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.610845 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] api_paste_config               = api-paste.ini {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.610937 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] api_workers                    = 4 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.611023 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] auth_strategy                  = keystone {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.611108 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] base_mac                       = fa:16:3e:00:00:00 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.611207 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] bind_host                      = 0.0.0.0 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.611294 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] bind_port                      = 9696 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.611378 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] client_socket_timeout          = 900 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.611506 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] config_dir                     = [] {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.611612 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] config_file                    = ['/etc/neutron/neutron.conf', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.611612 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] config_source                  = [] {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.611705 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] control_exchange               = neutron {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.611797 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] core_plugin                    = ml2 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.611884 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] debug                          = True {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.611967 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] default_availability_zones     = [] {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.612115 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] default_log_levels             = ['amqp=WARN', 'boto=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'OFPHandler=INFO', 'OfctlService=INFO', 'os_ken.base.app_manager=INFO', 'os_ken.controller.controller=INFO'] {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.612208 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] dhcp_agent_notification        = False {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.612266 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] dhcp_agents_per_network        = 1 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.612352 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] dhcp_lease_duration            = 86400 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.612452 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] dhcp_load_type                 = networks {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.612539 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] dns_domain                     = openstacklocal {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.612624 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] dvr_base_mac                   = fa:16:3f:00:00:00 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.612708 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] enable_default_route_bfd       = False {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.612809 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] enable_default_route_ecmp      = False {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.612877 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] enable_dvr                     = True {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.612958 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] enable_new_agents              = True {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.613039 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] enable_services_on_agents_with_admin_state_down = False {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.613120 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] enable_snat_by_default         = True {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.613202 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] enable_traditional_dhcp        = True {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.613280 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] external_dns_driver            = None {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.613360 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] filter_validation              = True {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.613461 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] global_physnet_mtu             = 1430 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.613525 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] graceful_shutdown_timeout      = 60 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.613654 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] host                           = np0000173183 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.613743 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] host_dvr_for_dhcp              = True {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.613824 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] http_retries                   = 3 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.613952 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] instance_format                = [instance: %(uuid)s]  {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.614003 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] instance_uuid_format           = [instance: %(uuid)s]  {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.614091 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] ipam_driver                    = internal {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.614145 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] ipv6_pd_enabled                = False {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.614240 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] l3_ha                          = False {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.614323 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] l3_ha_net_cidr                 = 169.254.192.0/18 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.614418 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] l3_ha_network_physical_name    =  {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.614513 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] l3_ha_network_type             =  {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.614602 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] log_color                      = False {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.614686 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] log_config_append              = None {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.614772 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] log_date_format                = %Y-%m-%d %H:%M:%S {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.614856 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] log_dir                        = None {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.614936 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] log_file                       = None {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.614996 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] log_options                    = True {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.615079 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] log_rotate_interval            = 1 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.615165 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] log_rotate_interval_type       = days {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.615248 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] log_rotation_type              = none {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.615298 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] logging_context_format_string  = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.615368 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] logging_debug_format_suffix    = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.615473 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] logging_default_format_string  = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.615559 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] logging_exception_prefix       = ERROR %(name)s %(instance)s {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.615629 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] logging_user_identity_format   = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.615700 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] max_allowed_address_pair       = 10 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.615781 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] max_dns_nameservers            = 5 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.615862 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] max_header_line                = 16384 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.615943 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] max_l3_agents_per_router       = 3 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.616024 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] max_logfile_count              = 30 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.616107 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] max_logfile_size_mb            = 200 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.616187 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] max_routes                     = 30 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.616270 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] max_subnet_host_routes         = 20 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.616353 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] my_ip                          = 199.204.45.156 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.616456 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] my_ipv6                        = 2604:e100:1:0:f816:3eff:fe51:d95a {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.616538 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] network_auto_schedule          = True {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.616621 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] network_link_prefix            = None {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.616702 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] network_scheduler_driver       = neutron.scheduler.dhcp_agent_scheduler.WeightScheduler {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.616787 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] notify_nova_on_port_data_changes = True {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.616873 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] notify_nova_on_port_status_changes = True {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.616959 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] pagination_max_limit           = -1 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.617040 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] periodic_fuzzy_delay           = 5 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.617121 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] periodic_interval              = 40 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.617202 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] publish_errors                 = False {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.617283 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] rate_limit_burst               = 0 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.617368 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] rate_limit_except_level        = CRITICAL {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.617466 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] rate_limit_interval            = 0 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.617547 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] router_distributed             = False {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.617648 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] rpc_resources_processing_step  = 20 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.617729 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] rpc_response_max_timeout       = 600 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.617813 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] rpc_response_timeout           = 60 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.617900 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] rpc_state_report_workers       = 0 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.617979 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] rpc_workers                    = None {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.618060 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] send_events_interval           = 2 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.618149 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] service_plugins                = ['ovn-router'] {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.618235 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] setproctitle                   = on {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.618315 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] shell_completion               = None {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.618397 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] state_path                     = /opt/stack/data/neutron {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.618504 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] syslog_log_facility            = LOG_USER {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.618586 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] tcp_keepidle                   = 600 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.618673 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] transport_url                  = **** {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.618754 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] use_journal                    = False {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.618835 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] use_json                       = False {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.618916 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] use_stderr                     = False {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.619000 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] use_syslog                     = False {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.619079 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] vlan_qinq                      = None {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.619158 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] vlan_transparent               = None {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.619239 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] wsgi_default_pool_size         = 100 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.619320 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] wsgi_keep_alive                = True {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.619418 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] wsgi_log_format                = %(client_ip)s "%(request_line)s" status: %(status_code)s  len: %(body_length)s time: %(wall_seconds).7f {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.619504 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] wsgi_server_debug              = False {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.619601 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] oslo_concurrency.disable_process_locking = False {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.619696 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] oslo_concurrency.lock_path     = /opt/stack/data/neutron/lock {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.619788 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] profiler.connection_string     = messaging:// {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.619880 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] profiler.enabled               = False {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.619967 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] profiler.es_doc_type           = notification {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.620052 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] profiler.es_scroll_size        = 10000 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.620138 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] profiler.es_scroll_time        = 2m {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.620222 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] profiler.filter_error_trace    = False {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.620310 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] profiler.hmac_keys             = **** {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.620396 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] profiler.sentinel_service_name = mymaster {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.620506 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] profiler.socket_timeout        = 0.1 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.620591 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] profiler.trace_requests        = False {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.620678 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] profiler.trace_sqlalchemy      = False {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.620766 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] profiler_jaeger.process_tags   = {} {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.620855 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] profiler_jaeger.service_name_prefix = None {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.620939 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] profiler_otlp.service_name_prefix = None {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.621028 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.621112 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] oslo_messaging_metrics.metrics_enabled = False {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.621198 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] oslo_messaging_metrics.metrics_process_name =  {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.621285 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.621369 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.621478 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] oslo_policy.enforce_new_defaults = True {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.621688 np0000173183 neutron-ovn-maintenance-worker[104836]: WARNING oslo_config.cfg [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Deprecated: Option "enforce_scope" from group "oslo_policy" is deprecated for removal (This configuration was added temporarily to facilitate a smooth transition to the new RBAC. OpenStack will always enforce scope checks. This configuration option is deprecated and will be removed in the 2025.2 cycle.).  Its value may be silently ignored in the future.
May 09 14:04:13.621774 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] oslo_policy.enforce_scope      = True {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.621858 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] oslo_policy.policy_default_rule = default {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.621950 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] oslo_policy.policy_dirs        = ['policy.d'] {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.622036 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] oslo_policy.policy_file        = /etc/neutron/policy.json {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.622124 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.622207 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.622288 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] oslo_policy.remote_ssl_client_crt_file = None {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.622368 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] oslo_policy.remote_ssl_client_key_file = None {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.622468 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.622557 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] oslo_policy.remote_timeout     = 60.0 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.622649 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.622737 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] service_providers.service_provider = [] {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.622828 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] nova.auth_section              = None {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.622918 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] nova.auth_type                 = password {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.623005 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] nova.auth_url                  = https://199.204.45.156/identity {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.623088 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] nova.cafile                    = None {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.623169 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] nova.certfile                  = None {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.623253 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] nova.collect_timing            = False {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.623335 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] nova.default_domain_id         = None {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.623431 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] nova.default_domain_name       = None {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.623514 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] nova.domain_id                 = None {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.623596 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] nova.domain_name               = None {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.623683 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] nova.endpoint_type             = public {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.623766 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] nova.insecure                  = False {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.623847 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] nova.keyfile                   = None {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.623933 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] nova.password                  = **** {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.624015 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] nova.project_domain_id         = None {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.624101 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] nova.project_domain_name       = Default {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.624189 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] nova.project_id                = None {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.624277 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] nova.project_name              = service {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.624364 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] nova.region_name               = RegionOne {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.624464 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] nova.split_loggers             = False {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.624545 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] nova.system_scope              = None {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.624627 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] nova.timeout                   = None {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.624708 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] nova.trust_id                  = None {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.624790 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] nova.user_domain_id            = None {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.624875 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] nova.user_domain_name          = Default {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.624958 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] nova.user_id                   = None {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.625046 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] nova.username                  = nova {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.625133 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] placement.auth_section         = None {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.625218 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] placement.auth_type            = password {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.625303 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] placement.cafile               = None {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.625382 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] placement.certfile             = None {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.625483 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] placement.collect_timing       = False {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.625587 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] placement.endpoint_type        = public {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.625670 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] placement.insecure             = False {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.625753 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] placement.keyfile              = None {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.625831 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] placement.region_name          = None {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.625913 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] placement.split_loggers        = False {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.625995 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] placement.timeout              = None {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.626081 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] ironic.auth_section            = None {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.626164 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] ironic.auth_type               = None {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.626244 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] ironic.cafile                  = None {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.626325 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] ironic.certfile                = None {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.626412 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] ironic.collect_timing          = False {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.626508 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] ironic.connect_retries         = None {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.626591 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] ironic.connect_retry_delay     = None {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.626675 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] ironic.enable_notifications    = False {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.626758 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] ironic.endpoint_override       = None {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.626839 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] ironic.insecure                = False {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.626921 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] ironic.interface               = None {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.627001 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] ironic.keyfile                 = None {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.627083 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] ironic.max_version             = None {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.627164 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] ironic.min_version             = None {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.627245 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] ironic.region_name             = None {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.627324 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] ironic.retriable_status_codes  = None {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.627420 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] ironic.service_name            = None {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.627504 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] ironic.service_type            = None {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.627586 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] ironic.split_loggers           = False {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.627666 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] ironic.status_code_retries     = None {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.627748 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] ironic.status_code_retry_delay = None {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.627830 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] ironic.timeout                 = None {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.627910 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] ironic.valid_interfaces        = None {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.627991 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] ironic.version                 = None {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.628077 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] cli_script.dry_run             = False {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.628170 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.628254 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.628339 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.628437 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.628522 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.628606 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.628690 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.628774 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.628861 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] oslo_messaging_rabbit.heartbeat_timeout_threshold = 60 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.628948 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] oslo_messaging_rabbit.hostname = np0000173183 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.629030 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] oslo_messaging_rabbit.kombu_compression = None {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.629117 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.629203 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.629289 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.629374 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] oslo_messaging_rabbit.kombu_reconnect_splay = 0.0 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.629477 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] oslo_messaging_rabbit.processname = neutron-ovn-maintenance-worker {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.629597 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.629675 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.629762 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.629846 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.629929 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.630011 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] oslo_messaging_rabbit.rabbit_quorum_max_memory_bytes = 0 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.630093 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] oslo_messaging_rabbit.rabbit_quorum_max_memory_length = 0 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.630177 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.630259 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.630343 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.630442 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] oslo_messaging_rabbit.rabbit_stream_fanout = False {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.630527 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.630609 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] oslo_messaging_rabbit.rabbit_transient_quorum_queue = False {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.630692 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.630776 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] oslo_messaging_rabbit.ssl      = False {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.630862 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] oslo_messaging_rabbit.ssl_ca_file =  {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.630947 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] oslo_messaging_rabbit.ssl_cert_file =  {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.631029 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.631114 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] oslo_messaging_rabbit.ssl_key_file =  {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.631200 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] oslo_messaging_rabbit.ssl_version =  {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.631283 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] oslo_messaging_rabbit.use_queue_manager = False {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.631370 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] oslo_messaging_notifications.driver = [] {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.631471 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] oslo_messaging_notifications.retry = -1 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.631563 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] oslo_messaging_notifications.topics = ['notifications'] {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.631644 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] oslo_messaging_notifications.transport_url = **** {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.631731 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] oslo_reports.file_event_handler = None {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.631814 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] oslo_reports.file_event_handler_interval = 1 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.631896 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] oslo_reports.log_dir           = None {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.632015 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.632066 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] QUOTAS.default_quota           = -1 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.632150 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] QUOTAS.quota_driver            = neutron.db.quota.driver_nolock.DbQuotaNoLockDriver {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.632234 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] QUOTAS.quota_floatingip        = 50 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.632317 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] QUOTAS.quota_network           = 100 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.632419 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] QUOTAS.quota_port              = 500 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.632500 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] QUOTAS.quota_rbac_policy       = 10 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.632582 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] QUOTAS.quota_router            = 10 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.632698 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] QUOTAS.quota_security_group    = 10 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.632744 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] QUOTAS.quota_security_group_rule = 100 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.632823 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] QUOTAS.quota_subnet            = 100 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.632996 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] QUOTAS.track_quota_usage       = True {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.632996 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] privsep.capabilities           = [21, 12, 1, 2, 19] {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.633083 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] privsep.group                  = None {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.633154 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] privsep.helper_command         = None {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.633237 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] privsep.log_daemon_traceback   = False {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.633324 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] privsep.logger_name            = oslo_privsep.daemon {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.633411 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] privsep.thread_pool_size       = 16 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.633508 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] privsep.user                   = None {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.633624 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] privsep_dhcp_release.capabilities = [21, 12] {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.633728 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] privsep_dhcp_release.group     = None {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.633784 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] privsep_dhcp_release.helper_command = None {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.633873 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] privsep_dhcp_release.log_daemon_traceback = False {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.633956 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] privsep_dhcp_release.logger_name = oslo_privsep.daemon {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.634039 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] privsep_dhcp_release.thread_pool_size = 16 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.634120 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] privsep_dhcp_release.user      = None {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.634208 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] privsep_ovs_vsctl.capabilities = [21, 12] {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.634288 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] privsep_ovs_vsctl.group        = None {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.634369 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] privsep_ovs_vsctl.helper_command = None {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.634466 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] privsep_ovs_vsctl.log_daemon_traceback = False {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.634552 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] privsep_ovs_vsctl.logger_name  = oslo_privsep.daemon {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.634635 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] privsep_ovs_vsctl.thread_pool_size = 16 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.634716 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] privsep_ovs_vsctl.user         = None {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.634804 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] privsep_namespace.capabilities = [21] {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.634886 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] privsep_namespace.group        = None {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.634965 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] privsep_namespace.helper_command = None {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.635048 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] privsep_namespace.log_daemon_traceback = False {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.635132 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] privsep_namespace.logger_name  = oslo_privsep.daemon {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.635214 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] privsep_namespace.thread_pool_size = 16 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.635295 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] privsep_namespace.user         = None {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.635380 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] privsep_conntrack.capabilities = [12] {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.635481 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] privsep_conntrack.group        = None {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.635562 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] privsep_conntrack.helper_command = None {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.635645 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] privsep_conntrack.log_daemon_traceback = False {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.635729 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] privsep_conntrack.logger_name  = oslo_privsep.daemon {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.635812 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] privsep_conntrack.thread_pool_size = 16 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.635896 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] privsep_conntrack.user         = None {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.635982 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] privsep_link.capabilities      = [12, 21] {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.636063 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] privsep_link.group             = None {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.636142 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] privsep_link.helper_command    = None {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.636225 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] privsep_link.log_daemon_traceback = False {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.636310 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] privsep_link.logger_name       = oslo_privsep.daemon {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.636393 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] privsep_link.thread_pool_size  = 16 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.636495 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] privsep_link.user              = None {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.636587 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] AGENT.root_helper              = sudo /opt/stack/data/venv/bin/neutron-rootwrap /etc/neutron/rootwrap.conf {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.636675 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] AGENT.root_helper_daemon       = sudo /opt/stack/data/venv/bin/neutron-rootwrap-daemon /etc/neutron/rootwrap.conf {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.636759 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] AGENT.use_helper_for_ns_read   = True {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.636852 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] SECURITYGROUP.enable_ipset     = True {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.636942 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] SECURITYGROUP.enable_security_group = True {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.637020 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] SECURITYGROUP.firewall_driver  = None {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.637105 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] SECURITYGROUP.permitted_ethertypes = [] {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.637191 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] experimental.ipv6_pd_enabled   = False {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.637288 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] ml2.extension_drivers          = ['port_security', 'qos'] {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.637371 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] ml2.external_network_type      = None {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.637478 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] ml2.mechanism_drivers          = ['ovn'] {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.637583 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] ml2.overlay_ip_version         = 4 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.637673 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] ml2.path_mtu                   = 0 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.637757 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] ml2.physical_network_mtus      = {} {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.637851 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] ml2.project_network_types      = ['geneve'] {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.637940 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] ml2.tunnelled_network_rp_name  = rp_tunnelled {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.638031 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] ml2.type_drivers               = ['local', 'flat', 'vlan', 'geneve'] {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.638124 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] ml2_type_flat.flat_networks    = ['public'] {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.638216 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] ml2_type_geneve.max_header_size = 38 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.638306 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] ml2_type_geneve.vni_ranges     = ['1:65536'] {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.638410 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] ml2_type_vlan.network_vlan_ranges = ['public'] {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.638505 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] network_log.burst_limit        = 25 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.638587 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] network_log.local_output_log_base = None {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.638671 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] network_log.rate_limit         = 100 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.638765 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] database.asyncio_connection    = **** {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.638840 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] database.asyncio_slave_connection = **** {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.638935 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] database.backend               = sqlalchemy {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.639020 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] database.connection            = **** {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.639103 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] database.connection_debug      = 0 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.639191 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] database.connection_parameters =  {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.639273 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] database.connection_recycle_time = 3600 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.639357 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] database.connection_trace      = False {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.639457 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] database.db_inc_retry_interval = True {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.639539 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] database.db_max_retries        = 20 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.639623 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] database.db_max_retry_interval = 10 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.639708 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] database.db_retry_interval     = 1 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.639792 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] database.max_overflow          = 50 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.639876 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] database.max_pool_size         = 5 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.639962 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] database.max_retries           = 10 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.640047 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] database.mysql_sql_mode        = TRADITIONAL {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.640130 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] database.mysql_wsrep_sync_wait = None {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.640212 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] database.pool_timeout          = None {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.640296 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] database.retry_interval        = 10 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.640380 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] database.slave_connection      = **** {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.640482 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] database.sqlite_synchronous    = True {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.640566 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] database.synchronous_reader    = True {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.640650 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] database.use_db_reconnect      = False {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.640741 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] ovn.bfd_min_rx                 = 1000 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.640826 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] ovn.bfd_min_tx                 = 100 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.640910 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] ovn.bfd_mult                   = 3 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.640995 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] ovn.broadcast_arps_to_all_routers = True {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.641080 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] ovn.dhcp_default_lease_time    = 43200 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.641164 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] ovn.disable_ovn_dhcp_for_baremetal_ports = False {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.641248 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] ovn.dns_records_ovn_owned      = False {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.641334 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] ovn.dns_servers                = [] {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.641434 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] ovn.enable_distributed_floating_ip = False {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.641520 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] ovn.fdb_age_threshold          = 300 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.641629 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] ovn.ha_failover_strategy       = normal {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.641718 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] ovn.live_migration_activation_strategy = rarp {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.641802 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] ovn.localnet_learn_fdb         = False {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.641887 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] ovn.mac_binding_age_threshold  = 300 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.641980 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] ovn.neutron_sync_mode          = log {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.642065 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] ovn.ovn_dhcp4_global_options   = {} {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.642150 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] ovn.ovn_dhcp6_global_options   = {} {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.642238 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] ovn.ovn_l3_scheduler           = leastloaded {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.642326 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] ovn.ovn_metadata_enabled       = True {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.642426 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] ovn.ovn_nb_ca_cert             = /opt/stack/data/CA/int-ca/ca-chain.pem {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.642514 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] ovn.ovn_nb_certificate         = /opt/stack/data/CA/int-ca/devstack-cert.crt {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.642605 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] ovn.ovn_nb_connection          = ['ssl:199.204.45.156:6641'] {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.642692 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] ovn.ovn_nb_private_key         = /opt/stack/data/CA/int-ca/private/devstack-cert.key {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.642777 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] ovn.ovn_router_indirect_snat   = False {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.642863 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] ovn.ovn_sb_ca_cert             = /opt/stack/data/CA/int-ca/ca-chain.pem {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.642950 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] ovn.ovn_sb_certificate         = /opt/stack/data/CA/int-ca/devstack-cert.crt {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.643053 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] ovn.ovn_sb_connection          = ['ssl:199.204.45.156:6642'] {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.643130 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] ovn.ovn_sb_private_key         = /opt/stack/data/CA/int-ca/private/devstack-cert.key {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.643214 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] ovn.ovs_create_tap             = False {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.643300 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] ovn.ovsdb_connection_timeout   = 180 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.643391 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] ovn.ovsdb_log_level            = INFO {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.643490 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] ovn.ovsdb_probe_interval       = 60000 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.643575 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] ovn.ovsdb_retry_max_interval   = 180 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.643660 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] ovn.stateless_nat_enabled      = False {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.643748 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] ovn.vhost_sock_dir             = /var/run/openvswitch {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.643838 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] OVS.bridge_mac_table_size      = 50000 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.643923 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] OVS.igmp_flood                 = False {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.644010 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] OVS.igmp_flood_reports         = True {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.644093 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] OVS.igmp_flood_unregistered    = False {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.644301 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] OVS.igmp_snooping_enable       = False {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.644301 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] OVS.ovsdb_timeout              = 10 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.644454 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] ovn_nb_global.fdb_removal_limit = 0 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.644454 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] ovn_nb_global.ignore_lsp_down  = False {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.644536 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] ovn_nb_global.mac_binding_removal_limit = 0 {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.644598 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG cotyledon.oslo_config_glue [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] ******************************************************************************** {{(pid=104836) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3013}}
May 09 14:04:13.644677 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG oslo_service.backend._threading.service [None req-77a68949-e818-458d-a567-563fdf0da3b4 None None] Selected the multiprocessing context: <multiprocessing.context.ForkContext object at 0x735c9cdbeab0> and updated it in Cotyledon ServiceManager: <multiprocessing.context.ForkContext object at 0x735c9cdbeab0> {{(pid=104836) launch_service /opt/stack/data/venv/lib/python3.12/site-packages/oslo_service/backend/_threading/service.py:394}}
May 09 14:04:13.677040 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] Full set of CONF: {{(pid=105581) _load_service_options /opt/stack/data/venv/lib/python3.12/site-packages/cotyledon/oslo_config_glue.py:65}}
May 09 14:04:13.677794 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] ******************************************************************************** {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2979}}
May 09 14:04:13.677794 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] Configuration options gathered from: {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2980}}
May 09 14:04:13.677794 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] command line args: ['--config-file', '/etc/neutron/neutron.conf', '--config-file', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2981}}
May 09 14:04:13.677794 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] config files: ['/etc/neutron/neutron.conf', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2982}}
May 09 14:04:13.677794 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] ================================================================================ {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2987}}
May 09 14:04:13.678088 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] agent_down_time                = 75 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.678244 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] allow_automatic_dhcp_failover  = True {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.678454 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] allow_bulk                     = True {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.678809 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] allowed_conntrack_helpers      = [{'amanda': 'tcp'}, {'ftp': 'tcp'}, {'h323': 'udp'}, {'h323': 'tcp'}, {'irc': 'tcp'}, {'netbios-ns': 'udp'}, {'pptp': 'tcp'}, {'sane': 'tcp'}, {'sip': 'udp'}, {'sip': 'tcp'}, {'snmp': 'udp'}, {'tftp': 'udp'}] {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.678981 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] api_extensions_path            =  {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.679146 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] api_paste_config               = api-paste.ini {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.679372 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] api_workers                    = 4 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.679609 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] auth_strategy                  = keystone {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.679777 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] base_mac                       = fa:16:3e:00:00:00 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.680095 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] bind_host                      = 0.0.0.0 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.680299 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] bind_port                      = 9696 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.680513 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] client_socket_timeout          = 900 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.680701 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] config_dir                     = [] {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.680806 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] config_file                    = ['/etc/neutron/neutron.conf', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.680971 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] config_source                  = [] {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.681151 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] control_exchange               = neutron {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.681305 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] core_plugin                    = ml2 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.681579 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] debug                          = True {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.681758 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] default_availability_zones     = [] {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.681952 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] default_log_levels             = ['amqp=WARN', 'boto=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'OFPHandler=INFO', 'OfctlService=INFO', 'os_ken.base.app_manager=INFO', 'os_ken.controller.controller=INFO'] {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.682118 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] dhcp_agent_notification        = False {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.682118 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] dhcp_agents_per_network        = 1 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.682244 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] dhcp_lease_duration            = 86400 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.682244 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] dhcp_load_type                 = networks {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.682348 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] dns_domain                     = openstacklocal {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.682477 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] dvr_base_mac                   = fa:16:3f:00:00:00 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.682563 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] enable_default_route_bfd       = False {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.682563 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] enable_default_route_ecmp      = False {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.682678 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] enable_dvr                     = True {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.682678 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] enable_new_agents              = True {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.682790 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] enable_services_on_agents_with_admin_state_down = False {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.682847 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] enable_snat_by_default         = True {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.682989 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] enable_traditional_dhcp        = True {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.682989 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] external_dns_driver            = None {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.683109 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] fatal_deprecations             = False {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.683109 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] filter_validation              = True {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.683233 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] global_physnet_mtu             = 1430 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.683233 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] graceful_shutdown_timeout      = 60 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.683468 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] host                           = np0000173183 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.683559 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] host_dvr_for_dhcp              = True {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.683625 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] http_retries                   = 3 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.683690 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] instance_format                = [instance: %(uuid)s]  {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.683747 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] instance_uuid_format           = [instance: %(uuid)s]  {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.683821 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] ipam_driver                    = internal {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.683881 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] ipv6_pd_enabled                = False {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.683959 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] l3_ha                          = False {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.684026 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] l3_ha_net_cidr                 = 169.254.192.0/18 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.684112 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] l3_ha_network_physical_name    =  {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.684171 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] l3_ha_network_type             =  {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.684252 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] log_color                      = False {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.684328 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] log_config_append              = None {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.684389 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] log_date_format                = %Y-%m-%d %H:%M:%S {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.684487 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] log_dir                        = None {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.684545 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] log_file                       = None {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.684599 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] log_options                    = True {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.684665 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] log_rotate_interval            = 1 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.684727 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] log_rotate_interval_type       = days {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.684784 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] log_rotation_type              = none {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.684839 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] logging_context_format_string  = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.684896 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] logging_debug_format_suffix    = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.684896 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] logging_default_format_string  = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.685015 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] logging_exception_prefix       = ERROR %(name)s %(instance)s {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.685015 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] logging_user_identity_format   = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.685196 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] max_allowed_address_pair       = 10 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.685196 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] max_dns_nameservers            = 5 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.685196 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] max_header_line                = 16384 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.685356 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] max_l3_agents_per_router       = 3 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.685356 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] max_logfile_count              = 30 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.685535 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] max_logfile_size_mb            = 200 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.685535 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] max_routes                     = 30 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.685535 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] max_subnet_host_routes         = 20 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.685727 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] my_ip                          = 199.204.45.156 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.685727 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] my_ipv6                        = 2604:e100:1:0:f816:3eff:fe51:d95a {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.685842 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] network_auto_schedule          = True {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.685842 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] network_link_prefix            = None {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.685940 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] network_scheduler_driver       = neutron.scheduler.dhcp_agent_scheduler.WeightScheduler {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.685940 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] notify_nova_on_port_data_changes = True {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.686046 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] notify_nova_on_port_status_changes = True {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.686110 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] pagination_max_limit           = -1 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.686168 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] periodic_fuzzy_delay           = 5 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.686249 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] periodic_interval              = 40 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.686308 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] publish_errors                 = False {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.686365 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] rate_limit_burst               = 0 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.686439 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] rate_limit_except_level        = CRITICAL {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.686494 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] rate_limit_interval            = 0 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.686588 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] router_distributed             = False {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.686647 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] rpc_resources_processing_step  = 20 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.686731 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] rpc_response_max_timeout       = 600 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.686792 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] rpc_response_timeout           = 60 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.686851 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] rpc_state_report_workers       = 0 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.686912 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] rpc_workers                    = None {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.686975 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] send_events_interval           = 2 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.687058 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] service_plugins                = ['ovn-router'] {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.687117 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] setproctitle                   = on {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.687183 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] shell_completion               = None {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.687242 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] state_path                     = /opt/stack/data/neutron {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.687306 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] syslog_log_facility            = LOG_USER {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.687368 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] tcp_keepidle                   = 600 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.687450 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] transport_url                  = **** {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.687504 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] use_journal                    = False {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.687560 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] use_json                       = False {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.687629 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] use_stderr                     = False {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.687715 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] use_syslog                     = False {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.687786 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] vlan_qinq                      = None {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.687845 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] vlan_transparent               = None {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.687897 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] wsgi_default_pool_size         = 100 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.687964 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] wsgi_keep_alive                = True {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.688022 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] wsgi_log_format                = %(client_ip)s "%(request_line)s" status: %(status_code)s  len: %(body_length)s time: %(wall_seconds).7f {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.688079 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] wsgi_server_debug              = False {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 14:04:13.688192 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] oslo_concurrency.disable_process_locking = False {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.688294 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] oslo_concurrency.lock_path     = /opt/stack/data/neutron/lock {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.688438 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] profiler.connection_string     = messaging:// {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.688541 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] profiler.enabled               = False {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.688622 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] profiler.es_doc_type           = notification {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.688691 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] profiler.es_scroll_size        = 10000 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.688753 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] profiler.es_scroll_time        = 2m {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.688818 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] profiler.filter_error_trace    = False {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.688906 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] profiler.hmac_keys             = **** {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.688973 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] profiler.sentinel_service_name = mymaster {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.689061 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] profiler.socket_timeout        = 0.1 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.689127 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] profiler.trace_requests        = False {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.689505 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] profiler.trace_sqlalchemy      = False {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.689505 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] profiler_jaeger.process_tags   = {} {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.689505 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] profiler_jaeger.service_name_prefix = None {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.689505 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] profiler_otlp.service_name_prefix = None {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.689505 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.689799 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.689799 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_metrics.metrics_process_name =  {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.689799 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.689889 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.689919 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.enforce_new_defaults = True {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.689945 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.enforce_scope      = True {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.690002 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.policy_default_rule = default {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.690053 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.policy_dirs        = ['policy.d'] {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.690104 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.policy_file        = /etc/neutron/policy.json {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.690168 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.690216 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.690252 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.690295 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.690358 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.690386 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.remote_timeout     = 60.0 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.690469 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.690533 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] service_providers.service_provider = [] {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.690603 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] nova.auth_section              = None {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.690659 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] nova.auth_type                 = password {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.690689 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] nova.auth_url                  = https://199.204.45.156/identity {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.690740 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] nova.cafile                    = None {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.690784 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] nova.certfile                  = None {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.690833 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] nova.collect_timing            = False {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.690866 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] nova.default_domain_id         = None {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.690913 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] nova.default_domain_name       = None {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.690954 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] nova.domain_id                 = None {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.690994 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] nova.domain_name               = None {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.691029 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] nova.endpoint_type             = public {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.691080 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] nova.insecure                  = False {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.691190 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] nova.keyfile                   = None {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.691190 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] nova.password                  = **** {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.691190 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] nova.project_domain_id         = None {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.691270 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] nova.project_domain_name       = Default {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.691270 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] nova.project_id                = None {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.691320 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] nova.project_name              = service {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.691346 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] nova.region_name               = RegionOne {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.691373 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] nova.split_loggers             = False {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.691435 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] nova.system_scope              = None {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.691479 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] nova.timeout                   = None {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.691524 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] nova.trust_id                  = None {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.691570 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] nova.user_domain_id            = None {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.691624 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] nova.user_domain_name          = Default {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.691668 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] nova.user_id                   = None {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.691702 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] nova.username                  = nova {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.691764 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] placement.auth_section         = None {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.691815 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] placement.auth_type            = password {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.691849 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] placement.cafile               = None {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.691888 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] placement.certfile             = None {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.691933 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] placement.collect_timing       = False {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.691967 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] placement.endpoint_type        = public {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.692011 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] placement.insecure             = False {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.692046 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] placement.keyfile              = None {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.692089 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] placement.region_name          = None {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.692129 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] placement.split_loggers        = False {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.692170 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] placement.timeout              = None {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.692216 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] ironic.auth_section            = None {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.692259 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] ironic.auth_type               = None {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.692297 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] ironic.cafile                  = None {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.692336 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] ironic.certfile                = None {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.692377 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] ironic.collect_timing          = False {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.692435 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] ironic.connect_retries         = None {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.692598 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] ironic.connect_retry_delay     = None {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.692598 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] ironic.enable_notifications    = False {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.692598 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] ironic.endpoint_override       = None {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.692598 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] ironic.insecure                = False {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.692693 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] ironic.interface               = None {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.692693 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] ironic.keyfile                 = None {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.692755 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] ironic.max_version             = None {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.692784 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] ironic.min_version             = None {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.692813 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] ironic.region_name             = None {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.692858 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] ironic.retriable_status_codes  = None {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.692897 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] ironic.service_name            = None {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.692927 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] ironic.service_type            = None {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.692970 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] ironic.split_loggers           = False {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.693019 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] ironic.status_code_retries     = None {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.693054 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] ironic.status_code_retry_delay = None {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.693097 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] ironic.timeout                 = None {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.693146 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] ironic.valid_interfaces        = None {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.693187 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] ironic.version                 = None {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.693231 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] cli_script.dry_run             = False {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.693305 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.693357 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.693420 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.693507 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.693569 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.693618 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.693666 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.693706 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.693745 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 60 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.693798 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.hostname = np0000173183 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.693845 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.693897 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.693929 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.693983 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.694014 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.kombu_reconnect_splay = 0.0 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.694062 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.processname = neutron-ovn-maintenance-worker {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.694092 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.694141 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.694188 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.694222 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.694250 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.694296 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_quorum_max_memory_bytes = 0 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.694331 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_quorum_max_memory_length = 0 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.694362 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.694432 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.694475 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.694514 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_stream_fanout = False {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.694559 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.694589 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_transient_quorum_queue = False {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.694652 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.694691 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.ssl      = False {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.694737 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.ssl_ca_file =  {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.694777 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.ssl_cert_file =  {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.694812 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.694854 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.ssl_key_file =  {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.694884 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.ssl_version =  {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.694921 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.use_queue_manager = False {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.694983 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_notifications.driver = [] {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.695036 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_notifications.retry = -1 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.695072 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.695117 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_notifications.transport_url = **** {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.695172 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] oslo_reports.file_event_handler = None {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.695211 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] oslo_reports.file_event_handler_interval = 1 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.695245 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] oslo_reports.log_dir           = None {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.695304 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.695378 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.default_quota           = -1 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.695446 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_driver            = neutron.db.quota.driver_nolock.DbQuotaNoLockDriver {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.695496 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_floatingip        = 50 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.695538 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_network           = 100 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.695579 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_port              = 500 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.695619 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_rbac_policy       = 10 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.695677 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_router            = 10 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.695727 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_security_group    = 10 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.695761 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_security_group_rule = 100 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.695810 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_subnet            = 100 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.695839 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.track_quota_usage       = True {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.695922 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] privsep.capabilities           = [21, 12, 1, 2, 19] {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.695962 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] privsep.group                  = None {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.696020 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] privsep.helper_command         = None {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.696072 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] privsep.log_daemon_traceback   = False {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.696122 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] privsep.logger_name            = oslo_privsep.daemon {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.696162 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] privsep.thread_pool_size       = 16 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.696210 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] privsep.user                   = None {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.696262 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] privsep_dhcp_release.capabilities = [21, 12] {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.696298 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] privsep_dhcp_release.group     = None {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.696342 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] privsep_dhcp_release.helper_command = None {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.696380 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] privsep_dhcp_release.log_daemon_traceback = False {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.696428 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] privsep_dhcp_release.logger_name = oslo_privsep.daemon {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.696483 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] privsep_dhcp_release.thread_pool_size = 16 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.696514 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] privsep_dhcp_release.user      = None {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.696559 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] privsep_ovs_vsctl.capabilities = [21, 12] {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.696592 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] privsep_ovs_vsctl.group        = None {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.696638 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] privsep_ovs_vsctl.helper_command = None {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.696702 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] privsep_ovs_vsctl.log_daemon_traceback = False {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.696745 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] privsep_ovs_vsctl.logger_name  = oslo_privsep.daemon {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.696782 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] privsep_ovs_vsctl.thread_pool_size = 16 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.696823 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] privsep_ovs_vsctl.user         = None {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.696878 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] privsep_namespace.capabilities = [21] {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.696925 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] privsep_namespace.group        = None {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.696956 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] privsep_namespace.helper_command = None {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.697002 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] privsep_namespace.log_daemon_traceback = False {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.697049 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] privsep_namespace.logger_name  = oslo_privsep.daemon {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.697080 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] privsep_namespace.thread_pool_size = 16 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.697122 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] privsep_namespace.user         = None {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.697159 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] privsep_conntrack.capabilities = [12] {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.697207 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] privsep_conntrack.group        = None {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.697253 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] privsep_conntrack.helper_command = None {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.697284 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] privsep_conntrack.log_daemon_traceback = False {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.697323 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] privsep_conntrack.logger_name  = oslo_privsep.daemon {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.697357 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] privsep_conntrack.thread_pool_size = 16 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.697397 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] privsep_conntrack.user         = None {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.697485 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] privsep_link.capabilities      = [12, 21] {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.697533 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] privsep_link.group             = None {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.697563 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] privsep_link.helper_command    = None {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.697597 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] privsep_link.log_daemon_traceback = False {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.697654 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] privsep_link.logger_name       = oslo_privsep.daemon {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.697709 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] privsep_link.thread_pool_size  = 16 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.697746 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] privsep_link.user              = None {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.697808 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] AGENT.root_helper              = sudo /opt/stack/data/venv/bin/neutron-rootwrap /etc/neutron/rootwrap.conf {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.697857 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] AGENT.root_helper_daemon       = sudo /opt/stack/data/venv/bin/neutron-rootwrap-daemon /etc/neutron/rootwrap.conf {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.697887 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] AGENT.use_helper_for_ns_read   = True {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.697961 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] SECURITYGROUP.enable_ipset     = True {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.698000 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] SECURITYGROUP.enable_security_group = True {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.698051 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] SECURITYGROUP.firewall_driver  = None {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.698093 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] SECURITYGROUP.permitted_ethertypes = [] {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.698147 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] experimental.ipv6_pd_enabled   = False {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.698208 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] ml2.extension_drivers          = ['port_security', 'qos'] {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.698261 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] ml2.external_network_type      = None {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.698302 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] ml2.mechanism_drivers          = ['ovn'] {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.698341 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] ml2.overlay_ip_version         = 4 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.698371 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] ml2.path_mtu                   = 0 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.698443 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] ml2.physical_network_mtus      = {} {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.698476 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] ml2.project_network_types      = ['geneve'] {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.698520 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] ml2.tunnelled_network_rp_name  = rp_tunnelled {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.698559 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] ml2.type_drivers               = ['local', 'flat', 'vlan', 'geneve'] {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.698629 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] ml2_type_flat.flat_networks    = ['public'] {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.698698 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] ml2_type_geneve.max_header_size = 38 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.698756 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] ml2_type_geneve.vni_ranges     = ['1:65536'] {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.698819 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] ml2_type_vlan.network_vlan_ranges = ['public'] {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.698883 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] network_log.burst_limit        = 25 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.698926 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] network_log.local_output_log_base = None {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.698967 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] network_log.rate_limit         = 100 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.699015 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] database.asyncio_connection    = **** {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.699058 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] database.asyncio_slave_connection = **** {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.699104 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] database.backend               = sqlalchemy {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.699152 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] database.connection            = **** {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.699183 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] database.connection_debug      = 0 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.699237 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] database.connection_parameters =  {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.699282 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] database.connection_recycle_time = 3600 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.699319 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] database.connection_trace      = False {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.699355 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] database.db_inc_retry_interval = True {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.699413 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] database.db_max_retries        = 20 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.699463 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] database.db_max_retry_interval = 10 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.699504 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] database.db_retry_interval     = 1 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.699549 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] database.max_overflow          = 50 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.699588 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] database.max_pool_size         = 5 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.699630 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] database.max_retries           = 10 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.699678 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] database.mysql_sql_mode        = TRADITIONAL {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.699730 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] database.mysql_wsrep_sync_wait = None {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.699766 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] database.pool_timeout          = None {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.699812 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] database.retry_interval        = 10 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.699881 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] database.slave_connection      = **** {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.699919 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] database.sqlite_synchronous    = True {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.699968 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] database.synchronous_reader    = True {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.700007 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] database.use_db_reconnect      = False {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.700074 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] ovn.bfd_min_rx                 = 1000 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.700109 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] ovn.bfd_min_tx                 = 100 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.700153 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] ovn.bfd_mult                   = 3 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.700184 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] ovn.broadcast_arps_to_all_routers = True {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.700227 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] ovn.dhcp_default_lease_time    = 43200 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.700258 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] ovn.disable_ovn_dhcp_for_baremetal_ports = False {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.700307 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] ovn.dns_records_ovn_owned      = False {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.700347 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] ovn.dns_servers                = [] {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.700381 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] ovn.enable_distributed_floating_ip = False {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.700484 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] ovn.fdb_age_threshold          = 300 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.700530 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] ovn.ha_failover_strategy       = normal {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.700571 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] ovn.live_migration_activation_strategy = rarp {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.700610 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] ovn.localnet_learn_fdb         = False {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.700638 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] ovn.mac_binding_age_threshold  = 300 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.700695 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] ovn.neutron_sync_mode          = log {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.700746 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_dhcp4_global_options   = {} {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.700782 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_dhcp6_global_options   = {} {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.700838 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_l3_scheduler           = leastloaded {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.700870 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_metadata_enabled       = True {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.700926 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_nb_ca_cert             = /opt/stack/data/CA/int-ca/ca-chain.pem {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.700983 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_nb_certificate         = /opt/stack/data/CA/int-ca/devstack-cert.crt {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.701022 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_nb_connection          = ['ssl:199.204.45.156:6641'] {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.701050 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_nb_private_key         = /opt/stack/data/CA/int-ca/private/devstack-cert.key {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.701098 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_router_indirect_snat   = False {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.701129 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_sb_ca_cert             = /opt/stack/data/CA/int-ca/ca-chain.pem {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.701177 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_sb_certificate         = /opt/stack/data/CA/int-ca/devstack-cert.crt {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.701214 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_sb_connection          = ['ssl:199.204.45.156:6642'] {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.701255 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_sb_private_key         = /opt/stack/data/CA/int-ca/private/devstack-cert.key {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.701286 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovs_create_tap             = False {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.701333 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovsdb_connection_timeout   = 180 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.701366 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovsdb_log_level            = INFO {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.701455 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovsdb_probe_interval       = 60000 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.701494 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovsdb_retry_max_interval   = 180 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.701525 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] ovn.stateless_nat_enabled      = False {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.701571 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] ovn.vhost_sock_dir             = /var/run/openvswitch {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.701624 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] OVS.bridge_mac_table_size      = 50000 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.701672 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] OVS.igmp_flood                 = False {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.701721 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] OVS.igmp_flood_reports         = True {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.701778 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] OVS.igmp_flood_unregistered    = False {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.701826 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] OVS.igmp_snooping_enable       = False {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.701861 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] OVS.ovsdb_timeout              = 10 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.701920 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] ovn_nb_global.fdb_removal_limit = 0 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.701968 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] ovn_nb_global.ignore_lsp_down  = False {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.702002 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] ovn_nb_global.mac_binding_removal_limit = 0 {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 14:04:13.702045 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon.oslo_config_glue [-] ******************************************************************************** {{(pid=105581) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3013}}
May 09 14:04:13.702197 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG cotyledon._service_worker [-] Run service ServiceWrapper(0) [105581] {{(pid=105581) wait_forever /opt/stack/data/venv/lib/python3.12/site-packages/cotyledon/_service_worker.py:218}}
May 09 14:04:13.703630 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron_lib.callbacks.manager [-] Publish callbacks ['neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver.post_fork_initialize-292508', 'neutron.services.ovn_l3.plugin.OVNL3RouterPlugin._post_fork_initialize-384538'] for process (None), after_init {{(pid=105581) _notify_loop /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:184}}
May 09 14:04:13.705783 np0000173183 neutron-ovn-maintenance-worker[105581]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Getting OvsdbNbOvnIdl for MaintenanceWorker with retry
May 09 14:04:13.833891 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl [-] Created lookup_table index Logical_Switch.name {{(pid=105581) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:87}}
May 09 14:04:13.834048 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl [-] Created lookup_table index Logical_Router.name {{(pid=105581) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:87}}
May 09 14:04:13.834048 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl [-] Created lookup_table index Load_Balancer.name {{(pid=105581) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:87}}
May 09 14:04:13.834159 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Copp.name {{(pid=105581) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
May 09 14:04:13.834236 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Meter.name {{(pid=105581) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
May 09 14:04:13.834293 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Load_Balancer_Group.name {{(pid=105581) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
May 09 14:04:13.834349 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index HA_Chassis_Group.name {{(pid=105581) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
May 09 14:04:13.834439 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Address_Set.name {{(pid=105581) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
May 09 14:04:13.834515 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Connection.target {{(pid=105581) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
May 09 14:04:13.834582 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Gateway_Chassis.name {{(pid=105581) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
May 09 14:04:13.834647 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Mirror.name {{(pid=105581) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
May 09 14:04:13.834702 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Chassis_Template_Var.chassis {{(pid=105581) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
May 09 14:04:13.834758 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Logical_Router_Port.name {{(pid=105581) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
May 09 14:04:13.834831 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Port_Group.name {{(pid=105581) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
May 09 14:04:13.834888 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Logical_Switch_Port.name {{(pid=105581) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
May 09 14:04:13.837942 np0000173183 neutron-ovn-maintenance-worker[105581]: INFO ovsdbapp.backend.ovs_idl.vlog [-] ssl:199.204.45.156:6641: connecting...
May 09 14:04:13.844550 np0000173183 neutron-ovn-maintenance-worker[105581]: INFO ovsdbapp.backend.ovs_idl.vlog [-] ssl:199.204.45.156:6641: connected
May 09 14:04:13.870046 np0000173183 neutron-ovn-maintenance-worker[105581]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Getting OvsdbSbOvnIdl for MaintenanceWorker with retry
May 09 14:04:13.904340 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl [-] Created lookup_table index Chassis.name {{(pid=105581) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:87}}
May 09 14:04:13.904526 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl [-] Created lookup_table index Port_Binding.logical_port {{(pid=105581) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:87}}
May 09 14:04:13.904526 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Chassis_Private.name {{(pid=105581) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
May 09 14:04:13.904526 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Datapath_Binding.tunnel_key {{(pid=105581) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
May 09 14:04:13.906768 np0000173183 neutron-ovn-maintenance-worker[105581]: INFO ovsdbapp.backend.ovs_idl.vlog [-] ssl:199.204.45.156:6642: connecting...
May 09 14:04:13.911977 np0000173183 neutron-ovn-maintenance-worker[105581]: INFO ovsdbapp.backend.ovs_idl.vlog [-] ssl:199.204.45.156:6642: connected
May 09 14:04:13.948778 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method PortForwardingPlugin._check_floatingip_request of <neutron.services.portforwarding.pf_plugin.PortForwardingPlugin object at 0x735c96e36690>> floatingip precommit_update 55550000, False {{(pid=105581) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.948939 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method PortForwardingPlugin._check_floatingip_request of <neutron.services.portforwarding.pf_plugin.PortForwardingPlugin object at 0x735c96e36690>> floatingip precommit_delete 55550000, False {{(pid=105581) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.949028 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method PortForwardingPlugin._check_port_has_port_forwarding of <neutron.services.portforwarding.pf_plugin.PortForwardingPlugin object at 0x735c96e36690>> floatingip before_create 55550000, False {{(pid=105581) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.949086 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method PortForwardingPlugin._check_port_has_port_forwarding of <neutron.services.portforwarding.pf_plugin.PortForwardingPlugin object at 0x735c96e36690>> floatingip before_update 55550000, False {{(pid=105581) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.949127 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method PortForwardingPlugin._process_port_request_handler of <neutron.services.portforwarding.pf_plugin.PortForwardingPlugin object at 0x735c96e36690>> port after_update 55550000, False {{(pid=105581) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.949222 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method PortForwardingPlugin._process_port_request_handler of <neutron.services.portforwarding.pf_plugin.PortForwardingPlugin object at 0x735c96e36690>> port precommit_delete 55550000, False {{(pid=105581) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.949500 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron_lib.callbacks.manager [-] Publish callbacks ['neutron.services.portforwarding.drivers.ovn.driver.OVNPortForwarding.register-384718'] for port_forwarding_plugin (None), after_init {{(pid=105581) _notify_loop /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:184}}
May 09 14:04:13.949570 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method OVNPortForwarding._handle_notification of <neutron.services.portforwarding.drivers.ovn.driver.OVNPortForwarding object at 0x735c96f1c860>> port_forwarding after_create 55550000, False {{(pid=105581) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.949625 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method OVNPortForwarding._handle_lb_on_ls of <neutron.services.portforwarding.drivers.ovn.driver.OVNPortForwarding object at 0x735c96f1c860>> router_interface after_create 55550000, False {{(pid=105581) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.949713 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method OVNPortForwarding._handle_notification of <neutron.services.portforwarding.drivers.ovn.driver.OVNPortForwarding object at 0x735c96f1c860>> port_forwarding after_update 55550000, False {{(pid=105581) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.949764 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method OVNPortForwarding._handle_notification of <neutron.services.portforwarding.drivers.ovn.driver.OVNPortForwarding object at 0x735c96f1c860>> port_forwarding after_delete 55550000, False {{(pid=105581) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.949841 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method OVNPortForwarding._handle_lb_on_ls of <neutron.services.portforwarding.drivers.ovn.driver.OVNPortForwarding object at 0x735c96f1c860>> router_interface after_delete 55550000, False {{(pid=105581) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.949969 np0000173183 neutron-ovn-maintenance-worker[105581]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.extensions.qos [-] Starting OVNClientQosExtension
May 09 14:04:13.950062 np0000173183 neutron-ovn-maintenance-worker[105581]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.extensions.placement [-] Starting OVNClientPlacementExtension
May 09 14:04:13.950463 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Plugin._prevent_segment_delete_with_subnet_associated of <neutron.services.segments.plugin.Plugin object at 0x735c96e37b90>> segment before_delete 55550000, False {{(pid=105581) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.950525 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Plugin._validate_auto_address_subnet_delete of <neutron.services.segments.plugin.Plugin object at 0x735c96e37b90>> subnet precommit_delete_associations 55550000, False {{(pid=105581) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.950861 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_host_addition_to_aggregate of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x735c96a87a10>> segment_host_mapping after_create 55550000, False {{(pid=105581) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.950933 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_host_removal_from_aggregate of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x735c96a87a10>> segment_host_mapping after_delete 55550000, False {{(pid=105581) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.950998 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_port_created_or_deleted of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x735c96a87a10>> port after_create 55550000, False {{(pid=105581) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.951040 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_port_created_or_deleted of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x735c96a87a10>> port after_delete 55550000, False {{(pid=105581) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.951123 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_port_updated of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x735c96a87a10>> port after_update 55550000, False {{(pid=105581) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.951171 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_segment_deleted of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x735c96a87a10>> segment after_delete 55550000, False {{(pid=105581) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.951238 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_subnet_created of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x735c96a87a10>> subnet after_create 55550000, False {{(pid=105581) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.951300 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_subnet_deleted of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x735c96a87a10>> subnet after_delete 55550000, False {{(pid=105581) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:13.951352 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_subnet_updated of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x735c96a87a10>> subnet after_update 55550000, False {{(pid=105581) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:14.002606 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SegmentHostRoutes.host_routes_after_create of <neutron.services.segments.plugin.SegmentHostRoutes object at 0x735c96ff4140>> subnet after_create 55550000, False {{(pid=105581) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:14.002751 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SegmentHostRoutes.host_routes_after_delete of <neutron.services.segments.plugin.SegmentHostRoutes object at 0x735c96ff4140>> subnet after_delete 55550000, False {{(pid=105581) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:14.002751 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SegmentHostRoutes.host_routes_before_create of <neutron.services.segments.plugin.SegmentHostRoutes object at 0x735c96ff4140>> subnet before_create 55550000, False {{(pid=105581) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:14.002808 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SegmentHostRoutes.host_routes_before_update of <neutron.services.segments.plugin.SegmentHostRoutes object at 0x735c96ff4140>> subnet before_update 55550000, False {{(pid=105581) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:14.004968 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method LoggingPlugin._clean_logs_by_resource_id of <neutron.services.logapi.logging_plugin.LoggingPlugin object at 0x735c96a961e0>> security_group after_delete 55550000, False {{(pid=105581) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:14.005059 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method LoggingPlugin._clean_logs_by_target_id of <neutron.services.logapi.logging_plugin.LoggingPlugin object at 0x735c96a961e0>> port after_delete 55550000, False {{(pid=105581) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:14.005199 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron_lib.callbacks.manager [-] Publish callbacks ['neutron.services.logapi.drivers.ovn.driver.DriverBase._register-47772'] for logging-plugin (None), after_init {{(pid=105581) _notify_loop /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:184}}
May 09 14:04:14.005323 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SecurityGroupRuleCallBack.handle_event of <neutron.services.logapi.common.sg_callback.SecurityGroupRuleCallBack object at 0x735c97e00830>> security_group_rule after_create 55550000, False {{(pid=105581) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:14.005386 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SecurityGroupRuleCallBack.handle_event of <neutron.services.logapi.common.sg_callback.SecurityGroupRuleCallBack object at 0x735c97e00830>> security_group_rule after_update 55550000, False {{(pid=105581) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:14.005467 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SecurityGroupRuleCallBack.handle_event of <neutron.services.logapi.common.sg_callback.SecurityGroupRuleCallBack object at 0x735c97e00830>> security_group_rule after_delete 55550000, False {{(pid=105581) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 14:04:23.417207 np0000173183 neutron-ovn-maintenance-worker[104836]: DEBUG dbcounter [-] [104836] Writing DB stats neutron:SELECT=4 {{(pid=104836) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 14:04:24.007739 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [-] OVN-Northbound DB sync process started @ 2026-05-09 14:04:24.007101 {{(pid=105581) do_sync /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:99}}
May 09 14:04:24.010572 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-02486e95-fb9f-4e98-8732-bf8b4c8c80be None None] OVN-NB Sync port groups started @ 2026-05-09 14:04:24.010076 {{(pid=105581) sync_port_groups /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:132}}
May 09 14:04:24.028668 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG dbcounter [-] [105581] Writer thread running {{(pid=105581) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:102}}
May 09 14:04:24.067736 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-02486e95-fb9f-4e98-8732-bf8b4c8c80be None None] OVN-NB Sync port groups completed @ 2026-05-09 14:04:24.067423 {{(pid=105581) sync_port_groups /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:196}}
May 09 14:04:24.067872 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-02486e95-fb9f-4e98-8732-bf8b4c8c80be None None] OVN-NB Sync networks, ports and DHCP options started @ 2026-05-09 14:04:24.067720 {{(pid=105581) sync_networks_ports_and_dhcp_opts /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1152}}
May 09 14:04:24.417155 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.services.segments.db [None req-02486e95-fb9f-4e98-8732-bf8b4c8c80be None None] neutron.services.segments.plugin.Plugin method get_segments called with arguments (<neutron_lib.context.Context object at 0x735c96ff76e0>,) {'filters': {'network_id': ['0b4a5724-2518-4e33-a6d3-0d92c749f303'], 'is_dynamic': False}} {{(pid=105581) wrapper /opt/stack/data/venv/lib/python3.12/site-packages/oslo_log/helpers.py:77}}
May 09 14:04:24.423035 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-02486e95-fb9f-4e98-8732-bf8b4c8c80be None None] OVN-NB Sync metadata ports started {{(pid=105581) _sync_metadata_ports /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1097}}
May 09 14:04:24.483954 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-02486e95-fb9f-4e98-8732-bf8b4c8c80be None None] OVN-NB Sync metadata ports completed {{(pid=105581) _sync_metadata_ports /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1149}}
May 09 14:04:24.484064 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-02486e95-fb9f-4e98-8732-bf8b4c8c80be None None] OVN-NB Sync DHCP options for Neutron subnets started {{(pid=105581) _sync_subnet_dhcp_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:949}}
May 09 14:04:24.554255 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-02486e95-fb9f-4e98-8732-bf8b4c8c80be None None] OVN-NB Sync DHCP options for Neutron subnets completed @ 2026-05-09 14:04:24.553933 {{(pid=105581) _sync_subnet_dhcp_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1022}}
May 09 14:04:24.555555 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] There are no commands to commit {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:78}}
May 09 14:04:24.555954 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-02486e95-fb9f-4e98-8732-bf8b4c8c80be None None] OVN-NB Sync DHCP options for Neutron ports with extra dhcp options assigned started {{(pid=105581) _sync_port_dhcp_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1027}}
May 09 14:04:24.556094 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-02486e95-fb9f-4e98-8732-bf8b4c8c80be None None] OVN-NB Sync DHCP options for Neutron ports with extra dhcp options assigned completed {{(pid=105581) _sync_port_dhcp_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1086}}
May 09 14:04:24.556191 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-02486e95-fb9f-4e98-8732-bf8b4c8c80be None None] OVN-NB Sync networks, ports and DHCP options completed @ 2026-05-09 14:04:24.556054 {{(pid=105581) sync_networks_ports_and_dhcp_opts /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1321}}
May 09 14:04:24.556332 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-02486e95-fb9f-4e98-8732-bf8b4c8c80be None None] OVN-NB Sync ACLs started @ 2026-05-09 14:04:24.556196 {{(pid=105581) sync_acls /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:239}}
May 09 14:04:24.610537 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-02486e95-fb9f-4e98-8732-bf8b4c8c80be None None] OVN-NB Sync ACLs completed @ 2026-05-09 14:04:24.608731 {{(pid=105581) sync_acls /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:431}}
May 09 14:04:24.610537 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-02486e95-fb9f-4e98-8732-bf8b4c8c80be None None] OVN-NB Sync routers and router ports started @ 2026-05-09 14:04:24.609005 {{(pid=105581) sync_routers_and_rports /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:736}}
May 09 14:04:24.661540 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] There are no commands to commit {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:78}}
May 09 14:04:24.661892 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-02486e95-fb9f-4e98-8732-bf8b4c8c80be None None] OVN-NB Sync routers and router ports completed @ 2026-05-09 14:04:24.661575 {{(pid=105581) sync_routers_and_rports /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:944}}
May 09 14:04:24.662020 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-02486e95-fb9f-4e98-8732-bf8b4c8c80be None None] OVN-NB Sync port QoS policies started @ 2026-05-09 14:04:24.661888 {{(pid=105581) sync_port_qos_policies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1369}}
May 09 14:04:24.662141 np0000173183 neutron-ovn-maintenance-worker[105581]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.extensions.qos [None req-02486e95-fb9f-4e98-8732-bf8b4c8c80be None None] Starting OVNClientQosExtension
May 09 14:04:24.677779 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-02486e95-fb9f-4e98-8732-bf8b4c8c80be None None] OVN-NB Sync port QoS policies completed @ 2026-05-09 14:04:24.677638 {{(pid=105581) sync_port_qos_policies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1392}}
May 09 14:04:24.677871 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-02486e95-fb9f-4e98-8732-bf8b4c8c80be None None] OVN-NB Sync Floating IP QoS policies started @ 2026-05-09 14:04:24.677777 {{(pid=105581) sync_fip_qos_policies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1397}}
May 09 14:04:24.677960 np0000173183 neutron-ovn-maintenance-worker[105581]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.extensions.qos [None req-02486e95-fb9f-4e98-8732-bf8b4c8c80be None None] Starting OVNClientQosExtension
May 09 14:04:24.698610 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-02486e95-fb9f-4e98-8732-bf8b4c8c80be None None] OVN-NB Sync Floating IP QoS policies completed @ 2026-05-09 14:04:24.698432 {{(pid=105581) sync_fip_qos_policies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1420}}
May 09 14:04:24.698777 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-02486e95-fb9f-4e98-8732-bf8b4c8c80be None None] OVN-NB Sync Floating IP NAT rules started @ 2026-05-09 14:04:24.698586 {{(pid=105581) sync_fip_dnat_rules /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1425}}
May 09 14:04:24.698964 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-02486e95-fb9f-4e98-8732-bf8b4c8c80be None None] OVN-NB Sync Floating IP NAT rules completed @ 2026-05-09 14:04:24.698838 {{(pid=105581) sync_fip_dnat_rules /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1451}}
May 09 14:04:24.699075 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-02486e95-fb9f-4e98-8732-bf8b4c8c80be None None] OVN-Northbound DB sync process completed @ 2026-05-09 14:04:24.698938 {{(pid=105581) do_sync /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:112}}
May 09 14:04:34.008998 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [-] OVN-Southbound DB sync process started @ 2026-05-09 14:04:34.008578 {{(pid=105581) do_sync /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1480}}
May 09 14:04:34.010788 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-253fa6ff-6398-48d0-8800-8b2ef12122a3 None None] OVN-SB Sync hostname and physical networks started @ 2026-05-09 14:04:34.010392 {{(pid=105581) sync_hostname_and_physical_networks /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1511}}
May 09 14:04:34.012218 np0000173183 neutron-ovn-maintenance-worker[105581]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.extensions.qos [-] Starting OVNClientQosExtension
May 09 14:04:34.014839 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.add_provnet_ext_id_to_localnet_ports {{(pid=105581) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 09 14:04:34.015492 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_baremetal_ports_dhcp_options {{(pid=105581) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 09 14:04:34.015601 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_dhcp_options_consistency {{(pid=105581) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 09 14:04:34.015697 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_fair_meter_consistency {{(pid=105581) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 09 14:04:34.015823 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_fdb_aging_settings {{(pid=105581) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 09 14:04:34.015823 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_for_igmp_snoop_support {{(pid=105581) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 09 14:04:34.015823 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_for_inconsistencies {{(pid=105581) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 09 14:04:34.016160 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_global_dhcp_opts {{(pid=105581) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 09 14:04:34.016160 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_localnet_port_has_learn_fdb {{(pid=105581) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 09 14:04:34.016160 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_network_broadcast_arps_to_all_routers {{(pid=105581) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 09 14:04:34.016160 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_provider_distributed_ports {{(pid=105581) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 09 14:04:34.016160 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_redirect_type_router_gateway_ports {{(pid=105581) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 09 14:04:34.016160 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip {{(pid=105581) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 09 14:04:34.016559 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.cleanup_old_hash_ring_nodes {{(pid=105581) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 09 14:04:34.016559 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.configure_nb_global {{(pid=105581) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 09 14:04:34.016559 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.migrate_lrp_gateway_chassis_to_ha_chassis_group {{(pid=105581) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 09 14:04:34.016559 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.set_fip_distributed_flag {{(pid=105581) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 09 14:04:34.016559 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.set_network_type_and_physnet {{(pid=105581) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 09 14:04:34.018448 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.set_ovn_owned_dns_option {{(pid=105581) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 09 14:04:34.018601 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.update_ha_failover {{(pid=105581) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 09 14:04:34.018729 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.update_lrouter_ports_ext_ids_name_prefix {{(pid=105581) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 09 14:04:34.018840 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.update_mac_aging_settings {{(pid=105581) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 09 14:04:34.018930 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.update_router_distributed_flag {{(pid=105581) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 09 14:04:34.019028 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.update_router_static_routes {{(pid=105581) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 09 14:04:34.020343 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.update_security_group_with_address_group {{(pid=105581) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 09 14:04:34.020501 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.update_virtual_port_parent_hostname {{(pid=105581) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 09 14:04:34.021932 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.add_provnet_ext_id_to_localnet_ports' {{(pid=105581) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 09 14:04:34.022824 np0000173183 neutron-ovn-maintenance-worker[105581]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver [-] Maintenance task thread has started
May 09 14:04:34.024783 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.agent.neutron_agent [None req-253fa6ff-6398-48d0-8800-8b2ef12122a3 None None] Agents populated in the agent cache: {{(pid=105581) populate /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/agent/neutron_agent.py:310}}
May 09 14:04:34.025120 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.agent.neutron_agent [None req-253fa6ff-6398-48d0-8800-8b2ef12122a3 None None] - Agent type: OVN Controller Gateway agent, hostname: np0000173183, name: 4524ca43-0613-40ef-9118-5c14197adb05 {{(pid=105581) populate /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/agent/neutron_agent.py:312}}
May 09 14:04:34.025612 np0000173183 neutron-ovn-maintenance-worker[105581]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver [-] MaintenanceWorker process has finished the post initialization
May 09 14:04:34.028519 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.agent.neutron_agent [None req-253fa6ff-6398-48d0-8800-8b2ef12122a3 None None] - Agent type: OVN Neutron agent, hostname: np0000173183, name: 4524ca43-0613-40ef-9118-5c14197adb05 {{(pid=105581) populate /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/agent/neutron_agent.py:312}}
May 09 14:04:34.034600 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_baremetal_ports_dhcp_options' {{(pid=105581) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 09 14:04:34.034926 np0000173183 neutron-ovn-maintenance-worker[105581]: WARNING neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-253fa6ff-6398-48d0-8800-8b2ef12122a3 None None] New host np0000173183 found in OVN SB DB, but not in Neutron. Add its SegmentHostMapping in Neutron
May 09 14:04:34.038450 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_dhcp_options_consistency' {{(pid=105581) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 09 14:04:34.038563 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fair_meter_consistency' {{(pid=105581) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 09 14:04:34.038638 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 09 14:04:34.038699 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_igmp_snoop_support' {{(pid=105581) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 09 14:04:34.038758 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=105581) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 09 14:04:34.038813 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_global_dhcp_opts' {{(pid=105581) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 09 14:04:34.038889 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_localnet_port_has_learn_fdb' {{(pid=105581) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 09 14:04:34.038945 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_network_broadcast_arps_to_all_routers' {{(pid=105581) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 09 14:04:34.039000 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_provider_distributed_ports' {{(pid=105581) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 09 14:04:34.039083 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_redirect_type_router_gateway_ports' {{(pid=105581) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 09 14:04:34.039348 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 09 14:04:34.039432 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.cleanup_old_hash_ring_nodes' {{(pid=105581) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 09 14:04:34.039493 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.configure_nb_global' {{(pid=105581) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 09 14:04:34.039577 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.migrate_lrp_gateway_chassis_to_ha_chassis_group' {{(pid=105581) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 09 14:04:34.039656 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 09 14:04:34.039777 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_network_type_and_physnet' {{(pid=105581) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 09 14:04:34.039777 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_ovn_owned_dns_option' {{(pid=105581) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 09 14:04:34.039875 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 09 14:04:34.039919 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_lrouter_ports_ext_ids_name_prefix' {{(pid=105581) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 09 14:04:34.040000 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 09 14:04:34.040100 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_router_distributed_flag' {{(pid=105581) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 09 14:04:34.040198 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_router_static_routes' {{(pid=105581) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 09 14:04:34.040293 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_security_group_with_address_group' {{(pid=105581) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 09 14:04:34.040388 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_virtual_port_parent_hostname' {{(pid=105581) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 09 14:04:34.047485 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.add_provnet_ext_id_to_localnet_ports' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105581) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 09 14:04:34.047710 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_baremetal_ports_dhcp_options {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:04:34.047872 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Sync baremetal port DHCP options with configuration. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:04:34.094328 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG oslo_concurrency.lockutils [None req-253fa6ff-6398-48d0-8800-8b2ef12122a3 None None] Acquiring lock "update_segment_host_mapping" by "neutron.services.segments.db.update_segment_host_mapping" {{(pid=105581) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:506}}
May 09 14:04:34.094911 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG oslo_concurrency.lockutils [None req-253fa6ff-6398-48d0-8800-8b2ef12122a3 None None] Lock "update_segment_host_mapping" acquired by "neutron.services.segments.db.update_segment_host_mapping" :: waited 0.000s {{(pid=105581) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:519}}
May 09 14:04:34.137256 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron_lib.callbacks.manager [None req-253fa6ff-6398-48d0-8800-8b2ef12122a3 None None] Publish callbacks ['neutron.plugins.ml2.plugin.DhcpAgentSchedulerDbMixin.auto_schedule_new_network_segments-1231248', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_host_addition_to_aggregate-15694889'] for segment_host_mapping (None), after_create {{(pid=105581) _notify_loop /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:184}}
May 09 14:04:34.142673 np0000173183 neutron-ovn-maintenance-worker[105581]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_baremetal_ports_dhcp_options finished in 0.095 seconds
May 09 14:04:34.145834 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_baremetal_ports_dhcp_options' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105581) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 09 14:04:34.146278 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_dhcp_options_consistency {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:04:34.146515 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Backfill network_id on DHCP_Options external_ids. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:04:34.168796 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.services.segments.db [None req-253fa6ff-6398-48d0-8800-8b2ef12122a3 None None] Segments {'08fea115-9b05-4667-a0d0-487c303084a5'} mapped to the host np0000173183 {{(pid=105581) update_segment_host_mapping /opt/stack/neutron/neutron/services/segments/db.py:249}}
May 09 14:04:34.173742 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG oslo_concurrency.lockutils [None req-253fa6ff-6398-48d0-8800-8b2ef12122a3 None None] Lock "update_segment_host_mapping" "released" by "neutron.services.segments.db.update_segment_host_mapping" :: held 0.079s {{(pid=105581) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:538}}
May 09 14:04:34.173873 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-253fa6ff-6398-48d0-8800-8b2ef12122a3 None None] OVN-SB Sync hostname and physical networks completed @ 2026-05-09 14:04:34.173733 {{(pid=105581) sync_hostname_and_physical_networks /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1538}}
May 09 14:04:34.196179 np0000173183 neutron-ovn-maintenance-worker[105581]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_dhcp_options_consistency finished in 0.049 seconds
May 09 14:04:34.196414 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_dhcp_options_consistency' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105581) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 09 14:04:34.196608 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fair_meter_consistency {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:04:34.196719 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Refresh ACL logging fair meter after configuration reload. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:04:34.197019 np0000173183 neutron-ovn-maintenance-worker[105581]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fair_meter_consistency finished in 0.000 seconds
May 09 14:04:34.197160 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fair_meter_consistency' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105581) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 09 14:04:34.197298 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:04:34.197465 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:04:34.198538 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:04:34.255687 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:04:34.255907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:34.256704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:04:34.256704 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:34.256704 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:04:34.256704 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:04:34.256704 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:34.256704 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:04:34.256704 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:34.256704 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:04:34.258168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.059 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:34.258168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:04:34.258168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:04:34.258168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:04:34.258168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:04:34.258168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:04:34.258168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:04:34.258168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:34.258168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:04:34.258168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:04:34.258168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:04:34.258168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:34.258472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:04:34.258472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:04:34.258472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:04:34.258472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:04:34.258472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:04:34.258472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:04:34.258472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:04:34.258472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:04:34.258472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:34.258472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:04:34.258472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:04:34.258472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:34.258472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:04:34.258742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:34.258742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:04:34.267011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call immediate 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:34.267011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:04:34.267011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:04:34.267011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:04:34.267011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:04:34.267011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:04:34.267011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:04:34.267011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:04:34.267011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:04:34.267011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:04:34.267011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:04:34.267011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:04:34.267011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:04:34.267011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:04:34.267011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:04:34.267011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:04:34.267378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:04:34.267378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:04:34.267378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:34.267378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:04:34.267378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:04:34.267378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:04:34.267378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:34.267378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:04:34.267378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:04:34.267378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:04:34.267378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:04:34.267378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:04:34.267378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:04:34.267378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:04:34.267378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:04:34.267378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:34.267378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:04:34.267378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:04:34.267378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:34.267764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:04:34.267764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:34.267764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:04:34.267764 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_for_igmp_snoop_support {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:04:34.267764 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Check for IGMP snoop support. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:04:34.268585 np0000173183 neutron-ovn-maintenance-worker[105581]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_for_igmp_snoop_support finished in 0.000 seconds
May 09 14:04:34.268585 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_igmp_snoop_support' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105581) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 09 14:04:34.268585 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_for_inconsistencies {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:04:34.268585 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Checking Neutron and OVN revision consistency. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:04:34.285257 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: No inconsistencies found. Skipping {{(pid=105581) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:437}}
May 09 14:04:34.286173 np0000173183 neutron-ovn-maintenance-worker[105581]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_for_inconsistencies finished in 0.018 seconds
May 09 14:04:34.286380 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_global_dhcp_opts {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:04:34.286512 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Check global DHCP options consistency. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:04:34.286704 np0000173183 neutron-ovn-maintenance-worker[105581]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_global_dhcp_opts finished in 0.000 seconds
May 09 14:04:34.286857 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_global_dhcp_opts' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105581) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 09 14:04:34.287033 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_localnet_port_has_learn_fdb {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:04:34.287141 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Ensure localnet ports have learn-fdb set per configuration. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:04:34.287746 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] There are no commands to commit {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:78}}
May 09 14:04:34.287989 np0000173183 neutron-ovn-maintenance-worker[105581]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_localnet_port_has_learn_fdb finished in 0.001 seconds
May 09 14:04:34.288317 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_localnet_port_has_learn_fdb' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105581) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 09 14:04:34.288483 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_network_broadcast_arps_to_all_routers {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:04:34.288590 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Sync broadcast-arps-to-all-routers on external networks. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:04:34.403141 np0000173183 neutron-ovn-maintenance-worker[105581]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_network_broadcast_arps_to_all_routers finished in 0.114 seconds
May 09 14:04:34.403663 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_network_broadcast_arps_to_all_routers' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105581) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 09 14:04:34.404097 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_provider_distributed_ports {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:04:34.405867 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Align reside-on-redirect-chassis on VLAN/FLAT distributed ports. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:04:34.470896 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(uuid=92dabe77-a12e-4f9a-a508-f2d61ef639f1, enabled=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-0b4a5724-2518-4e33-a6d3-0d92c749f303', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13', 'neutron:subnet_ids': '23eb2e2e-ec2f-4f98-bc47-7125a2f92952'}, gateway_chassis=[], ha_chassis_group=[], ipv6_prefix=[], ipv6_ra_configs={'address_mode': 'slaac', 'mtu': '1372', 'send_periodic': 'true'}, mac='fa:16:3e:a8:cc:8e', name='lrp-1c50398a-a235-4a8a-a01a-9a5769b68b85', networks=['fd33:1208:5349::1/64'], options={}, peer=[], status={}) old= {{(pid=105581) matches /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}}
May 09 14:04:34.472246 np0000173183 neutron-ovn-maintenance-worker[105581]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.extensions.qos [-] Starting OVNClientQosExtension
May 09 14:04:34.538598 np0000173183 neutron-ovn-maintenance-worker[105581]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_provider_distributed_ports finished in 0.132 seconds
May 09 14:04:34.538787 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_provider_distributed_ports' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105581) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 09 14:04:34.538997 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_redirect_type_router_gateway_ports {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:04:34.539321 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Align router gateway redirect-type for provider VLAN/FLAT networks. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:04:34.596522 np0000173183 neutron-ovn-maintenance-worker[105581]: INFO neutron.common.ovn.utils [-] HA Chassis Group neutron-0b4a5724-2518-4e33-a6d3-0d92c749f303 synchronized; highest priority chassis 4524ca43-0613-40ef-9118-5c14197adb05
May 09 14:04:34.598517 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=HA_Chassis_Group, record=neutron-0b4a5724-2518-4e33-a6d3-0d92c749f303, col_values=(('external_ids', {'neutron:availability_zone_hints': '', 'neutron:network_id': '0b4a5724-2518-4e33-a6d3-0d92c749f303', 'neutron:router_id': '5dcc5c62-3db6-43c1-9b38-e1c7de943f13'}),), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:04:34.598823 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): HAChassisGroupAddChassisCommand(_result=None, hcg=neutron-0b4a5724-2518-4e33-a6d3-0d92c749f303, chassis=4524ca43-0613-40ef-9118-5c14197adb05, priority=1, columns={}) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:04:34.599074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:34.602068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
May 09 14:04:34.602068 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:34.602068 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:04:34.602068 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:04:34.602068 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:34.602068 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:04:34.602068 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:34.602068 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:04:34.869031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event [-] Unexpected exception in notify_loop: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:34.869031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event Traceback (most recent call last):
May 09 14:04:34.869031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/event.py", line 185, in notify_loop
May 09 14:04:34.869031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event     match.run(event, row, updates)
May 09 14:04:34.869031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/services/ovn_l3/ovsdb_monitor.py", line 86, in run
May 09 14:04:34.869031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event     self.l3_plugin._ovn_client.link_network_ha_chassis_group(
May 09 14:04:34.869031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2416, in link_network_ha_chassis_group
May 09 14:04:34.869031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:04:34.869031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:34.869031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event     next(self.gen)
May 09 14:04:34.869031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:04:34.869031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event     with super().transaction(*args, **kwargs) as t:
May 09 14:04:34.869031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:34.869031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event     next(self.gen)
May 09 14:04:34.869031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:04:34.869031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event     with self.create_transaction(
May 09 14:04:34.869031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:04:34.869031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event     self.result = self.commit()
May 09 14:04:34.869031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event                   ^^^^^^^^^^^^^
May 09 14:04:34.869674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:04:34.869674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event     raise result.ex
May 09 14:04:34.869674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:34.869674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event     txn.results.put(txn.do_commit())
May 09 14:04:34.869674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event                     ^^^^^^^^^^^^^^^
May 09 14:04:34.869674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:34.869674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event     raise RuntimeError(msg)
May 09 14:04:34.869674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:34.869674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event 
May 09 14:04:34.916136 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] There are no commands to commit {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:78}}
May 09 14:04:34.917691 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-253fa6ff-6398-48d0-8800-8b2ef12122a3 None None] OVN-Southbound DB sync process completed @ 2026-05-09 14:04:34.917393 {{(pid=105581) do_sync /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1488}}
May 09 14:04:34.921885 np0000173183 neutron-ovn-maintenance-worker[105581]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_redirect_type_router_gateway_ports finished in 0.382 seconds
May 09 14:04:34.922138 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_redirect_type_router_gateway_ports' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105581) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 09 14:04:34.922372 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:04:34.922548 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:04:34.923934 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:04:34.924253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:34.925127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:04:34.925127 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:34.925127 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:04:34.925127 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:04:34.925127 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:34.925127 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:04:34.925127 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:34.925127 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:04:34.926186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:34.926186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:04:34.926186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:04:34.926186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:04:34.926186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:04:34.926186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:04:34.926186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:04:34.926186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:34.926186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:04:34.926186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:04:34.926186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:04:34.926186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:34.926546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:04:34.926546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:04:34.926546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:04:34.926546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:04:34.926546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:04:34.926546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:04:34.926546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:04:34.926546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:04:34.926546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:34.926546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:04:34.926546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:04:34.926546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:34.926546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:04:34.926866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:34.926866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:04:34.927236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call immediate 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:34.927236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:04:34.927236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:04:34.927236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:04:34.927236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:04:34.927236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:04:34.927236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:04:34.927236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:04:34.927236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:04:34.927236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:04:34.927236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:04:34.927236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:04:34.927236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:04:34.927236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:04:34.927236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:04:34.927236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:04:34.927672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:04:34.927672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:04:34.927672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:34.927672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:04:34.927672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:04:34.927672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:04:34.927672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:34.927672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:04:34.927672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:04:34.927672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:04:34.927672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:04:34.927672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:04:34.927672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:04:34.927672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:04:34.927672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:04:34.927672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:34.927672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:04:34.927672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:04:34.927672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:34.928152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:04:34.928152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:34.928152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:04:34.928152 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: cleanup_old_hash_ring_nodes {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:04:34.928152 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Purge stale hash ring nodes older than five days. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:04:34.935059 np0000173183 neutron-ovn-maintenance-worker[105581]: INFO neutron.db.ovn_hash_ring_db [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Cleaned up Hash Ring nodes older than 5 days
May 09 14:04:34.935255 np0000173183 neutron-ovn-maintenance-worker[105581]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task cleanup_old_hash_ring_nodes finished in 0.008 seconds
May 09 14:04:34.935524 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: configure_nb_global {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:04:34.935691 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply ovn_nb_global settings to NB_Global options. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:04:34.935990 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'ignore_lsp_down': 'false', 'fdb_removal_limit': '0', 'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:04:34.936591 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'ignore_lsp_down': 'false', 'fdb_removal_limit': '0', 'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:04:34.937006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:34.937796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:04:34.937796 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:34.937796 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:04:34.937796 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:04:34.937796 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:34.937796 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:04:34.937796 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:34.937796 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:04:34.938721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:34.938721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:04:34.938721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:04:34.938721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:04:34.938721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:34.938721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:04:34.938721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:04:34.938721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:04:34.938721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:34.938721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:04:34.938721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:04:34.938721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:04:34.938721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:04:34.938721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:04:34.938721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:04:34.939141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:04:34.939141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:04:34.939141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:34.939141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:04:34.939141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:04:34.939141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:34.939141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:04:34.939141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:34.939141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:04:34.939764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task configure_nb_global failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:34.939764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:04:34.939764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:04:34.939764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:04:34.939764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:04:34.939764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1038, in configure_nb_global
May 09 14:04:34.939764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:04:34.939764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:04:34.939764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:04:34.939764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:34.939764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:04:34.940104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:04:34.940104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:04:34.940104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:34.940104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:04:34.940104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:04:34.940104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:04:34.940104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:04:34.940104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:04:34.940104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:04:34.940104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:04:34.940104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:04:34.940104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:34.940104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:04:34.940104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:04:34.940467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:34.940467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:04:34.940467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:34.940467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:04:34.940839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call immediate 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.configure_nb_global' (it runs every 86400.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:34.940839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:04:34.940839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:04:34.940839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:04:34.940839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:04:34.940839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:04:34.940839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:04:34.940839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:04:34.940839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:04:34.940839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:04:34.940839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:04:34.940839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:04:34.940839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:04:34.940839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:04:34.940839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:04:34.940839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:04:34.941385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1038, in configure_nb_global
May 09 14:04:34.941385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:04:34.941385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:04:34.941385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:04:34.941385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:34.941385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:04:34.941385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:04:34.941385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:04:34.941385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:34.941385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:04:34.941385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:04:34.941385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:04:34.941385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:04:34.941385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:04:34.941385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:04:34.941385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:04:34.941385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:04:34.941385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:34.941385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:04:34.941945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:04:34.941945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:34.941945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:04:34.941945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:34.941945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:04:34.941945 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] There are no commands to commit {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:78}}
May 09 14:04:34.941945 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.migrate_lrp_gateway_chassis_to_ha_chassis_group' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105581) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 09 14:04:34.941945 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:04:34.941945 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:04:34.942201 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:04:34.942604 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:04:34.942887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:34.943740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:04:34.943740 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:34.943740 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:04:34.943740 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:04:34.943740 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:34.943740 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:04:34.943740 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:34.943740 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:04:34.944463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:34.944463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:04:34.944463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:04:34.944463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:04:34.944463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:34.944463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:04:34.944463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:04:34.944463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:04:34.944463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:34.944463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:04:34.944463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:04:34.944463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:04:34.944463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:04:34.944463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:04:34.944463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:04:34.945162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:04:34.945162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:04:34.945162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:34.945162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:04:34.945162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:04:34.945162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:34.945162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:04:34.945162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:34.945162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:04:34.945513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:34.945513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:04:34.945513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:04:34.945513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:04:34.945513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:04:34.945513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:04:34.945513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:04:34.945513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:04:34.945513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:04:34.945513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:04:34.945513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:34.945513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:04:34.946030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:04:34.946030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:04:34.946030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:34.946030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:04:34.946030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:04:34.946030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:04:34.946030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:04:34.946030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:04:34.946030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:04:34.946030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:04:34.946030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:04:34.946030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:34.946030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:04:34.946030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:04:34.946572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:34.946572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:04:34.946572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:34.946572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:04:34.946572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call immediate 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:34.946572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:04:34.946572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:04:34.946572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:04:34.946572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:04:34.946572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:04:34.946572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:04:34.946572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:04:34.946572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:04:34.946572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:04:34.947217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:04:34.947217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:04:34.947217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:04:34.947217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:04:34.947217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:04:34.947217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:04:34.947217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:04:34.947217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:04:34.947217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:04:34.947217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:04:34.947217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:04:34.947217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:34.947217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:04:34.947217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:04:34.947217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:04:34.947217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:34.947217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:04:34.947217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:04:34.947217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:04:34.948911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:04:34.948911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:04:34.948911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:04:34.948911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:04:34.948911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:04:34.948911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:34.948911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:04:34.948911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:04:34.948911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:34.948911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:04:34.948911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:34.948911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:04:34.948911 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_network_type_and_physnet {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:04:34.948911 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set network type and physnet external_ids on logical switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:04:34.952915 np0000173183 neutron-ovn-maintenance-worker[105581]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_network_type_and_physnet finished in 0.006 seconds
May 09 14:04:34.953063 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_network_type_and_physnet' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105581) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 09 14:04:34.953209 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_ovn_owned_dns_option {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:04:34.953370 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply ovn_owned option on DNS records per configuration. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:04:34.953887 np0000173183 neutron-ovn-maintenance-worker[105581]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_ovn_owned_dns_option finished in 0.000 seconds
May 09 14:04:34.954013 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_ovn_owned_dns_option' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105581) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 09 14:04:34.954151 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:04:34.954273 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:04:34.954469 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:04:34.955045 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:04:34.955382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:34.956120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:04:34.956120 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:34.956120 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:04:34.956120 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:04:34.956120 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:34.956120 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:04:34.956120 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:34.956120 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:04:34.956884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:34.956884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:04:34.956884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:04:34.956884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:04:34.956884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:34.956884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:04:34.956884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:04:34.956884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:04:34.956884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:34.956884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:04:34.956884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:04:34.956884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:04:34.956884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:04:34.956884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:04:34.956884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:04:34.957477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:04:34.957477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:04:34.957477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:34.957477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:04:34.957477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:04:34.957477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:34.957477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:04:34.957477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:34.957477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:04:34.957855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:34.957855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:04:34.957855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:04:34.957855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:04:34.957855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:04:34.957855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:04:34.957855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:04:34.957855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:04:34.957855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:04:34.957855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:34.957855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:04:34.958269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:04:34.958269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:04:34.958269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:34.958269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:04:34.958269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:04:34.958269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:04:34.958269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:04:34.958269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:04:34.958269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:04:34.958269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:04:34.958269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:04:34.958269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:34.958269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:04:34.958269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:04:34.958816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:34.958816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:04:34.958816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:34.958816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:04:34.958816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call immediate 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:34.958816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:04:34.958816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:04:34.958816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:04:34.958816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:04:34.958816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:04:34.958816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:04:34.958816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:04:34.958816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:04:34.958816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:04:34.959343 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:04:34.959343 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:04:34.959343 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:04:34.959343 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:04:34.959343 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:04:34.959343 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:04:34.959343 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:04:34.959343 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:04:34.959343 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:04:34.959343 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:04:34.959343 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:34.959343 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:04:34.959343 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:04:34.959343 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:04:34.959343 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:34.959343 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:04:34.959343 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:04:34.959343 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:04:34.959343 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:04:34.960179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:04:34.960179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:04:34.960179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:04:34.960179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:04:34.960179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:34.960179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:04:34.960179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:04:34.960179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:34.960179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:04:34.960179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:34.960179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:04:34.960179 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_lrouter_ports_ext_ids_name_prefix {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:04:34.960179 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: ('Check missing prefix router_name', 'in external_ids of LRPs.') {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:04:34.961125 np0000173183 neutron-ovn-maintenance-worker[105581]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_lrouter_ports_ext_ids_name_prefix finished in 0.001 seconds
May 09 14:04:34.961125 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_lrouter_ports_ext_ids_name_prefix' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105581) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 09 14:04:34.961125 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:04:34.961125 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:04:34.961125 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:04:34.961642 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:04:34.961642 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:04:34.961642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:34.962269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:04:34.962269 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:34.962269 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:04:34.962269 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:04:34.962269 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:34.962269 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:04:34.962269 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:34.962269 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:04:34.963087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:34.963087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:04:34.963087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:04:34.963087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:04:34.963087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:04:34.963087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:04:34.963087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:04:34.963087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:34.963087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:04:34.963087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:04:34.963087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:04:34.963087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:34.963933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:04:34.963933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:04:34.963933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:04:34.963933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:04:34.963933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:04:34.963933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:04:34.963933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:04:34.963933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:04:34.963933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:34.963933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:04:34.963933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:04:34.963933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:34.963933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:04:34.965049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:34.965049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:04:34.965049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call immediate 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:34.965049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:04:34.965049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:04:34.965049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:04:34.965049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:04:34.965049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:04:34.965049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:04:34.965049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:04:34.965049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:04:34.965049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:04:34.965049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:04:34.965049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:04:34.965049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:04:34.966232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:04:34.966232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:04:34.966232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:04:34.966232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:04:34.966232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:04:34.966232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:34.966232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:04:34.966232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:04:34.966232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:04:34.966232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:34.966232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:04:34.966232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:04:34.966232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:04:34.966232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:04:34.966232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:04:34.966232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:04:34.966232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:04:34.966232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:04:34.966232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:34.967705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:04:34.967705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:04:34.967705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:34.967705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:04:34.967705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:34.967705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:04:34.967705 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_router_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:04:34.967705 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Sync router.distributed with distributed floating IP config. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:04:34.967705 np0000173183 neutron-ovn-maintenance-worker[105581]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_router_distributed_flag finished in 0.003 seconds
May 09 14:04:34.967705 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_router_distributed_flag' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105581) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 09 14:04:34.968716 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_router_static_routes {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:04:34.968716 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Mark Neutron-owned static routes in OVN external_ids. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:04:34.968896 np0000173183 neutron-ovn-maintenance-worker[105581]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_router_static_routes finished in 0.001 seconds
May 09 14:04:34.969033 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_router_static_routes' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105581) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 09 14:04:34.969149 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_security_group_with_address_group {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:04:34.969368 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Repair Address_Sets and ACLs for address group rules. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:04:34.981041 np0000173183 neutron-ovn-maintenance-worker[105581]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_security_group_with_address_group finished in 0.012 seconds
May 09 14:04:34.981142 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_security_group_with_address_group' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105581) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 09 14:04:34.982081 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_virtual_port_parent_hostname' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105581) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 09 14:04:37.019636 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched DELETE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(uuid=acfe575c-ed78-4a59-999b-ee78ebd1122f, enabled=[], external_ids={'neutron:is_ext_gw': 'True', 'neutron:network_name': 'neutron-a7d094cd-23f3-4e0c-9e88-24be6ee17b2c', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13', 'neutron:subnet_ids': '2f0821e3-c6a2-49d6-80ee-3480a17ba8d2'}, gateway_chassis=[], ha_chassis_group=118bdf94-5fb8-4d43-9b60-a1f2af976c3d, ipv6_prefix=[], ipv6_ra_configs={}, mac='fa:16:3e:77:46:7f', name='lrp-e7178386-e89c-4b8b-94b7-02be25736637', networks=['172.24.5.191/24'], options={'gateway_mtu': '1372', 'reside-on-redirect-chassis': 'true'}, peer=[], status={'hosting-chassis': '4524ca43-0613-40ef-9118-5c14197adb05'}) old= {{(pid=105581) matches /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}}
May 09 14:04:37.020664 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(uuid=aec6cc37-9de3-4d90-afa0-28c17da3368e, enabled=[], external_ids={'neutron:is_ext_gw': 'True', 'neutron:network_name': 'neutron-a7d094cd-23f3-4e0c-9e88-24be6ee17b2c', 'neutron:revision_number': '10', 'neutron:router_name': 'neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13', 'neutron:subnet_ids': '2f0821e3-c6a2-49d6-80ee-3480a17ba8d2 644269b7-6b84-4d8a-ad28-d59a7c6b7c5e'}, gateway_chassis=[], ha_chassis_group=118bdf94-5fb8-4d43-9b60-a1f2af976c3d, ipv6_prefix=[], ipv6_ra_configs={}, mac='fa:16:3e:77:46:7f', name='lrp-e7178386-e89c-4b8b-94b7-02be25736637', networks=['172.24.5.191/24', '2001:db8::36/64'], options={'gateway_mtu': '1372', 'reside-on-redirect-chassis': 'true'}, peer=[], status={}) old= {{(pid=105581) matches /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}}
May 09 14:04:37.056468 np0000173183 neutron-ovn-maintenance-worker[105581]: INFO neutron.common.ovn.utils [-] HA Chassis Group neutron-0b4a5724-2518-4e33-a6d3-0d92c749f303 synchronized; highest priority chassis 4524ca43-0613-40ef-9118-5c14197adb05
May 09 14:04:37.057094 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=HA_Chassis_Group, record=neutron-0b4a5724-2518-4e33-a6d3-0d92c749f303, col_values=(('external_ids', {'neutron:availability_zone_hints': '', 'neutron:network_id': '0b4a5724-2518-4e33-a6d3-0d92c749f303', 'neutron:router_id': '5dcc5c62-3db6-43c1-9b38-e1c7de943f13'}),), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:04:37.057588 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): HAChassisGroupAddChassisCommand(_result=None, hcg=neutron-0b4a5724-2518-4e33-a6d3-0d92c749f303, chassis=4524ca43-0613-40ef-9118-5c14197adb05, priority=1, columns={}) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:04:37.057874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:37.058509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
May 09 14:04:37.058509 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:37.058509 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:04:37.058509 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:04:37.058509 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:37.058509 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:04:37.058509 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:37.058509 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:04:37.059690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event [-] Unexpected exception in notify_loop: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:37.059690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event Traceback (most recent call last):
May 09 14:04:37.059690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/event.py", line 185, in notify_loop
May 09 14:04:37.059690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event     match.run(event, row, updates)
May 09 14:04:37.059690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/services/ovn_l3/ovsdb_monitor.py", line 81, in run
May 09 14:04:37.059690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event     self.l3_plugin._ovn_client.update_router_ha_chassis_group(
May 09 14:04:37.059690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1665, in update_router_ha_chassis_group
May 09 14:04:37.059690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event     self.link_network_ha_chassis_group(context, network_id, router_id)
May 09 14:04:37.059690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2416, in link_network_ha_chassis_group
May 09 14:04:37.059690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:04:37.059690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:37.059690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event     next(self.gen)
May 09 14:04:37.059690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:04:37.059690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event     with super().transaction(*args, **kwargs) as t:
May 09 14:04:37.059690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:37.059690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event     next(self.gen)
May 09 14:04:37.059690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:04:37.059690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event     with self.create_transaction(
May 09 14:04:37.060490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:04:37.060490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event     self.result = self.commit()
May 09 14:04:37.060490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event                   ^^^^^^^^^^^^^
May 09 14:04:37.060490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:04:37.060490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event     raise result.ex
May 09 14:04:37.060490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:37.060490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event     txn.results.put(txn.do_commit())
May 09 14:04:37.060490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event                     ^^^^^^^^^^^^^^^
May 09 14:04:37.060490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:37.060490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event     raise RuntimeError(msg)
May 09 14:04:37.060490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:37.060490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event 
May 09 14:04:37.094090 np0000173183 neutron-ovn-maintenance-worker[105581]: INFO neutron.common.ovn.utils [-] HA Chassis Group neutron-0b4a5724-2518-4e33-a6d3-0d92c749f303 synchronized; highest priority chassis 4524ca43-0613-40ef-9118-5c14197adb05
May 09 14:04:37.094690 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=HA_Chassis_Group, record=neutron-0b4a5724-2518-4e33-a6d3-0d92c749f303, col_values=(('external_ids', {'neutron:availability_zone_hints': '', 'neutron:network_id': '0b4a5724-2518-4e33-a6d3-0d92c749f303', 'neutron:router_id': '5dcc5c62-3db6-43c1-9b38-e1c7de943f13'}),), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:04:37.095037 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): HAChassisGroupAddChassisCommand(_result=None, hcg=neutron-0b4a5724-2518-4e33-a6d3-0d92c749f303, chassis=4524ca43-0613-40ef-9118-5c14197adb05, priority=1, columns={}) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:04:37.095301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:37.095976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
May 09 14:04:37.095976 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:37.095976 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:04:37.095976 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:04:37.095976 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:37.095976 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:04:37.095976 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:37.095976 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:04:37.097640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event [-] Unexpected exception in notify_loop: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:37.097640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event Traceback (most recent call last):
May 09 14:04:37.097640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/event.py", line 185, in notify_loop
May 09 14:04:37.097640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event     match.run(event, row, updates)
May 09 14:04:37.097640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/services/ovn_l3/ovsdb_monitor.py", line 89, in run
May 09 14:04:37.097640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event     self.l3_plugin._ovn_client.update_router_ha_chassis_group(
May 09 14:04:37.097640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1665, in update_router_ha_chassis_group
May 09 14:04:37.097640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event     self.link_network_ha_chassis_group(context, network_id, router_id)
May 09 14:04:37.097640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2416, in link_network_ha_chassis_group
May 09 14:04:37.097640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:04:37.097640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:37.097640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event     next(self.gen)
May 09 14:04:37.097640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:04:37.097640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event     with super().transaction(*args, **kwargs) as t:
May 09 14:04:37.097640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:37.097640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event     next(self.gen)
May 09 14:04:37.097640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:04:37.097640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event     with self.create_transaction(
May 09 14:04:37.098424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:04:37.098424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event     self.result = self.commit()
May 09 14:04:37.098424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event                   ^^^^^^^^^^^^^
May 09 14:04:37.098424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:04:37.098424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event     raise result.ex
May 09 14:04:37.098424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:37.098424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event     txn.results.put(txn.do_commit())
May 09 14:04:37.098424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event                     ^^^^^^^^^^^^^^^
May 09 14:04:37.098424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:37.098424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event     raise RuntimeError(msg)
May 09 14:04:37.098424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:37.098424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event 
May 09 14:04:39.197737 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:04:39.198542 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:04:39.198692 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:04:39.200474 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:04:39.230131 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:04:39.230301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:39.230746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:04:39.230746 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:39.230746 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:04:39.230746 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:04:39.230746 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:39.230746 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:04:39.230746 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:39.230746 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:04:39.231480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:39.231480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:04:39.231480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:04:39.231480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:04:39.231480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:04:39.231480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:04:39.231480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:04:39.231480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:39.231480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:04:39.231480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:04:39.231480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:04:39.231480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:39.232375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:04:39.232375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:04:39.232375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:04:39.232375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:04:39.232375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:04:39.232375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:04:39.232375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:04:39.232375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:04:39.232375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:39.232375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:04:39.232375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:04:39.232375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:39.232375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:04:39.233033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:39.233033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:04:39.233033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:39.233033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:04:39.233033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:04:39.233033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:04:39.233033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:04:39.233033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:04:39.233033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:04:39.233033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:04:39.233033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:04:39.233033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:04:39.233033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:04:39.233033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:04:39.233033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:04:39.233617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:04:39.233617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:04:39.233617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:04:39.233617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:04:39.233617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:04:39.233617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:39.233617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:04:39.233617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:04:39.233617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:04:39.233617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:39.233617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:04:39.233617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:04:39.233617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:04:39.233617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:04:39.233617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:04:39.233617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:04:39.233617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:04:39.233617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:04:39.233617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:39.234337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:04:39.234337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:04:39.234337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:39.234337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:04:39.234337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:39.234337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:04:39.922697 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:04:39.923113 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:04:39.923204 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:04:39.924191 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:04:39.924494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:39.925307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:04:39.925307 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:39.925307 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:04:39.925307 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:04:39.925307 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:39.925307 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:04:39.925307 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:39.925307 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:04:39.926085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:39.926085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:04:39.926085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:04:39.926085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:04:39.926085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:04:39.926085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:04:39.926085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:04:39.926085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:39.926085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:04:39.926085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:04:39.926085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:04:39.926085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:39.926704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:04:39.926704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:04:39.926704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:04:39.926704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:04:39.926704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:04:39.926704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:04:39.926704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:04:39.926704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:04:39.926704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:39.926704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:04:39.926704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:04:39.926704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:39.926704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:04:39.927249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:39.927249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:04:39.927249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:39.927249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:04:39.927249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:04:39.927249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:04:39.927249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:04:39.927249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:04:39.927249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:04:39.927249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:04:39.927249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:04:39.927249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:04:39.927249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:04:39.927249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:04:39.927249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:04:39.927988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:04:39.927988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:04:39.927988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:04:39.927988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:04:39.927988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:04:39.927988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:39.927988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:04:39.927988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:04:39.927988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:04:39.927988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:39.927988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:04:39.927988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:04:39.927988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:04:39.927988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:04:39.927988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:04:39.927988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:04:39.927988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:04:39.927988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:04:39.927988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:39.928837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:04:39.928837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:04:39.928837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:39.928837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:04:39.928837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:39.928837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:04:39.942007 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:04:39.942268 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:04:39.942351 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:04:39.942432 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:04:39.942881 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:04:39.943176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:39.943877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:04:39.943877 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:39.943877 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:04:39.943877 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:04:39.943877 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:39.943877 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:04:39.943877 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:39.943877 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:04:39.944423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:39.944423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:04:39.944423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:04:39.944423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:04:39.944423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:39.944423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:04:39.944423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:04:39.944423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:04:39.944423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:39.944423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:04:39.944423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:04:39.944423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:04:39.944423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:04:39.944423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:04:39.944423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:04:39.945097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:04:39.945097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:04:39.945097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:39.945097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:04:39.945097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:04:39.945097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:39.945097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:04:39.945097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:39.945097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:04:39.945097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:39.945097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:04:39.945097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:04:39.945097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:04:39.945097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:04:39.945753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:04:39.945753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:04:39.945753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:04:39.945753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:04:39.945753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:04:39.945753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:39.945753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:04:39.945753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:04:39.945753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:04:39.945753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:39.945753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:04:39.945753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:04:39.945753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:04:39.946332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:04:39.946332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:04:39.946332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:04:39.946332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:04:39.946332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:04:39.946332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:39.946332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:04:39.946332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:04:39.946332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:39.946332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:04:39.946332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:39.946332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:04:39.946898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:39.946898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:04:39.946898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:04:39.946898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:04:39.946898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:04:39.946898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:04:39.946898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:04:39.946898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:04:39.946898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:04:39.946898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:04:39.946898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:04:39.946898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:04:39.946898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:04:39.946898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:04:39.946898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:04:39.946898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:04:39.947613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:04:39.947613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:04:39.947613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:04:39.947613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:04:39.947613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:04:39.947613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:39.947613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:04:39.947613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:04:39.947613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:04:39.947613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:39.947613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:04:39.947613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:04:39.947613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:04:39.947613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:04:39.947613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:04:39.947613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:04:39.947613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:04:39.947613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:04:39.948448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:39.948448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:04:39.948448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:04:39.948448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:39.948448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:04:39.948448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:39.948448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:04:39.954392 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:04:39.954705 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:04:39.954810 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:04:39.954934 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:04:39.955395 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:04:39.955790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:39.956488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:04:39.956488 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:39.956488 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:04:39.956488 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:04:39.956488 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:39.956488 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:04:39.956488 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:39.956488 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:04:39.957485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:39.957485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:04:39.957485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:04:39.957485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:04:39.957485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:39.957485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:04:39.957485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:04:39.957485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:04:39.957485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:39.957485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:04:39.957485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:04:39.957485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:04:39.957485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:04:39.957485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:04:39.957485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:04:39.957891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:04:39.957891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:04:39.957891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:39.957891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:04:39.957891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:04:39.957891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:39.957891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:04:39.957891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:39.957891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:04:39.958833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:39.958833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:04:39.958833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:04:39.958833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:04:39.958833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:04:39.958833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:04:39.958833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:04:39.958833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:04:39.958833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:04:39.958833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:39.958833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:04:39.959160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:04:39.959160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:04:39.959160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:39.959160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:04:39.959160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:04:39.959160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:04:39.959160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:04:39.959160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:04:39.959160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:04:39.959160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:04:39.959160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:04:39.959160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:39.959160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:04:39.959160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:04:39.959563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:39.959563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:04:39.959563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:39.959563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:04:39.960635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:39.960635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:04:39.960635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:04:39.960635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:04:39.960635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:04:39.960635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:04:39.960635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:04:39.960635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:04:39.960635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:04:39.960635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:04:39.960635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:04:39.960635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:04:39.960635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:04:39.960635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:04:39.960635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:04:39.960635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:04:39.960635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:04:39.961099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:04:39.961099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:04:39.961099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:04:39.961099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:39.961099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:04:39.961099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:04:39.961099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:04:39.961099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:39.961099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:04:39.961099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:04:39.961099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:04:39.961099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:04:39.961099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:04:39.961099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:04:39.961099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:04:39.961099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:04:39.961099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:39.961099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:04:39.961099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:04:39.961625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:39.961625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:04:39.961625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:39.961625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:04:39.961625 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:04:39.961625 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:04:39.961625 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:04:39.961922 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:04:39.962155 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:04:39.962517 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:04:39.962701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:39.963604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:04:39.963604 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:39.963604 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:04:39.963604 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:04:39.963604 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:39.963604 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:04:39.963604 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:39.963604 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:04:39.964938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:39.964938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:04:39.964938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:04:39.964938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:04:39.964938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:04:39.964938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:04:39.964938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:04:39.964938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:39.964938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:04:39.964938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:04:39.964938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:04:39.964938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:39.965467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:04:39.965467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:04:39.965467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:04:39.965467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:04:39.965467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:04:39.965467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:04:39.965467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:04:39.965467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:04:39.965467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:39.965467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:04:39.965467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:04:39.965467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:39.965467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:04:39.966036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:39.966036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:04:39.966438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:39.966438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:04:39.966438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:04:39.966438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:04:39.966438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:04:39.966438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:04:39.966438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:04:39.966438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:04:39.966438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:04:39.966438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:04:39.966438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:04:39.966438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:04:39.966438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:04:39.966438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:04:39.966438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:04:39.966438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:04:39.967280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:04:39.967280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:04:39.967280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:39.967280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:04:39.967280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:04:39.967280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:04:39.967280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:39.967280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:04:39.967280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:04:39.967280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:04:39.967280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:04:39.967280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:04:39.967280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:04:39.967280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:04:39.967280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:04:39.967280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:39.967280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:04:39.967280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:04:39.967280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:39.968126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:04:39.968126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:39.968126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:04:44.198765 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:04:44.199364 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:04:44.199458 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:04:44.200694 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:04:44.226226 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:04:44.226646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:44.227429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:04:44.227429 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:44.227429 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:04:44.227429 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:04:44.227429 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:44.227429 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:04:44.227429 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:44.227429 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:04:44.228242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.028 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:44.228242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:04:44.228242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:04:44.228242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:04:44.228242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:04:44.228242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:04:44.228242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:04:44.228242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:44.228242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:04:44.228242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:04:44.228242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:04:44.228242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:44.228847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:04:44.228847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:04:44.228847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:04:44.228847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:04:44.228847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:04:44.228847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:04:44.228847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:04:44.228847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:04:44.228847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:44.228847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:04:44.228847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:04:44.228847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:44.228847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:04:44.229460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:44.229460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:04:44.229460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:44.229460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:04:44.229460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:04:44.229460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:04:44.229460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:04:44.229460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:04:44.229460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:04:44.229460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:04:44.229460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:04:44.229460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:04:44.229460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:04:44.229460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:04:44.229460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:04:44.230138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:04:44.230138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:04:44.230138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:04:44.230138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:04:44.230138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:04:44.230138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:44.230138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:04:44.230138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:04:44.230138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:04:44.230138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:44.230138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:04:44.230138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:04:44.230138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:04:44.230138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:04:44.230138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:04:44.230138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:04:44.230138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:04:44.230138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:04:44.230138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:44.230975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:04:44.230975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:04:44.230975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:44.230975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:04:44.230975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:44.230975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:04:44.923449 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:04:44.927447 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:04:44.927447 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:04:44.929078 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:04:44.929078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:44.930789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:04:44.930789 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:44.930789 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:04:44.930789 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:04:44.930789 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:44.930789 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:04:44.930789 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:44.930789 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:04:44.930789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:44.930789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:04:44.930789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:04:44.930789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:04:44.930789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:04:44.930789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:04:44.930789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:04:44.931121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:44.931121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:04:44.931121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:04:44.931121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:04:44.931121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:44.931121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:04:44.931121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:04:44.931121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:04:44.931121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:04:44.931121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:04:44.931121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:04:44.931121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:04:44.931121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:04:44.931121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:44.931438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:04:44.931438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:04:44.931438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:44.931438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:04:44.931438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:44.931438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:04:44.931619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:44.931619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:04:44.931619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:04:44.931619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:04:44.931619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:04:44.931619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:04:44.931619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:04:44.931619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:04:44.931619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:04:44.931619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:04:44.931619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:04:44.931619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:04:44.931619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:04:44.931619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:04:44.931619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:04:44.931619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:04:44.931945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:04:44.931945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:04:44.931945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:44.931945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:04:44.931945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:04:44.931945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:04:44.931945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:44.931945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:04:44.931945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:04:44.931945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:04:44.931945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:04:44.931945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:04:44.931945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:04:44.931945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:04:44.931945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:04:44.931945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:44.931945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:04:44.931945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:04:44.931945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:44.932309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:04:44.932309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:44.932309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:04:44.942639 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:04:44.943119 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:04:44.943248 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:04:44.943373 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:04:44.943793 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:04:44.943955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:44.944387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:04:44.944387 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:44.944387 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:04:44.944387 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:04:44.944387 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:44.944387 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:04:44.944387 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:44.944387 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:04:44.945355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:44.945355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:04:44.945355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:04:44.945355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:04:44.945355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:44.945355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:04:44.945355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:04:44.945355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:04:44.945355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:44.945355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:04:44.945355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:04:44.945355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:04:44.945355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:04:44.945355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:04:44.945355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:04:44.945784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:04:44.945784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:04:44.945784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:44.945784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:04:44.945784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:04:44.945784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:44.945784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:04:44.945784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:44.945784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:04:44.946300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:44.946300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:04:44.946300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:04:44.946300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:04:44.946300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:04:44.946300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:04:44.946300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:04:44.946300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:04:44.946300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:04:44.946300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:04:44.946300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:44.946300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:04:44.946670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:04:44.946670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:04:44.946670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:44.946670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:04:44.946670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:04:44.946670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:04:44.946670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:04:44.946670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:04:44.946670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:04:44.946670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:04:44.946670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:04:44.946670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:44.946670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:04:44.946670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:04:44.947011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:44.947011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:04:44.947011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:44.947011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:04:44.947399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:44.947399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:04:44.947399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:04:44.947399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:04:44.947399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:04:44.947399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:04:44.947399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:04:44.947399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:04:44.947399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:04:44.947399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:04:44.947399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:04:44.947399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:04:44.947399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:04:44.947399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:04:44.947399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:04:44.947399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:04:44.947840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:04:44.947840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:04:44.947840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:04:44.947840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:04:44.947840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:04:44.947840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:44.947840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:04:44.947840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:04:44.947840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:04:44.947840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:44.947840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:04:44.947840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:04:44.947840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:04:44.947840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:04:44.947840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:04:44.947840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:04:44.947840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:04:44.947840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:04:44.948275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:44.948275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:04:44.948275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:04:44.948275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:44.948275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:04:44.948275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:44.948275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:04:44.954865 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:04:44.957141 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:04:44.957337 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:04:44.957577 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:04:44.958092 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:04:44.958275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:44.958846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:04:44.958846 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:44.958846 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:04:44.958846 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:04:44.958846 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:44.958846 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:04:44.958846 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:44.958846 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:04:44.960154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:44.960154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:04:44.960154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:04:44.960154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:04:44.960154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:44.960154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:04:44.960154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:04:44.960154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:04:44.960154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:44.960154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:04:44.960154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:04:44.960154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:04:44.960154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:04:44.960154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:04:44.960154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:04:44.960903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:04:44.960903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:04:44.960903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:44.960903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:04:44.960903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:04:44.960903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:44.960903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:04:44.960903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:44.960903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:04:44.961649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:44.961649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:04:44.961649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:04:44.961649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:04:44.961649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:04:44.961649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:04:44.961649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:04:44.961649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:04:44.961649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:04:44.961649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:44.961649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:04:44.962141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:04:44.962141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:04:44.962141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:44.962141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:04:44.962141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:04:44.962141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:04:44.962141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:04:44.962141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:04:44.962141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:04:44.962141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:04:44.962141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:04:44.962141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:44.962141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:04:44.962141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:04:44.962797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:44.962797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:04:44.962797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:44.962797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:04:44.963385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:44.963385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:04:44.963385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:04:44.963385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:04:44.963385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:04:44.963385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:04:44.963385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:04:44.963385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:04:44.963385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:04:44.963385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:04:44.963385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:04:44.963385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:04:44.963385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:04:44.963385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:04:44.963385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:04:44.963385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:04:44.963385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:04:44.964515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:04:44.964515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:04:44.964515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:04:44.964515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:44.964515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:04:44.964515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:04:44.964515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:04:44.964515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:44.964515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:04:44.964515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:04:44.964515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:04:44.964515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:04:44.964515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:04:44.964515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:04:44.964515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:04:44.964515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:04:44.964515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:44.964515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:04:44.964515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:04:44.965620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:44.965620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:04:44.965620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:44.965620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:04:44.965620 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:04:44.965620 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:04:44.965620 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:04:44.965620 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:04:44.965999 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:04:44.965999 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:04:44.965999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:44.966329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:04:44.966329 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:44.966329 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:04:44.966329 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:04:44.966329 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:44.966329 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:04:44.966329 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:44.966329 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:04:44.967599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:44.967599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:04:44.967599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:04:44.967599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:04:44.967599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:04:44.967599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:04:44.967599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:04:44.967599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:44.967599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:04:44.967599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:04:44.967599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:04:44.967599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:44.967976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:04:44.967976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:04:44.967976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:04:44.967976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:04:44.967976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:04:44.967976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:04:44.967976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:04:44.967976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:04:44.967976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:44.967976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:04:44.967976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:04:44.967976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:44.967976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:04:44.968332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:44.968332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:04:44.969189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:44.969189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:04:44.969189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:04:44.969189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:04:44.969189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:04:44.969189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:04:44.969189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:04:44.969189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:04:44.969189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:04:44.969189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:04:44.969189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:04:44.969189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:04:44.969189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:04:44.969189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:04:44.969189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:04:44.969189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:04:44.969898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:04:44.969898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:04:44.969898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:44.969898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:04:44.969898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:04:44.969898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:04:44.969898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:44.969898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:04:44.969898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:04:44.969898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:04:44.969898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:04:44.969898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:04:44.969898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:04:44.969898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:04:44.969898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:04:44.969898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:44.969898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:04:44.969898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:04:44.969898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:44.970735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:04:44.970735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:44.970735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:04:49.199524 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:04:49.200282 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:04:49.200385 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:04:49.201671 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:04:49.227325 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:04:49.227770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:49.228714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:04:49.228714 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:49.228714 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:04:49.228714 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:04:49.228714 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:49.228714 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:04:49.228714 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:49.228714 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:04:49.229666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.028 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:49.229666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:04:49.229666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:04:49.229666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:04:49.229666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:04:49.229666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:04:49.229666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:04:49.229666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:49.229666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:04:49.229666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:04:49.229666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:04:49.229666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:49.230281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:04:49.230281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:04:49.230281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:04:49.230281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:04:49.230281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:04:49.230281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:04:49.230281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:04:49.230281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:04:49.230281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:49.230281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:04:49.230281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:04:49.230281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:49.230281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:04:49.230945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:49.230945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:04:49.230945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:49.230945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:04:49.230945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:04:49.230945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:04:49.230945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:04:49.230945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:04:49.230945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:04:49.230945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:04:49.230945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:04:49.230945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:04:49.230945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:04:49.230945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:04:49.230945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:04:49.231690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:04:49.231690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:04:49.231690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:04:49.231690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:04:49.231690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:04:49.231690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:49.231690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:04:49.231690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:04:49.231690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:04:49.231690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:49.231690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:04:49.231690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:04:49.231690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:04:49.231690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:04:49.231690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:04:49.231690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:04:49.231690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:04:49.231690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:04:49.231690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:49.232579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:04:49.232579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:04:49.232579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:49.232579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:04:49.232579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:49.232579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:04:49.925039 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:04:49.925559 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:04:49.925757 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:04:49.926942 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:04:49.927382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:49.928359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:04:49.928359 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:49.928359 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:04:49.928359 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:04:49.928359 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:49.928359 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:04:49.928359 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:49.928359 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:04:49.929975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:49.929975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:04:49.929975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:04:49.929975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:04:49.929975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:04:49.929975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:04:49.929975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:04:49.929975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:49.929975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:04:49.929975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:04:49.929975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:04:49.929975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:49.930620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:04:49.930620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:04:49.930620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:04:49.930620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:04:49.930620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:04:49.930620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:04:49.930620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:04:49.930620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:04:49.930620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:49.930620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:04:49.930620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:04:49.930620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:49.930620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:04:49.931171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:49.931171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:04:49.932256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:49.932256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:04:49.932256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:04:49.932256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:04:49.932256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:04:49.932256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:04:49.932256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:04:49.932256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:04:49.932256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:04:49.932256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:04:49.932256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:04:49.932256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:04:49.932256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:04:49.932256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:04:49.932256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:04:49.932256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:04:49.932976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:04:49.932976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:04:49.932976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:49.932976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:04:49.932976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:04:49.932976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:04:49.932976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:49.932976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:04:49.932976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:04:49.932976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:04:49.932976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:04:49.932976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:04:49.932976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:04:49.932976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:04:49.932976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:04:49.932976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:49.932976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:04:49.932976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:04:49.932976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:49.933821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:04:49.933821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:49.933821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:04:49.943144 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:04:49.943445 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:04:49.943580 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:04:49.943703 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:04:49.944187 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:04:49.944534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:49.945286 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:04:49.945286 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:49.945286 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:04:49.945286 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:04:49.945286 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:49.945286 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:04:49.945286 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:49.945286 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:04:49.946565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:49.946565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:04:49.946565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:04:49.946565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:04:49.946565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:49.946565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:04:49.946565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:04:49.946565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:04:49.946565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:49.946565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:04:49.946565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:04:49.946565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:04:49.946565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:04:49.946565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:04:49.946565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:04:49.947267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:04:49.947267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:04:49.947267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:49.947267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:04:49.947267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:04:49.947267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:49.947267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:04:49.947267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:49.947267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:04:49.947698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:49.947698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:04:49.947698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:04:49.947698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:04:49.947698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:04:49.947698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:04:49.947698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:04:49.947698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:04:49.947698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:04:49.947698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:04:49.947698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:49.947698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:04:49.948243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:04:49.948243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:04:49.948243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:49.948243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:04:49.948243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:04:49.948243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:04:49.948243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:04:49.948243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:04:49.948243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:04:49.948243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:04:49.948243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:04:49.948243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:49.948243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:04:49.948243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:04:49.948862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:49.948862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:04:49.948862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:49.948862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:04:49.948862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:49.948862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:04:49.948862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:04:49.948862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:04:49.948862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:04:49.948862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:04:49.948862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:04:49.948862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:04:49.948862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:04:49.948862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:04:49.949483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:04:49.949483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:04:49.949483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:04:49.949483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:04:49.949483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:04:49.949483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:04:49.949483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:04:49.949483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:04:49.949483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:04:49.949483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:04:49.949483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:04:49.949483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:49.949483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:04:49.949483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:04:49.949483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:04:49.949483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:49.949483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:04:49.949483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:04:49.949483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:04:49.950300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:04:49.950300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:04:49.950300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:04:49.950300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:04:49.950300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:04:49.950300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:49.950300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:04:49.950300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:04:49.950300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:49.950300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:04:49.950300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:49.950300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:04:49.956956 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:04:49.957290 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:04:49.957523 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:04:49.957730 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:04:49.958229 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:04:49.958656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:49.959431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:04:49.959431 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:49.959431 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:04:49.959431 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:04:49.959431 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:49.959431 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:04:49.959431 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:49.959431 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:04:49.960727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:49.960727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:04:49.960727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:04:49.960727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:04:49.960727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:49.960727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:04:49.960727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:04:49.960727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:04:49.960727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:49.960727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:04:49.960727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:04:49.960727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:04:49.960727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:04:49.960727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:04:49.960727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:04:49.961371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:04:49.961371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:04:49.961371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:49.961371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:04:49.961371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:04:49.961371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:49.961371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:04:49.961371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:49.961371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:04:49.962006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:49.962006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:04:49.962006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:04:49.962006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:04:49.962006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:04:49.962006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:04:49.962006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:04:49.962006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:04:49.962006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:04:49.962006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:49.962006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:04:49.962544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:04:49.962544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:04:49.962544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:49.962544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:04:49.962544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:04:49.962544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:04:49.962544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:04:49.962544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:04:49.962544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:04:49.962544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:04:49.962544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:04:49.962544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:49.962544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:04:49.962544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:04:49.963169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:49.963169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:04:49.963169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:49.963169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:04:49.963789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:49.963789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:04:49.963789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:04:49.963789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:04:49.963789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:04:49.963789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:04:49.963789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:04:49.963789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:04:49.963789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:04:49.963789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:04:49.963789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:04:49.963789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:04:49.963789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:04:49.963789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:04:49.963789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:04:49.963789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:04:49.963789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:04:49.964599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:04:49.964599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:04:49.964599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:04:49.964599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:49.964599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:04:49.964599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:04:49.964599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:04:49.964599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:49.964599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:04:49.964599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:04:49.964599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:04:49.964599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:04:49.964599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:04:49.964599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:04:49.964599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:04:49.964599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:04:49.964599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:49.964599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:04:49.964599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:04:49.965438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:49.965438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:04:49.965438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:49.965438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:04:49.965438 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:04:49.965438 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:04:49.965438 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:04:49.965438 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:04:49.965845 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:04:49.965845 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:04:49.966055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:49.966933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:04:49.966933 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:49.966933 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:04:49.966933 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:04:49.966933 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:49.966933 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:04:49.966933 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:49.966933 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:04:49.968368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:49.968368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:04:49.968368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:04:49.968368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:04:49.968368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:04:49.968368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:04:49.968368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:04:49.968368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:49.968368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:04:49.968368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:04:49.968368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:04:49.968368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:49.968952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:04:49.968952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:04:49.968952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:04:49.968952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:04:49.968952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:04:49.968952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:04:49.968952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:04:49.968952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:04:49.968952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:49.968952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:04:49.968952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:04:49.968952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:49.968952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:04:49.969354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:49.969354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:04:49.970031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:49.970031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:04:49.970031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:04:49.970031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:04:49.970031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:04:49.970031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:04:49.970031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:04:49.970031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:04:49.970031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:04:49.970031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:04:49.970031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:04:49.970031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:04:49.970031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:04:49.970031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:04:49.970031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:04:49.970031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:04:49.970525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:04:49.970525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:04:49.970525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:49.970525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:04:49.970525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:04:49.970525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:04:49.970525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:49.970525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:04:49.970525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:04:49.970525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:04:49.970525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:04:49.970525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:04:49.970525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:04:49.970525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:04:49.970525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:04:49.970525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:49.970525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:04:49.970525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:04:49.970525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:49.971084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:04:49.971084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:49.971084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:04:54.200370 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:04:54.201003 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:04:54.201131 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:04:54.202372 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:04:54.234230 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:04:54.234486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:54.235071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:04:54.235071 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:54.235071 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:04:54.235071 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:04:54.235071 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:54.235071 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:04:54.235071 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:54.235071 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:04:54.235988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:54.235988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:04:54.235988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:04:54.235988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:04:54.235988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:04:54.235988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:04:54.235988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:04:54.235988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:54.235988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:04:54.235988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:04:54.235988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:04:54.235988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:54.236590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:04:54.236590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:04:54.236590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:04:54.236590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:04:54.236590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:04:54.236590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:04:54.236590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:04:54.236590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:04:54.236590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:54.236590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:04:54.236590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:04:54.236590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:54.236590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:04:54.237169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:54.237169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:04:54.237169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:54.237169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:04:54.237169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:04:54.237169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:04:54.237169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:04:54.237169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:04:54.237169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:04:54.237169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:04:54.237169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:04:54.237169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:04:54.237169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:04:54.237169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:04:54.237169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:04:54.237836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:04:54.237836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:04:54.237836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:04:54.237836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:04:54.237836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:04:54.237836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:54.237836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:04:54.237836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:04:54.237836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:04:54.237836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:54.237836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:04:54.237836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:04:54.237836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:04:54.237836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:04:54.237836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:04:54.237836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:04:54.237836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:04:54.237836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:04:54.237836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:54.238682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:04:54.238682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:04:54.238682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:54.238682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:04:54.238682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:54.238682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:04:54.925871 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:04:54.926251 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:04:54.926537 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:04:54.927587 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:04:54.927890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:54.928894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:04:54.928894 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:54.928894 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:04:54.928894 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:04:54.928894 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:54.928894 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:04:54.928894 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:54.928894 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:04:54.930628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:54.930628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:04:54.930628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:04:54.930628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:04:54.930628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:04:54.930628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:04:54.930628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:04:54.930628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:54.930628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:04:54.930628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:04:54.930628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:04:54.930628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:54.931496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:04:54.931496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:04:54.931496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:04:54.931496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:04:54.931496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:04:54.931496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:04:54.931496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:04:54.931496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:04:54.931496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:54.931496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:04:54.931496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:04:54.931496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:54.931496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:04:54.932446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:54.932446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:04:54.932981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:54.932981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:04:54.932981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:04:54.932981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:04:54.932981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:04:54.932981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:04:54.932981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:04:54.932981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:04:54.932981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:04:54.932981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:04:54.932981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:04:54.932981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:04:54.932981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:04:54.932981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:04:54.932981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:04:54.932981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:04:54.933792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:04:54.933792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:04:54.933792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:54.933792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:04:54.933792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:04:54.933792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:04:54.933792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:54.933792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:04:54.933792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:04:54.933792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:04:54.933792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:04:54.933792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:04:54.933792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:04:54.933792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:04:54.933792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:04:54.933792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:54.933792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:04:54.933792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:04:54.933792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:54.934664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:04:54.934664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:54.934664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:04:54.943685 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:04:54.943871 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:04:54.943981 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:04:54.944074 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:04:54.944650 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:04:54.944950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:54.945636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:04:54.945636 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:54.945636 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:04:54.945636 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:04:54.945636 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:54.945636 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:04:54.945636 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:54.945636 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:04:54.946357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:54.946357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:04:54.946357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:04:54.946357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:04:54.946357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:54.946357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:04:54.946357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:04:54.946357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:04:54.946357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:54.946357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:04:54.946357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:04:54.946357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:04:54.946357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:04:54.946357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:04:54.946357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:04:54.947120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:04:54.947120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:04:54.947120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:54.947120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:04:54.947120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:04:54.947120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:54.947120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:04:54.947120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:54.947120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:04:54.947560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:54.947560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:04:54.947560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:04:54.947560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:04:54.947560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:04:54.947560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:04:54.947560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:04:54.947560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:04:54.947560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:04:54.947560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:04:54.947560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:54.947560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:04:54.948087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:04:54.948087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:04:54.948087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:54.948087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:04:54.948087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:04:54.948087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:04:54.948087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:04:54.948087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:04:54.948087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:04:54.948087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:04:54.948087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:04:54.948087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:54.948087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:04:54.948087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:04:54.948999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:54.948999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:04:54.948999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:54.948999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:04:54.948999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:54.948999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:04:54.948999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:04:54.948999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:04:54.948999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:04:54.948999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:04:54.948999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:04:54.948999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:04:54.948999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:04:54.948999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:04:54.949623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:04:54.949623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:04:54.949623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:04:54.949623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:04:54.949623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:04:54.949623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:04:54.949623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:04:54.949623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:04:54.949623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:04:54.949623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:04:54.949623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:04:54.949623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:54.949623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:04:54.949623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:04:54.949623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:04:54.949623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:54.949623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:04:54.949623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:04:54.949623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:04:54.950468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:04:54.950468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:04:54.950468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:04:54.950468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:04:54.950468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:04:54.950468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:54.950468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:04:54.950468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:04:54.950468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:54.950468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:04:54.950468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:54.950468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:04:54.957344 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:04:54.958400 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:04:54.958530 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:04:54.958622 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:04:54.959138 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:04:54.959518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:54.960134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:04:54.960134 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:54.960134 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:04:54.960134 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:04:54.960134 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:54.960134 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:04:54.960134 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:54.960134 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:04:54.960728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:54.960728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:04:54.960728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:04:54.960728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:04:54.960728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:54.960728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:04:54.960728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:04:54.960728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:04:54.960728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:54.960728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:04:54.960728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:04:54.960728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:04:54.960728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:04:54.960728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:04:54.960728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:04:54.961428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:04:54.961428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:04:54.961428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:54.961428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:04:54.961428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:04:54.961428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:54.961428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:04:54.961428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:54.961428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:04:54.961428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:54.961428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:04:54.961428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:04:54.961428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:04:54.961428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:04:54.962056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:04:54.962056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:04:54.962056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:04:54.962056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:04:54.962056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:54.962056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:04:54.962056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:04:54.962056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:04:54.962056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:54.962056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:04:54.962056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:04:54.962056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:04:54.962056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:04:54.962666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:04:54.962666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:04:54.962666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:04:54.962666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:04:54.962666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:54.962666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:04:54.962666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:04:54.962666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:54.962666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:04:54.962666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:54.962666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:04:54.962666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:54.963200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:04:54.963200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:04:54.963200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:04:54.963200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:04:54.963200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:04:54.963200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:04:54.963200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:04:54.963200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:04:54.963200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:04:54.963200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:04:54.963200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:04:54.963200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:04:54.963200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:04:54.963200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:04:54.963200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:04:54.963200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:04:54.963200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:04:54.963200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:04:54.963200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:04:54.964109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:54.964109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:04:54.964109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:04:54.964109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:04:54.964109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:54.964109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:04:54.964109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:04:54.964109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:04:54.964109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:04:54.964109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:04:54.964109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:04:54.964109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:04:54.964109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:04:54.964109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:54.964109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:04:54.964109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:04:54.964109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:54.964109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:04:54.964109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:54.964109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:04:54.965475 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:04:54.965475 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:04:54.965475 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:04:54.965475 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:04:54.965703 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:04:54.965936 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:04:54.966104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:54.966780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:04:54.966780 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:54.966780 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:04:54.966780 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:04:54.966780 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:54.966780 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:04:54.966780 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:54.966780 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:04:54.967617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:54.967617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:04:54.967617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:04:54.967617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:04:54.967617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:04:54.967617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:04:54.967617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:04:54.967617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:54.967617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:04:54.967617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:04:54.967617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:04:54.967617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:54.968242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:04:54.968242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:04:54.968242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:04:54.968242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:04:54.968242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:04:54.968242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:04:54.968242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:04:54.968242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:04:54.968242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:54.968242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:04:54.968242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:04:54.968242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:54.968242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:04:54.968678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:54.968678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:04:54.968678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:54.968678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:04:54.968678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:04:54.968678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:04:54.968678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:04:54.968678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:04:54.968678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:04:54.968678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:04:54.968678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:04:54.968678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:04:54.968678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:04:54.968678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:04:54.968678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:04:54.969111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:04:54.969111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:04:54.969111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:04:54.969111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:04:54.969111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:04:54.969111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:54.969111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:04:54.969111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:04:54.969111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:04:54.969111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:54.969111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:04:54.969111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:04:54.969111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:04:54.969111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:04:54.969111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:04:54.969111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:04:54.969111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:04:54.969111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:04:54.969111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:54.969634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:04:54.969634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:04:54.969634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:54.969634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:04:54.969634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:54.969634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:04:59.200871 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:04:59.201562 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:04:59.201770 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:04:59.203105 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:04:59.237304 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:04:59.237725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:59.238651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:04:59.238651 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:59.238651 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:04:59.238651 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:04:59.238651 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:59.238651 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:04:59.238651 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:59.238651 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:04:59.239765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:59.239765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:04:59.239765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:04:59.239765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:04:59.239765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:04:59.239765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:04:59.239765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:04:59.239765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:59.239765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:04:59.239765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:04:59.239765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:04:59.239765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:59.240307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:04:59.240307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:04:59.240307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:04:59.240307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:04:59.240307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:04:59.240307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:04:59.240307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:04:59.240307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:04:59.240307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:59.240307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:04:59.240307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:04:59.240307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:59.240307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:04:59.240906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:59.240906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:04:59.240906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:59.240906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:04:59.240906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:04:59.240906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:04:59.240906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:04:59.240906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:04:59.240906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:04:59.240906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:04:59.240906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:04:59.240906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:04:59.240906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:04:59.240906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:04:59.240906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:04:59.241525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:04:59.241525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:04:59.241525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:04:59.241525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:04:59.241525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:04:59.241525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:59.241525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:04:59.241525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:04:59.241525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:04:59.241525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:59.241525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:04:59.241525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:04:59.241525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:04:59.241525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:04:59.241525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:04:59.241525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:04:59.241525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:04:59.241525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:04:59.241525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:59.242389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:04:59.242389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:04:59.242389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:59.242389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:04:59.242389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:59.242389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:04:59.926394 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:04:59.927076 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:04:59.927285 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:04:59.928365 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:04:59.928762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:59.929690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:04:59.929690 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:59.929690 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:04:59.929690 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:04:59.929690 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:59.929690 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:04:59.929690 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:59.929690 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:04:59.931311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:59.931311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:04:59.931311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:04:59.931311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:04:59.931311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:04:59.931311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:04:59.931311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:04:59.931311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:59.931311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:04:59.931311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:04:59.931311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:04:59.931311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:59.931612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:04:59.931612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:04:59.931612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:04:59.931612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:04:59.931612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:04:59.931612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:04:59.931612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:04:59.931612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:04:59.931612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:59.931612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:04:59.931612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:04:59.931612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:59.931612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:04:59.931868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:59.931868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:04:59.933595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:59.933595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:04:59.933595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:04:59.933595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:04:59.933595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:04:59.933595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:04:59.933595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:04:59.933595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:04:59.933595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:04:59.933595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:04:59.933595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:04:59.933595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:04:59.933595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:04:59.933595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:04:59.933595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:04:59.933595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:04:59.933981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:04:59.933981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:04:59.933981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:59.933981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:04:59.933981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:04:59.933981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:04:59.933981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:59.933981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:04:59.933981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:04:59.933981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:04:59.933981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:04:59.933981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:04:59.933981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:04:59.933981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:04:59.933981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:04:59.933981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:59.933981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:04:59.933981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:04:59.933981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:59.934349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:04:59.934349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:59.934349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:04:59.944105 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:04:59.944541 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:04:59.944723 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:04:59.944906 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:04:59.945472 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:04:59.945779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:59.946739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:04:59.946739 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:59.946739 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:04:59.946739 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:04:59.946739 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:59.946739 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:04:59.946739 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:59.946739 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:04:59.947938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:59.947938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:04:59.947938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:04:59.947938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:04:59.947938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:59.947938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:04:59.947938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:04:59.947938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:04:59.947938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:59.947938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:04:59.947938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:04:59.947938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:04:59.947938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:04:59.947938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:04:59.947938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:04:59.948242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:04:59.948242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:04:59.948242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:59.948242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:04:59.948242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:04:59.948242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:59.948242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:04:59.948242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:59.948242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:04:59.949293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:59.949293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:04:59.949293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:04:59.949293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:04:59.949293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:04:59.949293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:04:59.949293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:04:59.949293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:04:59.949293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:04:59.949293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:04:59.949293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:59.949293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:04:59.949547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:04:59.949547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:04:59.949547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:59.949547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:04:59.949547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:04:59.949547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:04:59.949547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:04:59.949547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:04:59.949547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:04:59.949547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:04:59.949547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:04:59.949547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:59.949547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:04:59.949547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:04:59.949811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:59.949811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:04:59.949811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:59.949811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:04:59.951501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:59.951501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:04:59.951501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:04:59.951501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:04:59.951501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:04:59.951501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:04:59.951501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:04:59.951501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:04:59.951501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:04:59.951501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:04:59.951501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:04:59.951501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:04:59.951501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:04:59.951501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:04:59.951501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:04:59.951501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:04:59.951812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:04:59.951812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:04:59.951812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:04:59.951812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:04:59.951812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:04:59.951812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:59.951812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:04:59.951812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:04:59.951812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:04:59.951812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:59.951812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:04:59.951812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:04:59.951812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:04:59.951812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:04:59.951812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:04:59.951812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:04:59.951812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:04:59.951812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:04:59.952139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:59.952139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:04:59.952139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:04:59.952139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:59.952139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:04:59.952139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:59.952139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:04:59.958727 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:04:59.959033 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:04:59.959152 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:04:59.959278 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:04:59.959855 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:04:59.960197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:59.960982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:04:59.960982 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:59.960982 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:04:59.960982 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:04:59.960982 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:59.960982 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:04:59.960982 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:59.960982 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:04:59.961900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:59.961900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:04:59.961900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:04:59.961900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:04:59.961900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:59.961900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:04:59.961900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:04:59.961900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:04:59.961900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:59.961900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:04:59.961900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:04:59.961900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:04:59.961900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:04:59.961900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:04:59.961900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:04:59.962182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:04:59.962182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:04:59.962182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:59.962182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:04:59.962182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:04:59.962182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:59.962182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:04:59.962182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:59.962182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:04:59.963260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:59.963260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:04:59.963260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:04:59.963260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:04:59.963260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:04:59.963260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:04:59.963260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:04:59.963260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:04:59.963260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:04:59.963260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:59.963260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:04:59.963518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:04:59.963518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:04:59.963518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:59.963518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:04:59.963518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:04:59.963518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:04:59.963518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:04:59.963518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:04:59.963518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:04:59.963518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:04:59.963518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:04:59.963518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:59.963518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:04:59.963518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:04:59.963805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:59.963805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:04:59.963805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:59.963805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:04:59.965190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:59.965190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:04:59.965190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:04:59.965190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:04:59.965190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:04:59.965190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:04:59.965190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:04:59.965190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:04:59.965190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:04:59.965190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:04:59.965190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:04:59.965190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:04:59.965190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:04:59.965190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:04:59.965190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:04:59.965190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:04:59.965190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:04:59.965520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:04:59.965520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:04:59.965520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:04:59.965520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:59.965520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:04:59.965520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:04:59.965520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:04:59.965520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:59.965520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:04:59.965520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:04:59.965520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:04:59.965520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:04:59.965520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:04:59.965520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:04:59.965520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:04:59.965520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:04:59.965520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:59.965520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:04:59.965520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:04:59.965860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:59.965860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:04:59.965860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:59.965860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:04:59.965860 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:04:59.965958 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:04:59.966088 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:04:59.966309 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:04:59.966901 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:04:59.967251 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:04:59.967456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:59.968284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:04:59.968284 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:59.968284 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:04:59.968284 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:04:59.968284 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:59.968284 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:04:59.968284 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:59.968284 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:04:59.969609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:59.969609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:04:59.969609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:04:59.969609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:04:59.969609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:04:59.969609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:04:59.969609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:04:59.969609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:59.969609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:04:59.969609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:04:59.969609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:04:59.969609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:59.969845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:04:59.969845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:04:59.969845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:04:59.969845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:04:59.969845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:04:59.969845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:04:59.969845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:04:59.969845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:04:59.969845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:59.969845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:04:59.969845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:04:59.969845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:59.969845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:04:59.970083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:59.970083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:04:59.970805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:59.970805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:04:59.970805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:04:59.970805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:04:59.970805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:04:59.970805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:04:59.970805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:04:59.970805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:04:59.970805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:04:59.970805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:04:59.970805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:04:59.970805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:04:59.970805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:04:59.970805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:04:59.970805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:04:59.970805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:04:59.971113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:04:59.971113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:04:59.971113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:59.971113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:04:59.971113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:04:59.971113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:04:59.971113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:04:59.971113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:04:59.971113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:04:59.971113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:04:59.971113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:04:59.971113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:04:59.971113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:04:59.971113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:04:59.971113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:04:59.971113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:04:59.971113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:04:59.971113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:04:59.971113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:04:59.971488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:04:59.971488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:04:59.971488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:05:04.201600 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:05:04.202231 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:05:04.202380 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:05:04.203651 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:05:04.229135 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:05:04.229543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:04.230299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:05:04.230299 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:04.230299 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:05:04.230299 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:05:04.230299 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:04.230299 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:05:04.230299 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:04.230299 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:05:04.231055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.028 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:04.231055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:05:04.231055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:04.231055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:05:04.231055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:05:04.231055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:05:04.231055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:05:04.231055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:04.231055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:04.231055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:04.231055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:05:04.231055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:04.231698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:04.231698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:04.231698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:05:04.231698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:04.231698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:05:04.231698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:05:04.231698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:04.231698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:05:04.231698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:04.231698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:05:04.231698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:05:04.231698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:04.231698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:05:04.232284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:04.232284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:05:04.232284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:04.232284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:05:04.232284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:05:04.232284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:05:04.232284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:05:04.232284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:05:04.232284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:04.232284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:05:04.232284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:05:04.232284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:05:04.232284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:05:04.232284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:05:04.232284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:04.233028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:04.233028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:05:04.233028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:05:04.233028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:05:04.233028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:05:04.233028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:04.233028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:04.233028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:04.233028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:05:04.233028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:04.233028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:04.233028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:04.233028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:05:04.233028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:04.233028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:05:04.233028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:05:04.233028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:04.233028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:05:04.233028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:04.233913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:05:04.233913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:05:04.233913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:04.233913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:05:04.233913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:04.233913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:05:04.927157 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:05:04.927923 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:05:04.928158 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:05:04.929047 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:05:04.929285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:04.930922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:05:04.930922 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:04.930922 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:05:04.930922 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:05:04.930922 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:04.930922 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:05:04.930922 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:04.930922 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:05:04.931759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:04.931759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:05:04.931759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:04.931759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:05:04.931759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:05:04.931759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:05:04.931759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:05:04.931759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:04.931759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:04.931759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:04.931759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:05:04.931759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:04.932445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:04.932445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:04.932445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:05:04.932445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:04.932445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:05:04.932445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:05:04.932445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:04.932445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:05:04.932445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:04.932445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:05:04.932445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:05:04.932445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:04.932445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:05:04.933070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:04.933070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:05:04.933612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:04.933612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:05:04.933612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:05:04.933612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:05:04.933612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:05:04.933612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:05:04.933612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:04.933612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:05:04.933612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:05:04.933612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:05:04.933612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:05:04.933612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:05:04.933612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:04.933612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:04.933612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:05:04.933612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:05:04.934447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:05:04.934447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:05:04.934447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:04.934447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:04.934447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:04.934447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:05:04.934447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:04.934447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:04.934447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:04.934447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:05:04.934447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:04.934447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:05:04.934447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:05:04.934447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:04.934447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:05:04.934447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:04.934447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:05:04.934447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:05:04.934447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:04.935548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:05:04.935548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:04.935548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:05:04.944604 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:05:04.944912 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:05:04.945009 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:05:04.945123 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:05:04.945720 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:05:04.945955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:04.946569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:05:04.946569 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:04.946569 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:05:04.946569 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:05:04.946569 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:04.946569 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:05:04.946569 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:04.946569 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:05:04.947319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:04.947319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:05:04.947319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:05:04.947319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:05:04.947319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:04.947319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:05:04.947319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:04.947319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:05:04.947319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:04.947319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:05:04.947319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:04.947319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:05:04.947319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:04.947319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:05:04.947319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:05:04.948132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:04.948132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:05:04.948132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:04.948132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:05:04.948132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:05:04.948132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:04.948132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:05:04.948132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:04.948132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:05:04.948597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:04.948597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:05:04.948597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:04.948597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:05:04.948597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:05:04.948597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:05:04.948597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:05:04.948597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:05:04.948597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:05:04.948597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:05:04.948597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:04.948597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:04.949205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:04.949205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:05:04.949205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:04.949205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:04.949205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:04.949205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:05:04.949205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:04.949205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:05:04.949205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:05:04.949205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:04.949205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:05:04.949205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:04.949205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:05:04.949205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:05:04.949990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:04.949990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:05:04.949990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:04.949990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:05:04.949990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:04.949990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:05:04.949990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:05:04.949990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:05:04.949990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:05:04.949990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:05:04.949990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:04.949990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:05:04.949990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:05:04.949990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:05:04.950740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:05:04.950740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:05:04.950740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:04.950740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:04.950740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:05:04.950740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:05:04.950740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:05:04.950740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:05:04.950740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:05:04.950740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:05:04.950740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:05:04.950740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:04.950740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:04.950740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:04.950740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:05:04.950740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:04.950740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:04.950740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:04.950740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:05:04.951688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:04.951688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:05:04.951688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:05:04.951688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:04.951688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:05:04.951688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:04.951688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:05:04.951688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:05:04.951688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:04.951688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:05:04.951688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:04.951688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:05:04.959196 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:05:04.959669 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:05:04.959857 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:05:04.960059 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:05:04.960591 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:05:04.960883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:04.961564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:05:04.961564 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:04.961564 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:05:04.961564 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:05:04.961564 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:04.961564 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:05:04.961564 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:04.961564 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:05:04.962814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:04.962814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:05:04.962814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:05:04.962814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:05:04.962814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:04.962814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:05:04.962814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:04.962814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:05:04.962814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:04.962814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:05:04.962814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:04.962814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:05:04.962814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:04.962814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:05:04.962814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:05:04.963210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:04.963210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:05:04.963210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:04.963210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:05:04.963210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:05:04.963210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:04.963210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:05:04.963210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:04.963210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:05:04.964386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:04.964386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:05:04.964386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:04.964386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:05:04.964386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:05:04.964386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:05:04.964386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:05:04.964386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:05:04.964386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:05:04.964386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:04.964386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:04.964805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:04.964805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:05:04.964805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:04.964805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:04.964805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:04.964805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:05:04.964805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:04.964805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:05:04.964805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:05:04.964805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:04.964805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:05:04.964805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:04.964805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:05:04.964805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:05:04.965483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:04.965483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:05:04.965483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:04.965483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:05:04.966297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:04.966297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:05:04.966297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:05:04.966297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:05:04.966297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:05:04.966297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:05:04.966297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:04.966297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:05:04.966297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:05:04.966297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:05:04.966297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:05:04.966297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:05:04.966297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:04.966297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:04.966297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:05:04.966297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:05:04.966297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:05:04.967142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:05:04.967142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:05:04.967142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:05:04.967142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:04.967142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:04.967142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:04.967142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:05:04.967142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:04.967142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:04.967142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:04.967142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:05:04.967142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:04.967142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:05:04.967142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:05:04.967142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:04.967142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:05:04.967142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:04.967142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:05:04.967142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:05:04.968107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:04.968107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:05:04.968107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:04.968107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:05:04.968107 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:05:04.968107 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:05:04.968107 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:05:04.968107 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:05:04.968569 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:05:04.968569 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:05:04.968569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:04.968738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:05:04.968738 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:04.968738 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:05:04.968738 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:05:04.968738 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:04.968738 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:05:04.968738 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:04.968738 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:05:04.969577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:04.969577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:05:04.969577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:04.969577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:05:04.969577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:05:04.969577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:05:04.969577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:05:04.969577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:04.969577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:04.969577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:04.969577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:05:04.969577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:04.970172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:04.970172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:04.970172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:05:04.970172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:04.970172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:05:04.970172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:05:04.970172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:04.970172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:05:04.970172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:04.970172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:05:04.970172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:05:04.970172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:04.970172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:05:04.970823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:04.970823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:05:04.970823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:04.970823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:05:04.970823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:05:04.970823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:05:04.970823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:05:04.970823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:05:04.970823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:04.970823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:05:04.970823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:05:04.970823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:05:04.970823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:05:04.970823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:05:04.970823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:04.971562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:04.971562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:05:04.971562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:05:04.971562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:05:04.971562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:05:04.971562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:04.971562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:04.971562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:04.971562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:05:04.971562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:04.971562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:04.971562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:04.971562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:05:04.971562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:04.971562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:05:04.971562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:05:04.971562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:04.971562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:05:04.971562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:04.972564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:05:04.972564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:05:04.972564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:04.972564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:05:04.972564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:04.972564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:05:09.202234 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:05:09.203036 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:05:09.203124 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:05:09.204311 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:05:09.234169 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:05:09.234609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:09.235465 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:05:09.235465 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:09.235465 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:05:09.235465 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:05:09.235465 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:09.235465 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:05:09.235465 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:09.235465 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:05:09.236451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:09.236451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:05:09.236451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:09.236451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:05:09.236451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:05:09.236451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:05:09.236451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:05:09.236451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:09.236451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:09.236451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:09.236451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:05:09.236451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:09.237059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:09.237059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:09.237059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:05:09.237059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:09.237059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:05:09.237059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:05:09.237059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:09.237059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:05:09.237059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:09.237059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:05:09.237059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:05:09.237059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:09.237059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:05:09.237826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:09.237826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:05:09.237826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:09.237826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:05:09.237826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:05:09.237826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:05:09.237826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:05:09.237826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:05:09.237826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:09.237826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:05:09.237826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:05:09.237826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:05:09.237826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:05:09.237826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:05:09.237826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:09.238596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:09.238596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:05:09.238596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:05:09.238596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:05:09.238596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:05:09.238596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:09.238596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:09.238596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:09.238596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:05:09.238596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:09.238596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:09.238596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:09.238596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:05:09.238596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:09.238596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:05:09.238596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:05:09.238596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:09.238596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:05:09.238596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:09.239843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:05:09.239843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:05:09.239843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:09.239843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:05:09.239843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:09.239843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:05:09.927987 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:05:09.928477 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:05:09.928902 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:05:09.929872 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:05:09.930247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:09.931191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:05:09.931191 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:09.931191 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:05:09.931191 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:05:09.931191 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:09.931191 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:05:09.931191 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:09.931191 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:05:09.933374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:09.933374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:05:09.933374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:09.933374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:05:09.933374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:05:09.933374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:05:09.933374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:05:09.933374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:09.933374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:09.933374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:09.933374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:05:09.933374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:09.934073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:09.934073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:09.934073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:05:09.934073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:09.934073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:05:09.934073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:05:09.934073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:09.934073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:05:09.934073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:09.934073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:05:09.934073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:05:09.934073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:09.934073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:05:09.934722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:09.934722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:05:09.935161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:09.935161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:05:09.935161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:05:09.935161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:05:09.935161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:05:09.935161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:05:09.935161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:09.935161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:05:09.935161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:05:09.935161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:05:09.935161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:05:09.935161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:05:09.935161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:09.935161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:09.935161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:05:09.935161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:05:09.935973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:05:09.935973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:05:09.935973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:09.935973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:09.935973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:09.935973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:05:09.935973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:09.935973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:09.935973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:09.935973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:05:09.935973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:09.935973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:05:09.935973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:05:09.935973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:09.935973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:05:09.935973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:09.935973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:05:09.935973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:05:09.935973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:09.936918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:05:09.936918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:09.936918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:05:09.945128 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:05:09.945420 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:05:09.945528 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:05:09.945655 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:05:09.946133 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:05:09.946485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:09.947297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:05:09.947297 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:09.947297 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:05:09.947297 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:05:09.947297 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:09.947297 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:05:09.947297 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:09.947297 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:05:09.948621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:09.948621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:05:09.948621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:05:09.948621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:05:09.948621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:09.948621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:05:09.948621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:09.948621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:05:09.948621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:09.948621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:05:09.948621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:09.948621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:05:09.948621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:09.948621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:05:09.948621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:05:09.949493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:09.949493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:05:09.949493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:09.949493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:05:09.949493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:05:09.949493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:09.949493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:05:09.949493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:09.949493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:05:09.950105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:09.950105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:05:09.950105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:09.950105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:05:09.950105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:05:09.950105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:05:09.950105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:05:09.950105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:05:09.950105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:05:09.950105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:05:09.950105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:09.950105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:09.950752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:09.950752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:05:09.950752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:09.950752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:09.950752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:09.950752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:05:09.950752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:09.950752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:05:09.950752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:05:09.950752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:09.950752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:05:09.950752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:09.950752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:05:09.950752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:05:09.951389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:09.951389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:05:09.951389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:09.951389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:05:09.951832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:09.951832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:05:09.951832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:05:09.951832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:05:09.951832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:05:09.951832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:05:09.951832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:09.951832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:05:09.951832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:05:09.951832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:05:09.951832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:05:09.951832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:05:09.951832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:09.951832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:09.951832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:05:09.951832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:05:09.952655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:05:09.952655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:05:09.952655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:05:09.952655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:05:09.952655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:05:09.952655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:09.952655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:09.952655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:09.952655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:05:09.952655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:09.952655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:09.952655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:09.952655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:05:09.952655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:09.952655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:05:09.952655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:05:09.952655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:09.952655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:05:09.953570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:09.953570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:05:09.953570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:05:09.953570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:09.953570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:05:09.953570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:09.953570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:05:09.959710 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:05:09.959980 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:05:09.960051 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:05:09.960189 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:05:09.960692 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:05:09.961046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:09.961804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:05:09.961804 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:09.961804 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:05:09.961804 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:05:09.961804 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:09.961804 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:05:09.961804 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:09.961804 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:05:09.962562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:09.962562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:05:09.962562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:05:09.962562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:05:09.962562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:09.962562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:05:09.962562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:09.962562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:05:09.962562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:09.962562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:05:09.962562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:09.962562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:05:09.962562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:09.962562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:05:09.962562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:05:09.963045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:09.963045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:05:09.963045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:09.963045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:05:09.963045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:05:09.963045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:09.963045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:05:09.963045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:09.963045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:05:09.963447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:09.963447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:05:09.963447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:09.963447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:05:09.963447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:05:09.963447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:05:09.963447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:05:09.963447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:05:09.963447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:05:09.963447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:09.963447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:09.964093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:09.964093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:05:09.964093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:09.964093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:09.964093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:09.964093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:05:09.964093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:09.964093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:05:09.964093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:05:09.964093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:09.964093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:05:09.964093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:09.964093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:05:09.964093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:05:09.964798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:09.964798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:05:09.964798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:09.964798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:05:09.964798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:09.964798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:05:09.964798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:05:09.964798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:05:09.964798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:05:09.964798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:05:09.964798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:09.964798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:05:09.964798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:05:09.964798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:05:09.966540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:05:09.966540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:05:09.966540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:09.966540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:09.966540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:05:09.966540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:05:09.966540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:05:09.966540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:05:09.966540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:05:09.966540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:05:09.966540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:09.966540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:09.966540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:09.966540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:05:09.966540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:09.966540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:09.966540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:09.966540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:05:09.966540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:09.967537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:05:09.967537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:05:09.967537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:09.967537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:05:09.967537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:09.967537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:05:09.967537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:05:09.967537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:09.967537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:05:09.967537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:09.967537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:05:09.967537 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:05:09.967537 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:05:09.968253 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:05:09.968253 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:05:09.968253 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:05:09.968519 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:05:09.968519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:09.969258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:05:09.969258 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:09.969258 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:05:09.969258 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:05:09.969258 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:09.969258 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:05:09.969258 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:09.969258 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:05:09.970091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:09.970091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:05:09.970091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:09.970091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:05:09.970091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:05:09.970091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:05:09.970091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:05:09.970091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:09.970091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:09.970091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:09.970091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:05:09.970091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:09.970818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:09.970818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:09.970818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:05:09.970818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:09.970818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:05:09.970818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:05:09.970818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:09.970818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:05:09.970818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:09.970818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:05:09.970818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:05:09.970818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:09.970818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:05:09.971496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:09.971496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:05:09.971496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:09.971496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:05:09.971496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:05:09.971496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:05:09.971496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:05:09.971496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:05:09.971496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:09.971496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:05:09.971496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:05:09.971496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:05:09.971496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:05:09.971496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:05:09.971496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:09.972319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:09.972319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:05:09.972319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:05:09.972319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:05:09.972319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:05:09.972319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:09.972319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:09.972319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:09.972319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:05:09.972319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:09.972319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:09.972319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:09.972319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:05:09.972319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:09.972319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:05:09.972319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:05:09.972319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:09.972319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:05:09.972319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:09.973918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:05:09.973918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:05:09.973918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:09.973918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:05:09.973918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:09.973918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:05:14.203803 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:05:14.204454 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:05:14.204454 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:05:14.205617 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:05:14.241463 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:05:14.241811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:14.242722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:05:14.242722 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:14.242722 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:05:14.242722 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:05:14.242722 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:14.242722 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:05:14.242722 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:14.242722 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:05:14.243798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:14.243798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:05:14.243798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:14.243798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:05:14.243798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:05:14.243798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:05:14.243798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:05:14.243798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:14.243798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:14.243798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:14.243798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:05:14.243798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:14.244389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:14.244389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:14.244389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:05:14.244389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:14.244389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:05:14.244389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:05:14.244389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:14.244389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:05:14.244389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:14.244389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:05:14.244389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:05:14.244389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:14.244389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:05:14.244979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:14.244979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:05:14.244979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:14.244979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:05:14.244979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:05:14.244979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:05:14.244979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:05:14.244979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:05:14.244979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:14.244979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:05:14.244979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:05:14.244979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:05:14.244979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:05:14.244979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:05:14.244979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:14.245677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:14.245677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:05:14.245677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:05:14.245677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:05:14.245677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:05:14.245677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:14.245677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:14.245677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:14.245677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:05:14.245677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:14.245677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:14.245677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:14.245677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:05:14.245677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:14.245677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:05:14.245677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:05:14.245677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:14.245677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:05:14.245677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:14.246506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:05:14.246506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:05:14.246506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:14.246506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:05:14.246506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:14.246506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:05:14.928684 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:05:14.929352 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:05:14.929569 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:05:14.930619 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:05:14.931021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:14.931939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:05:14.931939 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:14.931939 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:05:14.931939 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:05:14.931939 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:14.931939 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:05:14.931939 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:14.931939 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:05:14.933430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:14.933430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:05:14.933430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:14.933430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:05:14.933430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:05:14.933430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:05:14.933430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:05:14.933430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:14.933430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:14.933430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:14.933430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:05:14.933430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:14.933967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:14.933967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:14.933967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:05:14.933967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:14.933967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:05:14.933967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:05:14.933967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:14.933967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:05:14.933967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:14.933967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:05:14.933967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:05:14.933967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:14.933967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:05:14.934583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:14.934583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:05:14.935116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:14.935116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:05:14.935116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:05:14.935116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:05:14.935116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:05:14.935116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:05:14.935116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:14.935116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:05:14.935116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:05:14.935116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:05:14.935116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:05:14.935116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:05:14.935116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:14.935116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:14.935116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:05:14.935116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:05:14.935820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:05:14.935820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:05:14.935820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:14.935820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:14.935820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:14.935820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:05:14.935820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:14.935820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:14.935820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:14.935820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:05:14.935820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:14.935820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:05:14.935820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:05:14.935820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:14.935820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:05:14.935820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:14.935820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:05:14.935820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:05:14.935820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:14.936695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:05:14.936695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:14.936695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:05:14.945575 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:05:14.945943 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:05:14.946129 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:05:14.946303 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:05:14.946695 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:05:14.946913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:14.948663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:05:14.948663 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:14.948663 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:05:14.948663 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:05:14.948663 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:14.948663 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:05:14.948663 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:14.948663 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:05:14.949820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:14.949820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:05:14.949820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:05:14.949820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:05:14.949820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:14.949820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:05:14.949820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:14.949820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:05:14.949820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:14.949820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:05:14.949820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:14.949820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:05:14.949820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:14.949820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:05:14.949820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:05:14.950487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:14.950487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:05:14.950487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:14.950487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:05:14.950487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:05:14.950487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:14.950487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:05:14.950487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:14.950487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:05:14.951214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:14.951214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:05:14.951214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:14.951214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:05:14.951214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:05:14.951214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:05:14.951214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:05:14.951214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:05:14.951214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:05:14.951214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:05:14.951214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:14.951214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:14.951957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:14.951957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:05:14.951957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:14.951957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:14.951957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:14.951957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:05:14.951957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:14.951957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:05:14.951957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:05:14.951957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:14.951957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:05:14.951957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:14.951957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:05:14.951957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:05:14.952704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:14.952704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:05:14.952704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:14.952704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:05:14.953519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:14.953519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:05:14.953519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:05:14.953519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:05:14.953519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:05:14.953519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:05:14.953519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:14.953519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:05:14.953519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:05:14.953519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:05:14.953519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:05:14.953519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:05:14.953519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:14.953519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:14.953519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:05:14.953519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:05:14.954236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:05:14.954236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:05:14.954236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:05:14.954236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:05:14.954236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:05:14.954236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:14.954236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:14.954236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:14.954236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:05:14.954236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:14.954236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:14.954236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:14.954236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:05:14.954236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:14.954236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:05:14.954236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:05:14.954236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:14.954236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:05:14.955067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:14.955067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:05:14.955067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:05:14.955067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:14.955067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:05:14.955067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:14.955067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:05:14.960058 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:05:14.960314 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:05:14.960443 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:05:14.960578 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:05:14.961007 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:05:14.961349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:14.962049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:05:14.962049 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:14.962049 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:05:14.962049 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:05:14.962049 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:14.962049 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:05:14.962049 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:14.962049 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:05:14.963255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:14.963255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:05:14.963255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:05:14.963255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:05:14.963255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:14.963255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:05:14.963255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:14.963255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:05:14.963255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:14.963255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:05:14.963255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:14.963255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:05:14.963255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:14.963255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:05:14.963255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:05:14.963746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:14.963746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:05:14.963746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:14.963746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:05:14.963746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:05:14.963746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:14.963746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:05:14.963746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:14.963746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:05:14.964596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:14.964596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:05:14.964596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:14.964596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:05:14.964596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:05:14.964596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:05:14.964596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:05:14.964596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:05:14.964596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:05:14.964596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:14.964596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:14.964942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:14.964942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:05:14.964942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:14.964942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:14.964942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:14.964942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:05:14.964942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:14.964942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:05:14.964942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:05:14.964942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:14.964942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:05:14.964942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:14.964942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:05:14.964942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:05:14.965338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:14.965338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:05:14.965338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:14.965338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:05:14.966197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:14.966197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:05:14.966197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:05:14.966197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:05:14.966197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:05:14.966197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:05:14.966197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:14.966197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:05:14.966197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:05:14.966197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:05:14.966197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:05:14.966197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:05:14.966197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:14.966197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:14.966197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:05:14.966197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:05:14.966197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:05:14.966741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:05:14.966741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:05:14.966741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:05:14.966741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:14.966741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:14.966741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:14.966741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:05:14.966741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:14.966741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:14.966741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:14.966741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:05:14.966741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:14.966741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:05:14.966741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:05:14.966741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:14.966741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:05:14.966741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:14.966741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:05:14.966741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:05:14.967248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:14.967248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:05:14.967248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:14.967248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:05:14.967459 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:05:14.967800 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:05:14.967955 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:05:14.968172 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:05:14.968521 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:05:14.968732 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:05:14.968844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:14.969362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:05:14.969362 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:14.969362 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:05:14.969362 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:05:14.969362 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:14.969362 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:05:14.969362 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:14.969362 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:05:14.970633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:14.970633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:05:14.970633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:14.970633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:05:14.970633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:05:14.970633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:05:14.970633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:05:14.970633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:14.970633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:14.970633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:14.970633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:05:14.970633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:14.971010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:14.971010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:14.971010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:05:14.971010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:14.971010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:05:14.971010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:05:14.971010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:14.971010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:05:14.971010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:14.971010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:05:14.971010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:05:14.971010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:14.971010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:05:14.971375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:14.971375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:05:14.971726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:14.971726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:05:14.971726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:05:14.971726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:05:14.971726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:05:14.971726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:05:14.971726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:14.971726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:05:14.971726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:05:14.971726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:05:14.971726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:05:14.971726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:05:14.971726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:14.971726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:14.971726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:05:14.971726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:05:14.972181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:05:14.972181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:05:14.972181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:14.972181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:14.972181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:14.972181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:05:14.972181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:14.972181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:14.972181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:14.972181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:05:14.972181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:14.972181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:05:14.972181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:05:14.972181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:14.972181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:05:14.972181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:14.972181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:05:14.972181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:05:14.972181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:14.972727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:05:14.972727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:14.972727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:05:19.204380 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:05:19.205106 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:05:19.205106 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:05:19.206264 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:05:19.232703 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:05:19.232898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:19.233493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:05:19.233493 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:19.233493 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:05:19.233493 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:05:19.233493 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:19.233493 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:05:19.233493 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:19.233493 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:05:19.234274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.028 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:19.234274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:05:19.234274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:19.234274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:05:19.234274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:05:19.234274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:05:19.234274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:05:19.234274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:19.234274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:19.234274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:19.234274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:05:19.234274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:19.234855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:19.234855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:19.234855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:05:19.234855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:19.234855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:05:19.234855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:05:19.234855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:19.234855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:05:19.234855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:19.234855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:05:19.234855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:05:19.234855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:19.234855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:05:19.235431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:19.235431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:05:19.235431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:19.235431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:05:19.235431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:05:19.235431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:05:19.235431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:05:19.235431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:05:19.235431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:19.235431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:05:19.235431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:05:19.235431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:05:19.235431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:05:19.235431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:05:19.235431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:19.236049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:19.236049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:05:19.236049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:05:19.236049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:05:19.236049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:05:19.236049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:19.236049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:19.236049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:19.236049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:05:19.236049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:19.236049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:19.236049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:19.236049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:05:19.236049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:19.236049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:05:19.236049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:05:19.236049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:19.236049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:05:19.236049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:19.236975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:05:19.236975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:05:19.236975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:19.236975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:05:19.236975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:19.236975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:05:19.929519 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:05:19.930053 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:05:19.930294 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:05:19.931150 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:05:19.931425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:19.932315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:05:19.932315 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:19.932315 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:05:19.932315 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:05:19.932315 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:19.932315 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:05:19.932315 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:19.932315 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:05:19.933892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:19.933892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:05:19.933892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:19.933892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:05:19.933892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:05:19.933892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:05:19.933892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:05:19.933892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:19.933892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:19.933892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:19.933892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:05:19.933892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:19.934525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:19.934525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:19.934525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:05:19.934525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:19.934525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:05:19.934525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:05:19.934525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:19.934525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:05:19.934525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:19.934525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:05:19.934525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:05:19.934525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:19.934525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:05:19.935096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:19.935096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:05:19.935096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:19.935096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:05:19.935096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:05:19.935096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:05:19.935096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:05:19.935096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:05:19.935096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:19.935096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:05:19.935096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:05:19.935096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:05:19.935096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:05:19.935096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:05:19.935096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:19.935812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:19.935812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:05:19.935812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:05:19.935812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:05:19.935812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:05:19.935812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:19.935812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:19.935812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:19.935812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:05:19.935812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:19.935812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:19.935812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:19.935812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:05:19.935812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:19.935812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:05:19.935812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:05:19.935812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:19.935812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:05:19.935812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:19.936718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:05:19.936718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:05:19.936718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:19.936718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:05:19.936718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:19.936718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:05:19.946012 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:05:19.946560 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:05:19.946741 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:05:19.947032 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:05:19.947691 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:05:19.948066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:19.948968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:05:19.948968 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:19.948968 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:05:19.948968 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:05:19.948968 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:19.948968 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:05:19.948968 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:19.948968 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:05:19.950133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:19.950133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:05:19.950133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:05:19.950133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:05:19.950133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:19.950133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:05:19.950133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:19.950133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:05:19.950133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:19.950133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:05:19.950133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:19.950133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:05:19.950133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:19.950133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:05:19.950133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:05:19.950846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:19.950846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:05:19.950846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:19.950846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:05:19.950846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:05:19.950846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:19.950846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:05:19.950846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:19.950846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:05:19.951582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:19.951582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:05:19.951582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:19.951582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:05:19.951582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:05:19.951582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:05:19.951582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:05:19.951582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:05:19.951582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:05:19.951582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:05:19.951582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:19.951582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:19.952144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:19.952144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:05:19.952144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:19.952144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:19.952144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:19.952144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:05:19.952144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:19.952144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:05:19.952144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:05:19.952144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:19.952144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:05:19.952144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:19.952144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:05:19.952144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:05:19.952800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:19.952800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:05:19.952800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:19.952800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:05:19.953025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:19.953025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:05:19.953025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:05:19.953025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:05:19.953025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:05:19.953025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:05:19.953025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:19.953025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:05:19.953025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:05:19.953025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:05:19.953025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:05:19.953025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:05:19.953025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:19.953025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:19.953025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:05:19.953025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:05:19.953724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:05:19.953724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:05:19.953724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:05:19.953724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:05:19.953724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:05:19.953724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:19.953724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:19.953724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:19.953724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:05:19.953724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:19.953724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:19.953724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:19.953724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:05:19.953724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:19.953724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:05:19.953724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:05:19.953724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:19.953724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:05:19.954533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:19.954533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:05:19.954533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:05:19.954533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:19.954533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:05:19.954533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:19.954533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:05:19.960350 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:05:19.960541 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:05:19.960612 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:05:19.960708 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:05:19.960914 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:05:19.961074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:19.961612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:05:19.961612 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:19.961612 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:05:19.961612 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:05:19.961612 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:19.961612 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:05:19.961612 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:19.961612 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:05:19.962754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:19.962754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:05:19.962754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:05:19.962754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:05:19.962754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:19.962754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:05:19.962754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:19.962754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:05:19.962754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:19.962754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:05:19.962754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:19.962754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:05:19.962754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:19.962754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:05:19.962754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:05:19.963190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:19.963190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:05:19.963190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:19.963190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:05:19.963190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:05:19.963190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:19.963190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:05:19.963190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:19.963190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:05:19.964081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:19.964081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:05:19.964081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:19.964081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:05:19.964081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:05:19.964081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:05:19.964081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:05:19.964081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:05:19.964081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:05:19.964081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:19.964081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:19.964777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:19.964777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:05:19.964777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:19.964777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:19.964777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:19.964777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:05:19.964777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:19.964777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:05:19.964777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:05:19.964777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:19.964777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:05:19.964777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:19.964777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:05:19.964777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:05:19.965193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:19.965193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:05:19.965193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:19.965193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:05:19.965790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:19.965790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:05:19.965790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:05:19.965790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:05:19.965790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:05:19.965790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:05:19.965790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:19.965790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:05:19.965790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:05:19.965790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:05:19.965790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:05:19.965790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:05:19.965790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:19.965790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:19.965790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:05:19.965790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:05:19.965790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:05:19.966289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:05:19.966289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:05:19.966289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:05:19.966289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:19.966289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:19.966289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:19.966289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:05:19.966289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:19.966289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:19.966289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:19.966289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:05:19.966289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:19.966289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:05:19.966289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:05:19.966289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:19.966289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:05:19.966289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:19.966289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:05:19.966289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:05:19.966824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:19.966824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:05:19.966824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:19.966824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:05:19.967845 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:05:19.968194 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:05:19.968417 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:05:19.968664 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:05:19.969139 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:05:19.969510 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:05:19.969688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:19.970390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:05:19.970390 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:19.970390 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:05:19.970390 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:05:19.970390 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:19.970390 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:05:19.970390 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:19.970390 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:05:19.971525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:19.971525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:05:19.971525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:19.971525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:05:19.971525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:05:19.971525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:05:19.971525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:05:19.971525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:19.971525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:19.971525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:19.971525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:05:19.971525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:19.971875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:19.971875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:19.971875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:05:19.971875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:19.971875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:05:19.971875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:05:19.971875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:19.971875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:05:19.971875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:19.971875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:05:19.971875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:05:19.971875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:19.971875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:05:19.972685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:19.972685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:05:19.972685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:19.972685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:05:19.972685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:05:19.972685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:05:19.972685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:05:19.972685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:05:19.972685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:19.972685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:05:19.972685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:05:19.972685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:05:19.972685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:05:19.972685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:05:19.972685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:19.973114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:19.973114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:05:19.973114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:05:19.973114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:05:19.973114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:05:19.973114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:19.973114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:19.973114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:19.973114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:05:19.973114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:19.973114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:19.973114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:19.973114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:05:19.973114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:19.973114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:05:19.973114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:05:19.973114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:19.973114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:05:19.973114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:19.973668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:05:19.973668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:05:19.973668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:19.973668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:05:19.973668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:19.973668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:05:24.205008 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:05:24.205730 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:05:24.205889 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:05:24.207730 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:05:24.209793 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG dbcounter [-] [105581] Writing DB stats neutron:SELECT=580,neutron:INSERT=1,neutron:UPDATE=2,neutron:DELETE=1 {{(pid=105581) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 14:05:24.243858 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:05:24.244294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:24.245211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:05:24.245211 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:24.245211 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:05:24.245211 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:05:24.245211 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:24.245211 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:05:24.245211 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:24.245211 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:05:24.246232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:24.246232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:05:24.246232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:24.246232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:05:24.246232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:05:24.246232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:05:24.246232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:05:24.246232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:24.246232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:24.246232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:24.246232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:05:24.246232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:24.246641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:24.246641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:24.246641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:05:24.246641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:24.246641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:05:24.246641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:05:24.246641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:24.246641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:05:24.246641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:24.246641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:05:24.246641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:05:24.246641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:24.246641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:05:24.246975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:24.246975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:05:24.247439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:24.247439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:05:24.247439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:05:24.247439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:05:24.247439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:05:24.247439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:05:24.247439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:24.247439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:05:24.247439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:05:24.247439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:05:24.247439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:05:24.247439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:05:24.247439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:24.247439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:24.247439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:05:24.247439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:05:24.248333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:05:24.248333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:05:24.248333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:24.248333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:24.248333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:24.248333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:05:24.248333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:24.248333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:24.248333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:24.248333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:05:24.248333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:24.248333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:05:24.248333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:05:24.248333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:24.248333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:05:24.248333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:24.248333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:05:24.248333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:05:24.248333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:24.249262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:05:24.249262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:24.249262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:05:24.930227 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:05:24.931015 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:05:24.931097 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:05:24.931941 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:05:24.932181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:24.933968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:05:24.933968 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:24.933968 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:05:24.933968 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:05:24.933968 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:24.933968 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:05:24.933968 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:24.933968 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:05:24.935555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:24.935555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:05:24.935555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:24.935555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:05:24.935555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:05:24.935555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:05:24.935555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:05:24.935555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:24.935555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:24.935555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:24.935555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:05:24.935555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:24.936219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:24.936219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:24.936219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:05:24.936219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:24.936219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:05:24.936219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:05:24.936219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:24.936219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:05:24.936219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:24.936219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:05:24.936219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:05:24.936219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:24.936219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:05:24.936866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:24.936866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:05:24.937232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:24.937232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:05:24.937232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:05:24.937232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:05:24.937232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:05:24.937232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:05:24.937232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:24.937232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:05:24.937232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:05:24.937232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:05:24.937232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:05:24.937232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:05:24.937232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:24.937232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:24.937232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:05:24.937232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:05:24.938258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:05:24.938258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:05:24.938258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:24.938258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:24.938258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:24.938258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:05:24.938258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:24.938258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:24.938258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:24.938258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:05:24.938258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:24.938258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:05:24.938258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:05:24.938258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:24.938258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:05:24.938258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:24.938258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:05:24.938258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:05:24.938258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:24.939155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:05:24.939155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:24.939155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:05:24.946507 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:05:24.946809 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:05:24.946919 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:05:24.947043 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:05:24.947606 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:05:24.947895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:24.948617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:05:24.948617 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:24.948617 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:05:24.948617 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:05:24.948617 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:24.948617 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:05:24.948617 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:24.948617 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:05:24.949324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:24.949324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:05:24.949324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:05:24.949324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:05:24.949324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:24.949324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:05:24.949324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:24.949324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:05:24.949324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:24.949324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:05:24.949324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:24.949324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:05:24.949324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:24.949324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:05:24.949324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:05:24.949793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:24.949793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:05:24.949793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:24.949793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:05:24.949793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:05:24.949793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:24.949793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:05:24.949793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:24.949793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:05:24.950147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:24.950147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:05:24.950147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:24.950147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:05:24.950147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:05:24.950147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:05:24.950147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:05:24.950147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:05:24.950147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:05:24.950147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:05:24.950147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:24.950147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:24.950592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:24.950592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:05:24.950592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:24.950592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:24.950592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:24.950592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:05:24.950592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:24.950592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:05:24.950592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:05:24.950592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:24.950592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:05:24.950592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:24.950592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:05:24.950592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:05:24.951008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:24.951008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:05:24.951008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:24.951008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:05:24.951153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:24.951153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:05:24.951153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:05:24.951153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:05:24.951153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:05:24.951153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:05:24.951153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:24.951153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:05:24.951153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:05:24.951153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:05:24.951153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:05:24.951153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:05:24.951153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:24.951153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:24.951153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:05:24.951153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:05:24.951729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:05:24.951729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:05:24.951729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:05:24.951729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:05:24.951729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:05:24.951729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:24.951729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:24.951729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:24.951729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:05:24.951729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:24.951729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:24.951729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:24.951729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:05:24.951729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:24.951729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:05:24.951729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:05:24.951729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:24.951729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:05:24.952272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:24.952272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:05:24.952272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:05:24.952272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:24.952272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:05:24.952272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:24.952272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:05:24.960778 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:05:24.961059 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:05:24.961166 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:05:24.961285 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:05:24.961779 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:05:24.962127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:24.962848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:05:24.962848 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:24.962848 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:05:24.962848 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:05:24.962848 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:24.962848 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:05:24.962848 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:24.962848 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:05:24.963571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:24.963571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:05:24.963571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:05:24.963571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:05:24.963571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:24.963571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:05:24.963571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:24.963571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:05:24.963571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:24.963571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:05:24.963571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:24.963571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:05:24.963571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:24.963571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:05:24.963571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:05:24.964255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:24.964255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:05:24.964255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:24.964255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:05:24.964255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:05:24.964255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:24.964255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:05:24.964255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:24.964255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:05:24.964762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:24.964762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:05:24.964762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:24.964762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:05:24.964762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:05:24.964762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:05:24.964762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:05:24.964762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:05:24.964762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:05:24.964762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:24.964762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:24.965269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:24.965269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:05:24.965269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:24.965269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:24.965269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:24.965269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:05:24.965269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:24.965269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:05:24.965269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:05:24.965269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:24.965269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:05:24.965269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:24.965269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:05:24.965269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:05:24.965956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:24.965956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:05:24.965956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:24.965956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:05:24.965956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:24.965956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:05:24.965956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:05:24.965956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:05:24.965956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:05:24.965956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:05:24.965956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:24.965956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:05:24.965956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:05:24.965956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:05:24.966987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:05:24.966987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:05:24.966987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:24.966987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:24.966987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:05:24.966987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:05:24.966987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:05:24.966987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:05:24.966987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:05:24.966987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:05:24.966987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:24.966987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:24.966987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:24.966987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:05:24.966987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:24.966987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:24.966987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:24.966987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:05:24.966987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:24.968130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:05:24.968130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:05:24.968130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:24.968130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:05:24.968130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:24.968130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:05:24.968130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:05:24.968130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:24.968130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:05:24.968130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:24.968130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:05:24.968699 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:05:24.968699 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:05:24.968819 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:05:24.969013 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:05:24.969488 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:05:24.969821 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:05:24.969993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:24.970822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:05:24.970822 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:24.970822 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:05:24.970822 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:05:24.970822 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:24.970822 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:05:24.970822 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:24.970822 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:05:24.972204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:24.972204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:05:24.972204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:24.972204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:05:24.972204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:05:24.972204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:05:24.972204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:05:24.972204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:24.972204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:24.972204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:24.972204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:05:24.972204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:24.972765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:24.972765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:24.972765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:05:24.972765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:24.972765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:05:24.972765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:05:24.972765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:24.972765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:05:24.972765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:24.972765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:05:24.972765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:05:24.972765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:24.972765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:05:24.973339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:24.973339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:05:24.973708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:24.973708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:05:24.973708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:05:24.973708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:05:24.973708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:05:24.973708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:05:24.973708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:24.973708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:05:24.973708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:05:24.973708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:05:24.973708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:05:24.973708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:05:24.973708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:24.973708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:24.973708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:05:24.973708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:05:24.974503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:05:24.974503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:05:24.974503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:24.974503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:24.974503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:24.974503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:05:24.974503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:24.974503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:24.974503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:24.974503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:05:24.974503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:24.974503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:05:24.974503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:05:24.974503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:24.974503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:05:24.974503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:24.974503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:05:24.974503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:05:24.974503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:24.975324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:05:24.975324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:24.975324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:05:29.205776 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:05:29.206832 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:05:29.206832 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:05:29.208241 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:05:29.251728 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:05:29.251954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:29.252601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:05:29.252601 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:29.252601 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:05:29.252601 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:05:29.252601 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:29.252601 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:05:29.252601 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:29.252601 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:05:29.253527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.046 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:29.253527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:05:29.253527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:29.253527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:05:29.253527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:05:29.253527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:05:29.253527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:05:29.253527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:29.253527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:29.253527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:29.253527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:05:29.253527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:29.254141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:29.254141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:29.254141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:05:29.254141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:29.254141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:05:29.254141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:05:29.254141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:29.254141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:05:29.254141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:29.254141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:05:29.254141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:05:29.254141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:29.254141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:05:29.254735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:29.254735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:05:29.254735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:29.254735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:05:29.254735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:05:29.254735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:05:29.254735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:05:29.254735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:05:29.254735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:29.254735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:05:29.254735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:05:29.254735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:05:29.254735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:05:29.254735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:05:29.254735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:29.255452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:29.255452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:05:29.255452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:05:29.255452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:05:29.255452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:05:29.255452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:29.255452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:29.255452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:29.255452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:05:29.255452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:29.255452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:29.255452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:29.255452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:05:29.255452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:29.255452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:05:29.255452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:05:29.255452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:29.255452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:05:29.255452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:29.256322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:05:29.256322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:05:29.256322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:29.256322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:05:29.256322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:29.256322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:05:29.931206 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:05:29.931875 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:05:29.932059 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:05:29.932983 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:05:29.933344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:29.934255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:05:29.934255 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:29.934255 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:05:29.934255 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:05:29.934255 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:29.934255 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:05:29.934255 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:29.934255 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:05:29.935260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:29.935260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:05:29.935260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:29.935260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:05:29.935260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:05:29.935260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:05:29.935260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:05:29.935260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:29.935260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:29.935260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:29.935260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:05:29.935260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:29.935843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:29.935843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:29.935843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:05:29.935843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:29.935843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:05:29.935843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:05:29.935843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:29.935843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:05:29.935843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:29.935843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:05:29.935843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:05:29.935843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:29.935843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:05:29.936368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:29.936368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:05:29.936368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:29.936368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:05:29.936368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:05:29.936368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:05:29.936368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:05:29.936368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:05:29.936368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:29.936368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:05:29.936368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:05:29.936368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:05:29.936368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:05:29.936368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:05:29.936368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:29.937158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:29.937158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:05:29.937158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:05:29.937158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:05:29.937158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:05:29.937158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:29.937158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:29.937158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:29.937158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:05:29.937158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:29.937158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:29.937158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:29.937158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:05:29.937158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:29.937158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:05:29.937158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:05:29.937158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:29.937158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:05:29.937158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:29.938002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:05:29.938002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:05:29.938002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:29.938002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:05:29.938002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:29.938002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:05:29.947007 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:05:29.947493 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:05:29.947703 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:05:29.947905 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:05:29.948420 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:05:29.948842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:29.949881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:05:29.949881 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:29.949881 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:05:29.949881 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:05:29.949881 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:29.949881 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:05:29.949881 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:29.949881 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:05:29.951197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:29.951197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:05:29.951197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:05:29.951197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:05:29.951197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:29.951197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:05:29.951197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:29.951197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:05:29.951197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:29.951197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:05:29.951197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:29.951197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:05:29.951197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:29.951197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:05:29.951197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:05:29.951902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:29.951902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:05:29.951902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:29.951902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:05:29.951902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:05:29.951902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:29.951902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:05:29.951902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:29.951902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:05:29.952677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:29.952677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:05:29.952677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:29.952677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:05:29.952677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:05:29.952677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:05:29.952677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:05:29.952677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:05:29.952677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:05:29.952677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:05:29.952677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:29.952677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:29.953301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:29.953301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:05:29.953301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:29.953301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:29.953301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:29.953301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:05:29.953301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:29.953301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:05:29.953301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:05:29.953301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:29.953301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:05:29.953301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:29.953301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:05:29.953301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:05:29.953942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:29.953942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:05:29.953942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:29.953942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:05:29.954135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:29.954135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:05:29.954135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:05:29.954135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:05:29.954135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:05:29.954135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:05:29.954135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:29.954135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:05:29.954135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:05:29.954135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:05:29.954135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:05:29.954135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:05:29.954135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:29.954135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:29.954135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:05:29.954135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:05:29.954833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:05:29.954833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:05:29.954833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:05:29.954833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:05:29.954833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:05:29.954833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:29.954833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:29.954833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:29.954833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:05:29.954833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:29.954833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:29.954833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:29.954833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:05:29.954833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:29.954833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:05:29.954833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:05:29.954833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:29.954833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:05:29.955667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:29.955667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:05:29.955667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:05:29.955667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:29.955667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:05:29.955667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:29.955667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:05:29.961192 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:05:29.961477 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:05:29.961590 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:05:29.961714 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:05:29.962017 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:05:29.962250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:29.962841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:05:29.962841 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:29.962841 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:05:29.962841 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:05:29.962841 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:29.962841 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:05:29.962841 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:29.962841 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:05:29.964121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:29.964121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:05:29.964121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:05:29.964121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:05:29.964121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:29.964121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:05:29.964121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:29.964121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:05:29.964121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:29.964121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:05:29.964121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:29.964121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:05:29.964121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:29.964121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:05:29.964121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:05:29.964589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:29.964589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:05:29.964589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:29.964589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:05:29.964589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:05:29.964589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:29.964589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:05:29.964589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:29.964589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:05:29.965717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:29.965717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:05:29.965717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:29.965717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:05:29.965717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:05:29.965717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:05:29.965717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:05:29.965717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:05:29.965717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:05:29.965717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:29.965717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:29.966055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:29.966055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:05:29.966055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:29.966055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:29.966055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:29.966055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:05:29.966055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:29.966055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:05:29.966055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:05:29.966055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:29.966055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:05:29.966055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:29.966055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:05:29.966055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:05:29.966452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:29.966452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:05:29.966452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:29.966452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:05:29.967604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:29.967604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:05:29.967604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:05:29.967604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:05:29.967604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:05:29.967604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:05:29.967604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:29.967604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:05:29.967604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:05:29.967604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:05:29.967604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:05:29.967604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:05:29.967604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:29.967604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:29.967604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:05:29.967604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:05:29.967604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:05:29.968102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:05:29.968102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:05:29.968102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:05:29.968102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:29.968102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:29.968102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:29.968102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:05:29.968102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:29.968102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:29.968102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:29.968102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:05:29.968102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:29.968102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:05:29.968102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:05:29.968102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:29.968102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:05:29.968102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:29.968102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:05:29.968102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:05:29.968695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:29.968695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:05:29.968695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:29.968695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:05:29.968695 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:05:29.969124 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:05:29.969306 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:05:29.969664 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:05:29.970169 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:05:29.970558 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:05:29.970774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:29.971486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:05:29.971486 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:29.971486 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:05:29.971486 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:05:29.971486 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:29.971486 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:05:29.971486 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:29.971486 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:05:29.972324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:29.972324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:05:29.972324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:29.972324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:05:29.972324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:05:29.972324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:05:29.972324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:05:29.972324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:29.972324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:29.972324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:29.972324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:05:29.972324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:29.972995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:29.972995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:29.972995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:05:29.972995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:29.972995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:05:29.972995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:05:29.972995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:29.972995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:05:29.972995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:29.972995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:05:29.972995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:05:29.972995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:29.972995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:05:29.973646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:29.973646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:05:29.973646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:29.973646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:05:29.973646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:05:29.973646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:05:29.973646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:05:29.973646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:05:29.973646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:29.973646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:05:29.973646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:05:29.973646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:05:29.973646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:05:29.973646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:05:29.973646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:29.974397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:29.974397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:05:29.974397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:05:29.974397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:05:29.974397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:05:29.974397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:29.974397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:29.974397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:29.974397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:05:29.974397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:29.974397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:29.974397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:29.974397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:05:29.974397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:29.974397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:05:29.974397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:05:29.974397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:29.974397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:05:29.974397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:29.975335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:05:29.975335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:05:29.975335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:29.975335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:05:29.975335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:29.975335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:05:34.206716 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:05:34.207446 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:05:34.207446 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:05:34.208625 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:05:34.245058 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:05:34.245558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:34.246551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:05:34.246551 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:34.246551 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:05:34.246551 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:05:34.246551 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:34.246551 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:05:34.246551 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:34.246551 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:05:34.247367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:34.247367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:05:34.247367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:34.247367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:05:34.247367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:05:34.247367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:05:34.247367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:05:34.247367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:34.247367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:34.247367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:34.247367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:05:34.247367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:34.248207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:34.248207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:34.248207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:05:34.248207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:34.248207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:05:34.248207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:05:34.248207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:34.248207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:05:34.248207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:34.248207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:05:34.248207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:05:34.248207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:34.248207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:05:34.249149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:34.249149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:05:34.249149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:34.249149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:05:34.249149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:05:34.249149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:05:34.249149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:05:34.249149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:05:34.249149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:34.249149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:05:34.249149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:05:34.249149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:05:34.249149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:05:34.249149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:05:34.249149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:34.250164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:34.250164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:05:34.250164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:05:34.250164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:05:34.250164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:05:34.250164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:34.250164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:34.250164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:34.250164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:05:34.250164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:34.250164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:34.250164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:34.250164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:05:34.250164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:34.250164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:05:34.250164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:05:34.250164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:34.250164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:05:34.250164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:34.251557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:05:34.251557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:05:34.251557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:34.251557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:05:34.251557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:34.251557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:05:34.936398 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:05:34.937504 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:05:34.937504 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:05:34.938552 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:05:34.938788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:34.939531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:05:34.939531 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:34.939531 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:05:34.939531 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:05:34.939531 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:34.939531 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:05:34.939531 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:34.939531 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:05:34.941275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:34.941275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:05:34.941275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:34.941275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:05:34.941275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:05:34.941275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:05:34.941275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:05:34.941275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:34.941275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:34.941275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:34.941275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:05:34.941275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:34.941896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:34.941896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:34.941896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:05:34.941896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:34.941896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:05:34.941896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:05:34.941896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:34.941896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:05:34.941896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:34.941896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:05:34.941896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:05:34.941896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:34.941896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:05:34.942446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:34.942446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:05:34.943254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:34.943254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:05:34.943254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:05:34.943254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:05:34.943254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:05:34.943254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:05:34.943254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:34.943254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:05:34.943254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:05:34.943254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:05:34.943254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:05:34.943254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:05:34.943254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:34.943254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:34.943254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:05:34.943254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:05:34.944016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:05:34.944016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:05:34.944016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:34.944016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:34.944016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:34.944016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:05:34.944016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:34.944016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:34.944016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:34.944016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:05:34.944016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:34.944016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:05:34.944016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:05:34.944016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:34.944016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:05:34.944016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:34.944016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:05:34.944016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:05:34.944016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:34.944834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:05:34.944834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:34.944834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:05:34.947485 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:05:34.947872 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:05:34.948051 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:05:34.948254 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:05:34.948751 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:05:34.949051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:34.949821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:05:34.949821 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:34.949821 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:05:34.949821 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:05:34.949821 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:34.949821 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:05:34.949821 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:34.949821 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:05:34.950648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:34.950648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:05:34.950648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:05:34.950648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:05:34.950648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:34.950648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:05:34.950648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:34.950648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:05:34.950648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:34.950648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:05:34.950648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:34.950648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:05:34.950648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:34.950648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:05:34.950648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:05:34.951093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:34.951093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:05:34.951093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:34.951093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:05:34.951093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:05:34.951093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:34.951093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:05:34.951093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:34.951093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:05:34.951432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:34.951432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:05:34.951432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:34.951432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:05:34.951432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:05:34.951432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:05:34.951432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:05:34.951432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:05:34.951432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:05:34.951432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:05:34.951432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:34.951432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:34.951844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:34.951844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:05:34.951844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:34.951844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:34.951844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:34.951844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:05:34.951844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:34.951844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:05:34.951844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:05:34.951844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:34.951844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:05:34.951844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:34.951844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:05:34.951844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:05:34.952195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:34.952195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:05:34.952195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:34.952195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:05:34.952461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:34.952461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:05:34.952461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:05:34.952461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:05:34.952461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:05:34.952461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:05:34.952461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:34.952461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:05:34.952461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:05:34.952461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:05:34.952461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:05:34.952461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:05:34.952461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:34.952461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:34.952461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:05:34.952461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:05:34.953061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:05:34.953061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:05:34.953061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:05:34.953061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:05:34.953061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:05:34.953061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:34.953061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:34.953061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:34.953061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:05:34.953061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:34.953061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:34.953061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:34.953061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:05:34.953061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:34.953061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:05:34.953061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:05:34.953061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:34.953061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:05:34.953552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:34.953552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:05:34.953552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:05:34.953552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:34.953552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:05:34.953552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:34.953552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:05:34.961587 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:05:34.961888 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:05:34.961997 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:05:34.962118 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:05:34.962598 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:05:34.962949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:34.963632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:05:34.963632 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:34.963632 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:05:34.963632 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:05:34.963632 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:34.963632 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:05:34.963632 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:34.963632 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:05:34.964495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:34.964495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:05:34.964495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:05:34.964495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:05:34.964495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:34.964495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:05:34.964495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:34.964495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:05:34.964495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:34.964495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:05:34.964495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:34.964495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:05:34.964495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:34.964495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:05:34.964495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:05:34.964916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:34.964916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:05:34.964916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:34.964916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:05:34.964916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:05:34.964916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:34.964916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:05:34.964916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:34.964916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:05:34.965594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:34.965594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:05:34.965594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:34.965594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:05:34.965594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:05:34.965594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:05:34.965594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:05:34.965594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:05:34.965594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:05:34.965594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:34.965594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:34.965915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:34.965915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:05:34.965915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:34.965915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:34.965915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:34.965915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:05:34.965915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:34.965915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:05:34.965915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:05:34.965915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:34.965915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:05:34.965915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:34.965915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:05:34.965915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:05:34.966264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:34.966264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:05:34.966264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:34.966264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:05:34.967193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:34.967193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:05:34.967193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:05:34.967193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:05:34.967193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:05:34.967193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:05:34.967193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:34.967193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:05:34.967193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:05:34.967193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:05:34.967193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:05:34.967193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:05:34.967193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:34.967193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:34.967193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:05:34.967193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:05:34.967193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:05:34.967753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:05:34.967753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:05:34.967753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:05:34.967753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:34.967753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:34.967753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:34.967753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:05:34.967753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:34.967753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:34.967753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:34.967753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:05:34.967753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:34.967753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:05:34.967753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:05:34.967753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:34.967753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:05:34.967753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:34.967753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:05:34.967753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:05:34.968254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:34.968254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:05:34.968254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:34.968254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:05:34.969138 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:05:34.969468 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:05:34.969590 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:05:34.969737 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:05:34.970135 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:05:34.970539 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:05:34.970712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:34.971434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:05:34.971434 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:34.971434 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:05:34.971434 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:05:34.971434 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:34.971434 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:05:34.971434 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:34.971434 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:05:34.972352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:34.972352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:05:34.972352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:34.972352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:05:34.972352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:05:34.972352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:05:34.972352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:05:34.972352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:34.972352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:34.972352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:34.972352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:05:34.972352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:34.972698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:34.972698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:34.972698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:05:34.972698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:34.972698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:05:34.972698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:05:34.972698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:34.972698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:05:34.972698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:34.972698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:05:34.972698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:05:34.972698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:34.972698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:05:34.973013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:34.973013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:05:34.973358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:34.973358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:05:34.973358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:05:34.973358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:05:34.973358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:05:34.973358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:05:34.973358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:34.973358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:05:34.973358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:05:34.973358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:05:34.973358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:05:34.973358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:05:34.973358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:34.973358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:34.973358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:05:34.973358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:05:34.973982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:05:34.973982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:05:34.973982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:34.973982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:34.973982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:34.973982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:05:34.973982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:34.973982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:34.973982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:34.973982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:05:34.973982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:34.973982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:05:34.973982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:05:34.973982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:34.973982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:05:34.973982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:34.973982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:05:34.973982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:05:34.973982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:34.974621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:05:34.974621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:34.974621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:05:39.210609 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:05:39.214894 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:05:39.215151 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:05:39.220764 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:05:39.259740 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:05:39.260154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:39.261103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:05:39.261103 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:39.261103 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:05:39.261103 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:05:39.261103 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:39.261103 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:05:39.261103 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:39.261103 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:05:39.262194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.046 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:39.262194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:05:39.262194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:39.262194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:05:39.262194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:05:39.262194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:05:39.262194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:05:39.262194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:39.262194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:39.262194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:39.262194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:05:39.262194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:39.262829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:39.262829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:39.262829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:05:39.262829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:39.262829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:05:39.262829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:05:39.262829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:39.262829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:05:39.262829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:39.262829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:05:39.262829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:05:39.262829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:39.262829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:05:39.263433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:39.263433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:05:39.263433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:39.263433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:05:39.263433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:05:39.263433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:05:39.263433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:05:39.263433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:05:39.263433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:39.263433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:05:39.263433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:05:39.263433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:05:39.263433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:05:39.263433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:05:39.263433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:39.264090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:39.264090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:05:39.264090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:05:39.264090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:05:39.264090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:05:39.264090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:39.264090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:39.264090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:39.264090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:05:39.264090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:39.264090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:39.264090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:39.264090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:05:39.264090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:39.264090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:05:39.264090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:05:39.264090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:39.264090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:05:39.264090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:39.264960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:05:39.264960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:05:39.264960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:39.264960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:05:39.264960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:39.264960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:05:39.937012 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:05:39.937561 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:05:39.937732 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:05:39.938575 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:05:39.938892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:39.939816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:05:39.939816 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:39.939816 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:05:39.939816 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:05:39.939816 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:39.939816 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:05:39.939816 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:39.939816 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:05:39.941308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:39.941308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:05:39.941308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:39.941308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:05:39.941308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:05:39.941308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:05:39.941308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:05:39.941308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:39.941308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:39.941308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:39.941308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:05:39.941308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:39.941940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:39.941940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:39.941940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:05:39.941940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:39.941940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:05:39.941940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:05:39.941940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:39.941940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:05:39.941940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:39.941940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:05:39.941940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:05:39.941940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:39.941940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:05:39.942650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:39.942650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:05:39.943064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:39.943064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:05:39.943064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:05:39.943064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:05:39.943064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:05:39.943064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:05:39.943064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:39.943064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:05:39.943064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:05:39.943064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:05:39.943064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:05:39.943064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:05:39.943064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:39.943064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:39.943064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:05:39.943064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:05:39.943913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:05:39.943913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:05:39.943913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:39.943913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:39.943913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:39.943913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:05:39.943913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:39.943913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:39.943913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:39.943913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:05:39.943913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:39.943913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:05:39.943913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:05:39.943913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:39.943913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:05:39.943913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:39.943913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:05:39.943913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:05:39.943913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:39.944456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:05:39.944456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:39.944456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:05:39.947937 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:05:39.948221 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:05:39.948338 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:05:39.948494 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:05:39.948924 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:05:39.949199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:39.949942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:05:39.949942 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:39.949942 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:05:39.949942 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:05:39.949942 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:39.949942 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:05:39.949942 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:39.949942 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:05:39.950703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:39.950703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:05:39.950703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:05:39.950703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:05:39.950703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:39.950703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:05:39.950703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:39.950703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:05:39.950703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:39.950703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:05:39.950703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:39.950703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:05:39.950703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:39.950703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:05:39.950703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:05:39.951137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:39.951137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:05:39.951137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:39.951137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:05:39.951137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:05:39.951137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:39.951137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:05:39.951137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:39.951137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:05:39.951536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:39.951536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:05:39.951536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:39.951536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:05:39.951536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:05:39.951536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:05:39.951536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:05:39.951536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:05:39.951536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:05:39.951536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:05:39.951536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:39.951536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:39.951976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:39.951976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:05:39.951976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:39.951976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:39.951976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:39.951976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:05:39.951976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:39.951976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:05:39.951976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:05:39.951976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:39.951976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:05:39.951976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:39.951976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:05:39.951976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:05:39.952333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:39.952333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:05:39.952333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:39.952333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:05:39.952568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:39.952568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:05:39.952568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:05:39.952568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:05:39.952568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:05:39.952568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:05:39.952568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:39.952568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:05:39.952568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:05:39.952568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:05:39.952568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:05:39.952568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:05:39.952568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:39.952568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:39.952568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:05:39.952568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:05:39.953046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:05:39.953046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:05:39.953046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:05:39.953046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:05:39.953046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:05:39.953046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:39.953046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:39.953046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:39.953046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:05:39.953046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:39.953046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:39.953046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:39.953046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:05:39.953046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:39.953046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:05:39.953046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:05:39.953046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:39.953046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:05:39.953521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:39.953521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:05:39.953521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:05:39.953521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:39.953521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:05:39.953521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:39.953521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:05:39.962056 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:05:39.962322 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:05:39.962440 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:05:39.962579 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:05:39.962987 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:05:39.963336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:39.964115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:05:39.964115 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:39.964115 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:05:39.964115 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:05:39.964115 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:39.964115 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:05:39.964115 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:39.964115 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:05:39.965220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:39.965220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:05:39.965220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:05:39.965220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:05:39.965220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:39.965220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:05:39.965220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:39.965220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:05:39.965220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:39.965220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:05:39.965220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:39.965220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:05:39.965220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:39.965220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:05:39.965220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:05:39.965985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:39.965985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:05:39.965985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:39.965985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:05:39.965985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:05:39.965985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:39.965985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:05:39.965985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:39.965985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:05:39.967249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:39.967249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:05:39.967249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:39.967249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:05:39.967249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:05:39.967249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:05:39.967249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:05:39.967249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:05:39.967249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:05:39.967249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:39.967249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:39.967802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:39.967802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:05:39.967802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:39.967802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:39.967802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:39.967802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:05:39.967802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:39.967802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:05:39.967802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:05:39.967802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:39.967802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:05:39.967802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:39.967802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:05:39.967802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:05:39.968427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:39.968427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:05:39.968427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:39.968427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:05:39.968427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:39.968427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:05:39.968427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:05:39.968427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:05:39.968427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:05:39.968427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:05:39.968427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:39.968427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:05:39.968427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:05:39.968427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:05:39.969053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:05:39.969053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:05:39.969053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:39.969053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:39.969053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:05:39.969053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:05:39.969053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:05:39.969053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:05:39.969053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:05:39.969053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:05:39.969053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:39.969053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:39.969053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:39.969053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:05:39.969053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:39.969053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:39.969053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:39.969053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:05:39.969053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:39.969852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:05:39.969852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:05:39.969852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:39.969852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:05:39.969852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:39.969852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:05:39.969852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:05:39.969852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:39.969852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:05:39.969852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:39.969852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:05:39.969852 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:05:39.970431 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:05:39.970431 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:05:39.970431 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:05:39.970759 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:05:39.971079 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:05:39.971249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:39.972055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:05:39.972055 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:39.972055 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:05:39.972055 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:05:39.972055 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:39.972055 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:05:39.972055 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:39.972055 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:05:39.972985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:39.972985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:05:39.972985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:39.972985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:05:39.972985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:05:39.972985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:05:39.972985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:05:39.972985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:39.972985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:39.972985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:39.972985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:05:39.972985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:39.973319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:39.973319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:39.973319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:05:39.973319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:39.973319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:05:39.973319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:05:39.973319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:39.973319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:05:39.973319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:39.973319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:05:39.973319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:05:39.973319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:39.973319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:05:39.974163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:39.974163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:05:39.974163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:39.974163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:05:39.974163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:05:39.974163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:05:39.974163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:05:39.974163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:05:39.974163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:39.974163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:05:39.974163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:05:39.974163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:05:39.974163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:05:39.974163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:05:39.974163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:39.974600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:39.974600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:05:39.974600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:05:39.974600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:05:39.974600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:05:39.974600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:39.974600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:39.974600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:39.974600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:05:39.974600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:39.974600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:39.974600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:39.974600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:05:39.974600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:39.974600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:05:39.974600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:05:39.974600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:39.974600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:05:39.974600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:39.975135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:05:39.975135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:05:39.975135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:39.975135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:05:39.975135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:39.975135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:05:44.212700 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:05:44.213477 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:05:44.213619 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:05:44.214839 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:05:44.247982 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:05:44.248477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:44.249336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:05:44.249336 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:44.249336 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:05:44.249336 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:05:44.249336 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:44.249336 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:05:44.249336 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:44.249336 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:05:44.250313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:44.250313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:05:44.250313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:44.250313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:05:44.250313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:05:44.250313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:05:44.250313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:05:44.250313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:44.250313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:44.250313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:44.250313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:05:44.250313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:44.250939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:44.250939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:44.250939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:05:44.250939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:44.250939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:05:44.250939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:05:44.250939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:44.250939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:05:44.250939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:44.250939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:05:44.250939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:05:44.250939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:44.250939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:05:44.251559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:44.251559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:05:44.251559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:44.251559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:05:44.251559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:05:44.251559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:05:44.251559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:05:44.251559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:05:44.251559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:44.251559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:05:44.251559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:05:44.251559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:05:44.251559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:05:44.251559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:05:44.251559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:44.252251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:44.252251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:05:44.252251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:05:44.252251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:05:44.252251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:05:44.252251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:44.252251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:44.252251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:44.252251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:05:44.252251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:44.252251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:44.252251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:44.252251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:05:44.252251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:44.252251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:05:44.252251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:05:44.252251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:44.252251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:05:44.252251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:44.253624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:05:44.253624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:05:44.253624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:44.253624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:05:44.253624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:44.253624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:05:44.937694 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:05:44.938356 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:05:44.938603 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:05:44.939497 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:05:44.939863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:44.940751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:05:44.940751 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:44.940751 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:05:44.940751 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:05:44.940751 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:44.940751 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:05:44.940751 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:44.940751 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:05:44.942328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:44.942328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:05:44.942328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:44.942328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:05:44.942328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:05:44.942328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:05:44.942328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:05:44.942328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:44.942328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:44.942328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:44.942328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:05:44.942328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:44.943667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:44.943667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:44.943667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:05:44.943667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:44.943667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:05:44.943667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:05:44.943667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:44.943667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:05:44.943667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:44.943667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:05:44.943667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:05:44.943667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:44.943667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:05:44.944914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:44.944914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:05:44.944914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:44.944914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:05:44.944914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:05:44.944914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:05:44.944914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:05:44.944914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:05:44.944914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:44.944914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:05:44.944914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:05:44.944914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:05:44.944914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:05:44.944914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:05:44.944914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:44.946363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:44.946363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:05:44.946363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:05:44.946363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:05:44.946363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:05:44.946363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:44.946363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:44.946363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:44.946363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:05:44.946363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:44.946363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:44.946363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:44.946363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:05:44.946363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:44.946363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:05:44.946363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:05:44.946363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:44.946363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:05:44.946363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:44.947167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:05:44.947167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:05:44.947167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:44.947167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:05:44.947167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:44.947167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:05:44.948322 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:05:44.948631 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:05:44.948750 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:05:44.948865 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:05:44.949270 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:05:44.949607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:44.950293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:05:44.950293 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:44.950293 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:05:44.950293 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:05:44.950293 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:44.950293 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:05:44.950293 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:44.950293 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:05:44.951047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:44.951047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:05:44.951047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:05:44.951047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:05:44.951047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:44.951047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:05:44.951047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:44.951047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:05:44.951047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:44.951047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:05:44.951047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:44.951047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:05:44.951047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:44.951047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:05:44.951047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:05:44.951769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:44.951769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:05:44.951769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:44.951769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:05:44.951769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:05:44.951769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:44.951769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:05:44.951769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:44.951769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:05:44.952023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:44.952023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:05:44.952023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:44.952023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:05:44.952023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:05:44.952023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:05:44.952023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:05:44.952023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:05:44.952023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:05:44.952023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:05:44.952023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:44.952023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:44.952381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:44.952381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:05:44.952381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:44.952381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:44.952381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:44.952381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:05:44.952381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:44.952381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:05:44.952381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:05:44.952381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:44.952381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:05:44.952381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:44.952381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:05:44.952381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:05:44.952763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:44.952763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:05:44.952763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:44.952763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:05:44.952986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:44.952986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:05:44.952986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:05:44.952986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:05:44.952986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:05:44.952986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:05:44.952986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:44.952986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:05:44.952986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:05:44.952986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:05:44.952986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:05:44.952986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:05:44.952986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:44.952986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:44.952986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:05:44.952986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:05:44.953423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:05:44.953423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:05:44.953423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:05:44.953423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:05:44.953423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:05:44.953423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:44.953423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:44.953423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:44.953423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:05:44.953423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:44.953423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:44.953423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:44.953423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:05:44.953423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:44.953423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:05:44.953423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:05:44.953423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:44.953423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:05:44.954107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:44.954107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:05:44.954107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:05:44.954107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:44.954107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:05:44.954107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:44.954107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:05:44.962443 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:05:44.962761 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:05:44.962853 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:05:44.963003 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:05:44.963554 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:05:44.963912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:44.964806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:05:44.964806 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:44.964806 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:05:44.964806 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:05:44.964806 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:44.964806 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:05:44.964806 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:44.964806 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:05:44.966262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:44.966262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:05:44.966262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:05:44.966262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:05:44.966262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:44.966262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:05:44.966262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:44.966262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:05:44.966262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:44.966262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:05:44.966262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:44.966262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:05:44.966262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:44.966262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:05:44.966262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:05:44.967400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:44.967400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:05:44.967400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:44.967400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:05:44.967400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:05:44.967400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:44.967400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:05:44.967400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:44.967400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:05:44.967903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:44.967903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:05:44.967903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:44.967903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:05:44.967903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:05:44.967903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:05:44.967903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:05:44.967903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:05:44.967903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:05:44.967903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:44.967903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:44.968972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:44.968972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:05:44.968972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:44.968972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:44.968972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:44.968972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:05:44.968972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:44.968972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:05:44.968972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:05:44.968972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:44.968972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:05:44.968972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:44.968972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:05:44.968972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:05:44.969820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:44.969820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:05:44.969820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:44.969820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:05:44.969820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:44.969820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:05:44.969820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:05:44.969820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:05:44.969820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:05:44.969820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:05:44.969820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:44.969820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:05:44.969820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:05:44.969820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:05:44.970577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:05:44.970577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:05:44.970577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:44.970577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:44.970577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:05:44.970577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:05:44.970577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:05:44.970577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:05:44.970577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:05:44.970577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:05:44.970577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:44.970577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:44.970577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:44.970577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:05:44.970577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:44.970577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:44.970577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:44.970577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:05:44.970577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:44.972525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:05:44.972525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:05:44.972525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:44.972525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:05:44.972525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:44.972525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:05:44.972525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:05:44.972525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:44.972525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:05:44.972525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:44.972525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:05:44.972525 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:05:44.972525 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:05:44.974016 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:05:44.974016 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:05:44.974016 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:05:44.974016 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:05:44.974016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:44.974016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:05:44.974016 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:44.974016 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:05:44.974016 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:05:44.975073 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:44.975073 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:05:44.975073 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:44.975073 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:05:44.975073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:44.975073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:05:44.975073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:44.975073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:05:44.975073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:05:44.975073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:05:44.975073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:05:44.975073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:44.975073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:44.975073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:44.976101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:05:44.976101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:44.976101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:44.976101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:44.976101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:05:44.976101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:44.976101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:05:44.976101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:05:44.976101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:44.976101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:05:44.976101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:44.976101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:05:44.976101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:05:44.976101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:44.976737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:05:44.976737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:44.976737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:05:44.976737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:44.976737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:05:44.976737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:05:44.976737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:05:44.976737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:05:44.976737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:05:44.976737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:44.976737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:05:44.976737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:05:44.976737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:05:44.976737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:05:44.976737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:05:44.976737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:44.977365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:44.977365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:05:44.977365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:05:44.977365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:05:44.977365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:05:44.977365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:44.977365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:44.977365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:44.977365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:05:44.977365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:44.977365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:44.977365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:44.977365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:05:44.977365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:44.977365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:05:44.977365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:05:44.977365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:44.977365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:05:44.977365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:44.977890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:05:44.977890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:05:44.977890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:44.977890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:05:44.977890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:44.977890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:05:49.213375 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:05:49.214055 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:05:49.214275 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:05:49.215564 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:05:49.249757 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:05:49.250197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:49.251073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:05:49.251073 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:49.251073 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:05:49.251073 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:05:49.251073 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:49.251073 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:05:49.251073 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:49.251073 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:05:49.252058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:49.252058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:05:49.252058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:49.252058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:05:49.252058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:05:49.252058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:05:49.252058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:05:49.252058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:49.252058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:49.252058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:49.252058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:05:49.252058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:49.252931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:49.252931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:49.252931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:05:49.252931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:49.252931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:05:49.252931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:05:49.252931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:49.252931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:05:49.252931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:49.252931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:05:49.252931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:05:49.252931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:49.252931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:05:49.253652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:49.253652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:05:49.253652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:49.253652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:05:49.253652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:05:49.253652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:05:49.253652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:05:49.253652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:05:49.253652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:49.253652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:05:49.253652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:05:49.253652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:05:49.253652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:05:49.253652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:05:49.253652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:49.254379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:49.254379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:05:49.254379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:05:49.254379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:05:49.254379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:05:49.254379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:49.254379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:49.254379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:49.254379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:05:49.254379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:49.254379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:49.254379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:49.254379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:05:49.254379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:49.254379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:05:49.254379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:05:49.254379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:49.254379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:05:49.254379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:49.255257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:05:49.255257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:05:49.255257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:49.255257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:05:49.255257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:49.255257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:05:49.938386 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:05:49.938978 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:05:49.939166 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:05:49.940028 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:05:49.940298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:49.941222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:05:49.941222 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:49.941222 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:05:49.941222 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:05:49.941222 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:49.941222 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:05:49.941222 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:49.941222 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:05:49.942629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:49.942629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:05:49.942629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:49.942629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:05:49.942629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:05:49.942629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:05:49.942629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:05:49.942629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:49.942629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:49.942629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:49.942629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:05:49.942629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:49.943214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:49.943214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:49.943214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:05:49.943214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:49.943214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:05:49.943214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:05:49.943214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:49.943214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:05:49.943214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:49.943214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:05:49.943214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:05:49.943214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:49.943214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:05:49.943852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:49.943852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:05:49.944249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:49.944249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:05:49.944249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:05:49.944249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:05:49.944249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:05:49.944249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:05:49.944249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:49.944249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:05:49.944249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:05:49.944249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:05:49.944249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:05:49.944249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:05:49.944249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:49.944249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:49.944249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:05:49.944249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:05:49.944965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:05:49.944965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:05:49.944965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:49.944965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:49.944965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:49.944965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:05:49.944965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:49.944965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:49.944965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:49.944965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:05:49.944965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:49.944965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:05:49.944965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:05:49.944965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:49.944965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:05:49.944965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:49.944965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:05:49.944965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:05:49.944965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:49.946059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:05:49.946059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:49.946059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:05:49.948681 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:05:49.948958 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:05:49.949084 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:05:49.949181 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:05:49.949583 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:05:49.949738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:49.950199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:05:49.950199 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:49.950199 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:05:49.950199 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:05:49.950199 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:49.950199 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:05:49.950199 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:49.950199 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:05:49.951030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:49.951030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:05:49.951030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:05:49.951030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:05:49.951030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:49.951030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:05:49.951030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:49.951030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:05:49.951030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:49.951030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:05:49.951030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:49.951030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:05:49.951030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:49.951030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:05:49.951030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:05:49.951970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:49.951970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:05:49.951970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:49.951970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:05:49.951970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:05:49.951970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:49.951970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:05:49.951970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:49.951970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:05:49.952428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:49.952428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:05:49.952428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:49.952428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:05:49.952428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:05:49.952428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:05:49.952428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:05:49.952428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:05:49.952428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:05:49.952428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:05:49.952428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:49.952428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:49.953049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:49.953049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:05:49.953049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:49.953049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:49.953049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:49.953049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:05:49.953049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:49.953049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:05:49.953049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:05:49.953049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:49.953049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:05:49.953049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:49.953049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:05:49.953049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:05:49.953765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:49.953765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:05:49.953765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:49.953765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:05:49.953765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:49.953765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:05:49.953765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:05:49.953765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:05:49.953765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:05:49.953765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:05:49.953765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:49.953765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:05:49.953765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:05:49.953765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:05:49.954755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:05:49.954755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:05:49.954755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:49.954755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:49.954755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:05:49.954755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:05:49.954755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:05:49.954755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:05:49.954755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:05:49.954755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:05:49.954755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:05:49.954755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:49.954755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:49.954755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:49.954755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:05:49.954755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:49.954755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:49.954755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:49.954755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:05:49.955819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:49.955819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:05:49.955819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:05:49.955819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:49.955819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:05:49.955819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:49.955819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:05:49.955819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:05:49.955819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:49.955819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:05:49.955819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:49.955819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:05:49.962800 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:05:49.963040 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:05:49.963177 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:05:49.963425 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:05:49.964165 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:05:49.964272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:49.965043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:05:49.965043 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:49.965043 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:05:49.965043 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:05:49.965043 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:49.965043 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:05:49.965043 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:49.965043 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:05:49.966153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:49.966153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:05:49.966153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:05:49.966153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:05:49.966153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:49.966153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:05:49.966153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:49.966153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:05:49.966153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:49.966153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:05:49.966153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:49.966153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:05:49.966153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:49.966153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:05:49.966153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:05:49.966794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:49.966794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:05:49.966794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:49.966794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:05:49.966794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:05:49.966794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:49.966794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:05:49.966794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:49.966794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:05:49.967551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:49.967551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:05:49.967551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:49.967551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:05:49.967551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:05:49.967551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:05:49.967551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:05:49.967551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:05:49.967551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:05:49.967551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:49.967551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:49.967960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:49.967960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:05:49.967960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:49.967960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:49.967960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:49.967960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:05:49.967960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:49.967960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:05:49.967960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:05:49.967960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:49.967960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:05:49.967960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:49.967960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:05:49.967960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:05:49.968522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:49.968522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:05:49.968522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:49.968522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:05:49.969188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:49.969188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:05:49.969188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:05:49.969188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:05:49.969188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:05:49.969188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:05:49.969188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:49.969188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:05:49.969188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:05:49.969188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:05:49.969188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:05:49.969188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:05:49.969188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:49.969188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:49.969188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:05:49.969188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:05:49.969188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:05:49.969828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:05:49.969828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:05:49.969828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:05:49.969828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:49.969828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:49.969828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:49.969828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:05:49.969828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:49.969828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:49.969828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:49.969828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:05:49.969828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:49.969828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:05:49.969828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:05:49.969828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:49.969828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:05:49.969828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:49.969828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:05:49.969828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:05:49.970542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:49.970542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:05:49.970542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:49.970542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:05:49.970542 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:05:49.970817 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:05:49.970948 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:05:49.971199 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:05:49.971670 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:05:49.972006 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:05:49.972164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:49.972764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:05:49.972764 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:49.972764 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:05:49.972764 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:05:49.972764 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:49.972764 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:05:49.972764 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:49.972764 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:05:49.973382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:49.973382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:05:49.973382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:49.973382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:05:49.973382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:05:49.973382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:05:49.973382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:05:49.973382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:49.973382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:49.973382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:49.973382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:05:49.973382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:49.973944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:49.973944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:49.973944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:05:49.973944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:49.973944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:05:49.973944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:05:49.973944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:49.973944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:05:49.973944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:49.973944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:05:49.973944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:05:49.973944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:49.973944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:05:49.974544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:49.974544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:05:49.974544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:49.974544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:05:49.974544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:05:49.974544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:05:49.974544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:05:49.974544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:05:49.974544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:49.974544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:05:49.974544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:05:49.974544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:05:49.974544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:05:49.974544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:05:49.974544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:49.975176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:49.975176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:05:49.975176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:05:49.975176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:05:49.975176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:05:49.975176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:49.975176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:49.975176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:49.975176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:05:49.975176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:49.975176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:49.975176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:49.975176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:05:49.975176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:49.975176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:05:49.975176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:05:49.975176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:49.975176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:05:49.975176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:49.976062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:05:49.976062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:05:49.976062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:49.976062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:05:49.976062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:49.976062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:05:54.256173 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:05:54.257011 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:05:54.257011 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:05:54.258603 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:05:54.294050 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:05:54.294378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:54.295219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:05:54.295219 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:54.295219 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:05:54.295219 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:05:54.295219 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:54.295219 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:05:54.295219 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:54.295219 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:05:54.296133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:54.296133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:05:54.296133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:54.296133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:05:54.296133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:05:54.296133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:05:54.296133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:05:54.296133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:54.296133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:54.296133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:54.296133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:05:54.296133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:54.297466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:54.297466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:54.297466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:05:54.297466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:54.297466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:05:54.297466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:05:54.297466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:54.297466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:05:54.297466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:54.297466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:05:54.297466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:05:54.297466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:54.297466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:05:54.298079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:54.298079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:05:54.298079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:54.298079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:05:54.298079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:05:54.298079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:05:54.298079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:05:54.298079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:05:54.298079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:54.298079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:05:54.298079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:05:54.298079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:05:54.298079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:05:54.298079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:05:54.298079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:54.298755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:54.298755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:05:54.298755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:05:54.298755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:05:54.298755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:05:54.298755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:54.298755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:54.298755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:54.298755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:05:54.298755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:54.298755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:54.298755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:54.298755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:05:54.298755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:54.298755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:05:54.298755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:05:54.298755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:54.298755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:05:54.298755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:54.300305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:05:54.300305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:05:54.300305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:54.300305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:05:54.300305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:54.300305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:05:54.939251 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:05:54.939694 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:05:54.940148 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:05:54.941682 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:05:54.941682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:54.941939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:05:54.941939 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:54.941939 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:05:54.941939 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:05:54.941939 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:54.941939 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:05:54.941939 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:54.941939 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:05:54.945443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:54.945443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:05:54.945443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:54.945443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:05:54.945443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:05:54.945443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:05:54.945443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:05:54.945443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:54.945443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:54.945443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:54.945443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:05:54.945443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:54.946097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:54.946097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:54.946097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:05:54.946097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:54.946097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:05:54.946097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:05:54.946097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:54.946097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:05:54.946097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:54.946097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:05:54.946097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:05:54.946097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:54.946097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:05:54.946701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:54.946701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:05:54.947157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:54.947157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:05:54.947157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:05:54.947157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:05:54.947157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:05:54.947157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:05:54.947157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:54.947157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:05:54.947157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:05:54.947157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:05:54.947157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:05:54.947157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:05:54.947157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:54.947157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:54.947157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:05:54.947157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:05:54.947716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:05:54.947716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:05:54.947716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:54.947716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:54.947716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:54.947716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:05:54.947716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:54.947716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:54.947716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:54.947716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:05:54.947716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:54.947716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:05:54.947716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:05:54.947716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:54.947716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:05:54.947716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:54.947716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:05:54.947716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:05:54.947716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:54.948347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:05:54.948347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:54.948347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:05:54.949045 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:05:54.949539 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:05:54.949539 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:05:54.949539 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:05:54.949779 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:05:54.950296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:54.952995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:05:54.952995 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:54.952995 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:05:54.952995 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:05:54.952995 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:54.952995 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:05:54.952995 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:54.952995 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:05:54.952995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:54.952995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:05:54.952995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:05:54.952995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:05:54.952995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:54.952995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:05:54.952995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:54.952995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:05:54.952995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:54.952995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:05:54.954007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:54.954007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:05:54.954007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:54.954007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:05:54.954007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:05:54.954007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:54.954007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:05:54.954007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:54.954007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:05:54.954007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:05:54.954007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:54.954007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:05:54.954007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:54.954007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:05:54.954007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:54.954801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:05:54.954801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:54.954801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:05:54.954801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:05:54.954801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:05:54.954801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:05:54.954801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:05:54.954801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:05:54.954801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:05:54.954801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:54.954801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:54.954801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:54.954801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:05:54.955516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:54.955516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:54.955516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:54.955516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:05:54.955516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:54.955516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:05:54.955516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:05:54.955516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:54.955516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:05:54.955516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:54.955516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:05:54.955516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:05:54.955516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:54.955516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:05:54.956125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:54.956125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:05:54.956125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:54.956125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:05:54.956125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:05:54.956125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:05:54.956125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:05:54.956125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:05:54.956125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:54.956125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:05:54.956125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:05:54.956125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:05:54.956125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:05:54.956125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:05:54.956125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:54.956545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:54.956545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:05:54.956545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:05:54.956545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:05:54.956545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:05:54.956545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:05:54.956545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:05:54.956545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:05:54.956545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:54.956545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:54.956545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:54.956545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:05:54.956545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:54.956545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:54.956545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:54.956545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:05:54.956545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:54.956545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:05:54.956545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:05:54.957011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:54.957011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:05:54.957011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:54.957011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:05:54.957011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:05:54.957011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:54.957011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:05:54.957011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:54.957011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:05:54.963290 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:05:54.963530 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:05:54.963661 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:05:54.963661 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:05:54.965005 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:05:54.965005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:54.965207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:05:54.965207 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:54.965207 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:05:54.965207 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:05:54.965207 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:54.965207 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:05:54.965207 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:54.965207 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:05:54.967093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:54.967093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:05:54.967093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:05:54.967093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:05:54.967093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:54.967093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:05:54.967093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:54.967093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:05:54.967093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:54.967093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:05:54.967093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:54.967093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:05:54.967093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:54.967093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:05:54.967093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:05:54.967504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:54.967504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:05:54.967504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:54.967504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:05:54.967504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:05:54.967504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:54.967504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:05:54.967504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:54.967504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:05:54.967504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:54.967504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:05:54.967504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:54.967504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:05:54.967504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:05:54.968125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:05:54.968125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:05:54.968125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:05:54.968125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:05:54.968125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:54.968125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:54.968125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:54.968125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:05:54.968125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:54.968125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:54.968125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:54.968125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:05:54.968125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:54.969532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:05:54.969532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:05:54.969532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:54.969532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:05:54.969532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:54.969532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:05:54.969532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:05:54.969532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:54.969532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:05:54.969532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:54.969532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:05:54.969532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:54.970528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:05:54.970528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:05:54.970528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:05:54.970528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:05:54.970528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:05:54.970528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:54.970528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:05:54.970528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:05:54.970528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:05:54.970528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:05:54.970528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:05:54.970528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:54.970528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:54.970528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:05:54.970528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:05:54.970528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:05:54.970528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:05:54.970528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:05:54.970528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:05:54.972129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:54.972129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:54.972129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:54.972129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:05:54.972129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:54.972129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:54.972129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:54.972129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:05:54.972129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:54.972129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:05:54.972129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:05:54.972129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:54.972129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:05:54.972129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:54.972129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:05:54.972129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:05:54.972129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:54.972129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:05:54.972129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:54.972129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:05:54.972931 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:05:54.974194 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:05:54.974194 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:05:54.974194 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:05:54.974194 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:05:54.974675 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:05:54.974798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:54.976792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:05:54.976792 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:54.976792 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:05:54.976792 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:05:54.976792 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:54.976792 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:05:54.976792 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:54.976792 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:05:54.976792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:54.976792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:05:54.976792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:54.976792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:05:54.976792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:05:54.976792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:05:54.976792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:05:54.977374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:54.977374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:54.977374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:54.977374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:05:54.977374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:54.977374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:54.977374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:54.977374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:05:54.977374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:54.977374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:05:54.977374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:05:54.977374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:54.977374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:05:54.977374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:54.977872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:05:54.977872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:05:54.977872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:54.977872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:05:54.977872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:54.977872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:05:54.977872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:54.977872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:05:54.977872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:05:54.977872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:05:54.977872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:05:54.977872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:05:54.977872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:54.977872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:05:54.978382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:05:54.978382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:05:54.978382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:05:54.978382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:05:54.978382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:54.978382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:54.978382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:05:54.978382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:05:54.978382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:05:54.978382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:05:54.978382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:54.978382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:54.978382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:54.978382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:05:54.978382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:54.978382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:54.978382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:54.978382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:05:54.978382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:54.978382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:05:54.979127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:05:54.979127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:54.979127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:05:54.979127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:54.979127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:05:54.979127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:05:54.979127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:54.979127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:05:54.979127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:54.979127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:05:59.256716 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:05:59.257531 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:05:59.257675 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:05:59.258958 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:05:59.292927 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:05:59.293318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:59.294140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:05:59.294140 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:59.294140 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:05:59.294140 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:05:59.294140 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:59.294140 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:05:59.294140 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:59.294140 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:05:59.295018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:59.295018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:05:59.295018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:59.295018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:05:59.295018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:05:59.295018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:05:59.295018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:05:59.295018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:59.295018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:59.295018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:59.295018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:05:59.295018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:59.296045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:59.296045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:59.296045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:05:59.296045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:59.296045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:05:59.296045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:05:59.296045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:59.296045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:05:59.296045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:59.296045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:05:59.296045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:05:59.296045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:59.296045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:05:59.296761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:59.296761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:05:59.296761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:59.296761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:05:59.296761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:05:59.296761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:05:59.296761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:05:59.296761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:05:59.296761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:59.296761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:05:59.296761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:05:59.296761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:05:59.296761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:05:59.296761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:05:59.296761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:59.298163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:59.298163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:05:59.298163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:05:59.298163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:05:59.298163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:05:59.298163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:59.298163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:59.298163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:59.298163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:05:59.298163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:59.298163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:59.298163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:59.298163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:05:59.298163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:59.298163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:05:59.298163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:05:59.298163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:59.298163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:05:59.298163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:59.299940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:05:59.299940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:05:59.299940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:59.299940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:05:59.299940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:59.299940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:05:59.939848 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:05:59.940503 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:05:59.940717 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:05:59.941635 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:05:59.941961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:59.942944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:05:59.942944 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:59.942944 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:05:59.942944 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:05:59.942944 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:59.942944 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:05:59.942944 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:59.942944 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:05:59.944534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:59.944534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:05:59.944534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:59.944534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:05:59.944534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:05:59.944534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:05:59.944534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:05:59.944534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:59.944534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:59.944534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:59.944534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:05:59.944534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:59.945122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:59.945122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:59.945122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:05:59.945122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:59.945122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:05:59.945122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:05:59.945122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:59.945122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:05:59.945122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:59.945122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:05:59.945122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:05:59.945122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:59.945122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:05:59.945725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:59.945725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:05:59.946610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:59.946610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:05:59.946610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:05:59.946610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:05:59.946610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:05:59.946610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:05:59.946610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:59.946610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:05:59.946610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:05:59.946610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:05:59.946610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:05:59.946610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:05:59.946610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:59.946610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:59.946610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:05:59.946610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:05:59.947958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:05:59.947958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:05:59.947958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:59.947958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:59.947958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:59.947958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:05:59.947958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:59.947958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:59.947958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:59.947958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:05:59.947958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:59.947958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:05:59.947958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:05:59.947958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:59.947958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:05:59.947958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:59.947958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:05:59.947958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:05:59.947958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:59.949302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:05:59.949302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:59.949302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:05:59.949608 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:05:59.949717 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:05:59.949822 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:05:59.949924 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:05:59.950387 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:05:59.950713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:59.951515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:05:59.951515 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:59.951515 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:05:59.951515 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:05:59.951515 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:59.951515 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:05:59.951515 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:59.951515 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:05:59.952297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:59.952297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:05:59.952297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:05:59.952297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:05:59.952297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:59.952297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:05:59.952297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:59.952297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:05:59.952297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:59.952297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:05:59.952297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:59.952297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:05:59.952297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:59.952297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:05:59.952297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:05:59.953217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:59.953217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:05:59.953217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:59.953217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:05:59.953217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:05:59.953217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:59.953217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:05:59.953217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:59.953217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:05:59.953217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:59.953217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:05:59.953217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:59.953217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:05:59.953217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:05:59.953783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:05:59.953783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:05:59.953783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:05:59.953783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:05:59.953783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:05:59.953783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:59.953783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:59.953783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:59.953783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:05:59.953783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:59.953783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:59.953783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:59.953783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:05:59.954118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:59.954118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:05:59.954118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:05:59.954118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:59.954118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:05:59.954118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:59.954118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:05:59.954118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:05:59.954118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:59.954118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:05:59.954118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:59.954118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:05:59.954438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:59.954438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:05:59.954438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:05:59.954438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:05:59.954438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:05:59.954438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:05:59.954438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:59.954438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:05:59.954438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:05:59.954438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:05:59.954438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:05:59.954438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:05:59.954438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:59.954438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:59.954438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:05:59.954438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:05:59.954850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:05:59.954850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:05:59.954850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:05:59.954850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:05:59.954850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:05:59.954850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:59.954850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:59.954850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:59.954850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:05:59.954850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:59.954850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:59.954850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:59.954850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:05:59.954850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:59.954850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:05:59.954850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:05:59.954850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:59.954850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:05:59.955296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:59.955296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:05:59.955296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:05:59.955296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:59.955296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:05:59.955296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:59.955296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:05:59.963708 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:05:59.964141 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:05:59.964338 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:05:59.964647 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:05:59.965173 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:05:59.965639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:59.966570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:05:59.966570 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:59.966570 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:05:59.966570 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:05:59.966570 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:59.966570 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:05:59.966570 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:59.966570 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:05:59.967899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:59.967899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:05:59.967899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:05:59.967899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:05:59.967899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:59.967899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:05:59.967899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:59.967899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:05:59.967899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:59.967899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:05:59.967899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:59.967899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:05:59.967899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:59.967899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:05:59.967899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:05:59.968630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:59.968630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:05:59.968630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:59.968630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:05:59.968630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:05:59.968630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:59.968630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:05:59.968630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:59.968630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:05:59.969324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:59.969324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:05:59.969324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:59.969324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:05:59.969324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:05:59.969324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:05:59.969324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:05:59.969324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:05:59.969324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:05:59.969324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:59.969324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:59.969865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:59.969865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:05:59.969865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:59.969865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:59.969865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:59.969865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:05:59.969865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:59.969865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:05:59.969865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:05:59.969865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:59.969865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:05:59.969865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:59.969865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:05:59.969865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:05:59.970422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:59.970422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:05:59.970422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:59.970422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:05:59.971150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:59.971150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:05:59.971150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:05:59.971150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:05:59.971150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:05:59.971150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:05:59.971150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:59.971150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:05:59.971150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:05:59.971150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:05:59.971150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:05:59.971150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:05:59.971150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:59.971150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:59.971150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:05:59.971150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:05:59.971150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:05:59.971915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:05:59.971915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:05:59.971915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:05:59.971915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:59.971915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:59.971915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:59.971915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:05:59.971915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:59.971915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:59.971915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:59.971915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:05:59.971915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:59.971915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:05:59.971915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:05:59.971915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:59.971915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:05:59.971915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:59.971915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:05:59.971915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:05:59.972687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:59.972687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:05:59.972687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:59.972687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:05:59.973493 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:05:59.973809 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:05:59.973921 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:05:59.974046 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:05:59.974525 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:05:59.974925 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:05:59.975149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:59.976086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:05:59.976086 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:59.976086 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:05:59.976086 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:05:59.976086 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:59.976086 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:05:59.976086 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:59.976086 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:05:59.976886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:59.976886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:05:59.976886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:59.976886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:05:59.976886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:05:59.976886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:05:59.976886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:05:59.976886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:59.976886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:59.976886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:59.976886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:05:59.976886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:59.977861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:05:59.977861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:59.977861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:05:59.977861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:59.977861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:05:59.977861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:05:59.977861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:59.977861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:05:59.977861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:59.977861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:05:59.977861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:05:59.977861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:59.977861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:05:59.978508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:59.978508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:05:59.978508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:59.978508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:05:59.978508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:05:59.978508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:05:59.978508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:05:59.978508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:05:59.978508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:59.978508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:05:59.978508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:05:59.978508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:05:59.978508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:05:59.978508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:05:59.978508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:05:59.979443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:05:59.979443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:05:59.979443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:05:59.979443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:05:59.979443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:05:59.979443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:59.979443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:59.979443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:05:59.979443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:05:59.979443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:05:59.979443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:05:59.979443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:05:59.979443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:05:59.979443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:05:59.979443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:05:59.979443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:05:59.979443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:05:59.979443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:05:59.979443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:05:59.979914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:05:59.979914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:05:59.979914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:05:59.979914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:05:59.979914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:05:59.979914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:06:04.257340 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:06:04.258245 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:06:04.258245 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:06:04.259599 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:06:04.301436 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:06:04.301819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:04.302753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:06:04.302753 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:04.302753 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:06:04.302753 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:06:04.302753 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:04.302753 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:06:04.302753 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:04.302753 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:06:04.303776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.045 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:04.303776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:06:04.303776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:04.303776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:06:04.303776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:06:04.303776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:06:04.303776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:06:04.303776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:04.303776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:04.303776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:04.303776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:06:04.303776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:04.304537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:04.304537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:04.304537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:06:04.304537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:04.304537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:06:04.304537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:06:04.304537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:04.304537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:06:04.304537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:04.304537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:06:04.304537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:06:04.304537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:04.304537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:06:04.305227 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:04.305227 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:06:04.305227 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:04.305227 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:06:04.305227 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:06:04.305227 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:06:04.305227 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:06:04.305227 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:06:04.305227 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:04.305227 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:06:04.305227 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:06:04.305227 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:06:04.305227 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:06:04.305227 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:06:04.305227 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:04.305984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:04.305984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:06:04.305984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:06:04.305984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:06:04.305984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:06:04.305984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:04.305984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:04.305984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:04.305984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:06:04.305984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:04.305984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:04.305984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:04.305984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:06:04.305984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:04.305984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:06:04.305984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:06:04.305984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:04.305984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:06:04.305984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:04.306693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:06:04.306693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:06:04.306693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:04.306693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:06:04.306693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:04.306693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:06:04.940598 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:06:04.941283 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:06:04.941525 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:06:04.942496 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:06:04.942882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:04.943872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:06:04.943872 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:04.943872 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:06:04.943872 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:06:04.943872 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:04.943872 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:06:04.943872 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:04.943872 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:06:04.945437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:04.945437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:06:04.945437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:04.945437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:06:04.945437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:06:04.945437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:06:04.945437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:06:04.945437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:04.945437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:04.945437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:04.945437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:06:04.945437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:04.946038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:04.946038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:04.946038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:06:04.946038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:04.946038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:06:04.946038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:06:04.946038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:04.946038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:06:04.946038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:04.946038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:06:04.946038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:06:04.946038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:04.946038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:06:04.946649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:04.946649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:06:04.947079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:04.947079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:06:04.947079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:06:04.947079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:06:04.947079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:06:04.947079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:06:04.947079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:04.947079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:06:04.947079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:06:04.947079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:06:04.947079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:06:04.947079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:06:04.947079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:04.947079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:04.947079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:06:04.947079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:06:04.947942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:06:04.947942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:06:04.947942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:04.947942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:04.947942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:04.947942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:06:04.947942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:04.947942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:04.947942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:04.947942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:06:04.947942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:04.947942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:06:04.947942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:06:04.947942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:04.947942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:06:04.947942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:04.947942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:06:04.947942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:06:04.947942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:04.949083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:06:04.949083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:04.949083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:06:04.949931 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:06:04.950276 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:06:04.950480 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:06:04.950669 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:06:04.951017 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:06:04.951242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:04.951743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:06:04.951743 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:04.951743 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:06:04.951743 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:06:04.951743 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:04.951743 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:06:04.951743 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:04.951743 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:06:04.952480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:04.952480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:06:04.952480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:06:04.952480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:06:04.952480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:04.952480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:06:04.952480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:04.952480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:06:04.952480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:04.952480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:06:04.952480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:04.952480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:06:04.952480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:04.952480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:06:04.952480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:06:04.953294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:04.953294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:06:04.953294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:04.953294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:06:04.953294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:06:04.953294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:04.953294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:06:04.953294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:04.953294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:06:04.953294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:04.953294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:06:04.953294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:04.953294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:06:04.953294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:06:04.953890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:06:04.953890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:06:04.953890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:06:04.953890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:06:04.953890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:06:04.953890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:04.953890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:04.953890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:04.953890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:06:04.953890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:04.953890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:04.953890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:04.953890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:06:04.954267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:04.954267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:06:04.954267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:06:04.954267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:04.954267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:06:04.954267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:04.954267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:06:04.954267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:06:04.954267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:04.954267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:06:04.954267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:04.954267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:06:04.954734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:04.954734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:06:04.954734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:06:04.954734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:06:04.954734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:06:04.954734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:06:04.954734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:04.954734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:06:04.954734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:06:04.954734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:06:04.954734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:06:04.954734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:06:04.954734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:04.954734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:04.954734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:06:04.954734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:06:04.955181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:06:04.955181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:06:04.955181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:06:04.955181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:06:04.955181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:06:04.955181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:04.955181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:04.955181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:04.955181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:06:04.955181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:04.955181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:04.955181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:04.955181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:06:04.955181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:04.955181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:06:04.955181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:06:04.955181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:04.955181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:06:04.955701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:04.955701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:06:04.955701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:06:04.955701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:04.955701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:06:04.955701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:04.955701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:06:04.964225 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:06:04.964532 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:06:04.964636 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:06:04.964761 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:06:04.965323 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:06:04.965780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:04.966675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:06:04.966675 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:04.966675 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:06:04.966675 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:06:04.966675 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:04.966675 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:06:04.966675 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:04.966675 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:06:04.967815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:04.967815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:06:04.967815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:06:04.967815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:06:04.967815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:04.967815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:06:04.967815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:04.967815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:06:04.967815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:04.967815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:06:04.967815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:04.967815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:06:04.967815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:04.967815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:06:04.967815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:06:04.968473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:04.968473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:06:04.968473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:04.968473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:06:04.968473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:06:04.968473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:04.968473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:06:04.968473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:04.968473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:06:04.969199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:04.969199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:06:04.969199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:04.969199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:06:04.969199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:06:04.969199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:06:04.969199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:06:04.969199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:06:04.969199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:06:04.969199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:04.969199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:04.969783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:04.969783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:06:04.969783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:04.969783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:04.969783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:04.969783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:06:04.969783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:04.969783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:06:04.969783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:06:04.969783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:04.969783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:06:04.969783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:04.969783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:06:04.969783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:06:04.970433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:04.970433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:06:04.970433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:04.970433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:06:04.970887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:04.970887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:06:04.970887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:06:04.970887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:06:04.970887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:06:04.970887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:06:04.970887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:04.970887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:06:04.970887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:06:04.970887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:06:04.970887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:06:04.970887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:06:04.970887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:04.970887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:04.970887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:06:04.970887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:06:04.970887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:06:04.971773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:06:04.971773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:06:04.971773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:06:04.971773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:04.971773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:04.971773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:04.971773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:06:04.971773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:04.971773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:04.971773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:04.971773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:06:04.971773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:04.971773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:06:04.971773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:06:04.971773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:04.971773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:06:04.971773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:04.971773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:06:04.971773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:06:04.972852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:04.972852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:06:04.972852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:04.972852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:06:04.973944 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:06:04.974182 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:06:04.974286 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:06:04.974457 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:06:04.974923 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:06:04.975375 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:06:04.975643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:04.976421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:06:04.976421 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:04.976421 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:06:04.976421 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:06:04.976421 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:04.976421 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:06:04.976421 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:04.976421 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:06:04.977229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:04.977229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:06:04.977229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:04.977229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:06:04.977229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:06:04.977229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:06:04.977229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:06:04.977229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:04.977229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:04.977229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:04.977229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:06:04.977229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:04.977615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:04.977615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:04.977615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:06:04.977615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:04.977615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:06:04.977615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:06:04.977615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:04.977615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:06:04.977615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:04.977615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:06:04.977615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:06:04.977615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:04.977615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:06:04.977972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:04.977972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:06:04.978157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:04.978157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:06:04.978157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:06:04.978157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:06:04.978157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:06:04.978157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:06:04.978157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:04.978157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:06:04.978157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:06:04.978157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:06:04.978157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:06:04.978157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:06:04.978157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:04.978157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:04.978157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:06:04.978157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:06:04.978674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:06:04.978674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:06:04.978674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:04.978674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:04.978674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:04.978674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:06:04.978674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:04.978674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:04.978674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:04.978674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:06:04.978674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:04.978674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:06:04.978674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:06:04.978674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:04.978674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:06:04.978674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:04.978674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:06:04.978674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:06:04.978674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:04.979184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:06:04.979184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:04.979184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:06:09.257925 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:06:09.258954 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:06:09.258954 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:06:09.260091 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:06:09.289904 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:06:09.290324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:09.291283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:06:09.291283 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:09.291283 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:06:09.291283 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:06:09.291283 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:09.291283 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:06:09.291283 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:09.291283 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:06:09.291999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:09.291999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:06:09.291999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:09.291999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:06:09.291999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:06:09.291999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:06:09.291999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:06:09.291999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:09.291999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:09.291999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:09.291999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:06:09.291999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:09.292665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:09.292665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:09.292665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:06:09.292665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:09.292665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:06:09.292665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:06:09.292665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:09.292665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:06:09.292665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:09.292665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:06:09.292665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:06:09.292665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:09.292665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:06:09.293280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:09.293280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:06:09.293280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:09.293280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:06:09.293280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:06:09.293280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:06:09.293280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:06:09.293280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:06:09.293280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:09.293280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:06:09.293280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:06:09.293280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:06:09.293280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:06:09.293280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:06:09.293280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:09.296948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:09.296948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:06:09.296948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:06:09.296948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:06:09.296948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:06:09.296948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:09.296948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:09.296948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:09.296948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:06:09.296948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:09.296948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:09.296948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:09.296948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:06:09.296948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:09.296948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:06:09.296948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:06:09.296948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:09.296948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:06:09.296948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:09.297959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:06:09.297959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:06:09.297959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:09.297959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:06:09.297959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:09.297959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:06:09.941291 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:06:09.942001 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:06:09.942154 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:06:09.943648 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:06:09.944072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:09.944952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:06:09.944952 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:09.944952 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:06:09.944952 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:06:09.944952 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:09.944952 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:06:09.944952 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:09.944952 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:06:09.945967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:09.945967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:06:09.945967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:09.945967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:06:09.945967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:06:09.945967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:06:09.945967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:06:09.945967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:09.945967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:09.945967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:09.945967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:06:09.945967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:09.946651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:09.946651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:09.946651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:06:09.946651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:09.946651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:06:09.946651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:06:09.946651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:09.946651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:06:09.946651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:09.946651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:06:09.946651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:06:09.946651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:09.946651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:06:09.947277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:09.947277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:06:09.947277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:09.947277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:06:09.947277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:06:09.947277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:06:09.947277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:06:09.947277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:06:09.947277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:09.947277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:06:09.947277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:06:09.947277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:06:09.947277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:06:09.947277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:06:09.947277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:09.947767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:09.947767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:06:09.947767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:06:09.947767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:06:09.947767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:06:09.947767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:09.947767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:09.947767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:09.947767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:06:09.947767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:09.947767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:09.947767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:09.947767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:06:09.947767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:09.947767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:06:09.947767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:06:09.947767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:09.947767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:06:09.947767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:09.948306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:06:09.948306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:06:09.948306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:09.948306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:06:09.948306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:09.948306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:06:09.950383 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:06:09.950707 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:06:09.950904 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:06:09.950904 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:06:09.951244 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:06:09.951434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:09.951946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:06:09.951946 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:09.951946 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:06:09.951946 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:06:09.951946 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:09.951946 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:06:09.951946 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:09.951946 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:06:09.952691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:09.952691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:06:09.952691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:06:09.952691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:06:09.952691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:09.952691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:06:09.952691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:09.952691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:06:09.952691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:09.952691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:06:09.952691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:09.952691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:06:09.952691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:09.952691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:06:09.952691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:06:09.953450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:09.953450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:06:09.953450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:09.953450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:06:09.953450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:06:09.953450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:09.953450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:06:09.953450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:09.953450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:06:09.953450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:09.953450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:06:09.953450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:09.953450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:06:09.953450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:06:09.954126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:06:09.954126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:06:09.954126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:06:09.954126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:06:09.954126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:06:09.954126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:09.954126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:09.954126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:09.954126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:06:09.954126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:09.954126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:09.954126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:09.954126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:06:09.954853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:09.954853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:06:09.954853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:06:09.954853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:09.954853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:06:09.954853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:09.954853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:06:09.954853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:06:09.954853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:09.954853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:06:09.954853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:09.954853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:06:09.955481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:09.955481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:06:09.955481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:06:09.955481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:06:09.955481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:06:09.955481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:06:09.955481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:09.955481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:06:09.955481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:06:09.955481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:06:09.955481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:06:09.955481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:06:09.955481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:09.955481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:09.955481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:06:09.955481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:06:09.956224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:06:09.956224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:06:09.956224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:06:09.956224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:06:09.956224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:06:09.956224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:09.956224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:09.956224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:09.956224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:06:09.956224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:09.956224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:09.956224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:09.956224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:06:09.956224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:09.956224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:06:09.956224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:06:09.956224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:09.956224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:06:09.957089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:09.957089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:06:09.957089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:06:09.957089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:09.957089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:06:09.957089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:09.957089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:06:09.964637 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:06:09.964883 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:06:09.965003 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:06:09.965121 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:06:09.965617 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:06:09.965963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:09.966641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:06:09.966641 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:09.966641 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:06:09.966641 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:06:09.966641 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:09.966641 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:06:09.966641 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:09.966641 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:06:09.967386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:09.967386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:06:09.967386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:06:09.967386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:06:09.967386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:09.967386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:06:09.967386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:09.967386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:06:09.967386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:09.967386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:06:09.967386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:09.967386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:06:09.967386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:09.967386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:06:09.967386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:06:09.968260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:09.968260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:06:09.968260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:09.968260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:06:09.968260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:06:09.968260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:09.968260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:06:09.968260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:09.968260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:06:09.968260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:09.968260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:06:09.968260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:09.968260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:06:09.968260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:06:09.969103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:06:09.969103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:06:09.969103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:06:09.969103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:06:09.969103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:09.969103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:09.969103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:09.969103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:06:09.969103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:09.969103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:09.969103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:09.969103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:06:09.969103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:09.969769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:06:09.969769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:06:09.969769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:09.969769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:06:09.969769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:09.969769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:06:09.969769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:06:09.969769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:09.969769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:06:09.969769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:09.969769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:06:09.969769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:09.970683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:06:09.970683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:06:09.970683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:06:09.970683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:06:09.970683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:06:09.970683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:09.970683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:06:09.970683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:06:09.970683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:06:09.970683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:06:09.970683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:06:09.970683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:09.970683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:09.970683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:06:09.970683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:06:09.970683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:06:09.970683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:06:09.970683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:06:09.970683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:06:09.971928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:09.971928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:09.971928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:09.971928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:06:09.971928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:09.971928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:09.971928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:09.971928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:06:09.971928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:09.971928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:06:09.971928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:06:09.971928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:09.971928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:06:09.971928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:09.971928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:06:09.971928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:06:09.971928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:09.971928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:06:09.971928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:09.971928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:06:09.974318 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:06:09.974630 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:06:09.974712 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:06:09.974881 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:06:09.975194 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:06:09.975423 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:06:09.975532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:09.976019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:06:09.976019 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:09.976019 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:06:09.976019 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:06:09.976019 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:09.976019 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:06:09.976019 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:09.976019 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:06:09.976835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:09.976835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:06:09.976835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:09.976835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:06:09.976835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:06:09.976835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:06:09.976835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:06:09.976835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:09.976835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:09.976835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:09.976835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:06:09.976835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:09.977211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:09.977211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:09.977211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:06:09.977211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:09.977211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:06:09.977211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:06:09.977211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:09.977211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:06:09.977211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:09.977211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:06:09.977211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:06:09.977211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:09.977211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:06:09.977622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:09.977622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:06:09.977799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:09.977799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:06:09.977799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:06:09.977799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:06:09.977799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:06:09.977799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:06:09.977799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:09.977799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:06:09.977799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:06:09.977799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:06:09.977799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:06:09.977799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:06:09.977799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:09.977799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:09.977799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:06:09.977799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:06:09.978312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:06:09.978312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:06:09.978312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:09.978312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:09.978312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:09.978312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:06:09.978312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:09.978312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:09.978312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:09.978312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:06:09.978312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:09.978312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:06:09.978312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:06:09.978312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:09.978312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:06:09.978312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:09.978312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:06:09.978312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:06:09.978312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:09.978892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:06:09.978892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:09.978892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:06:14.258678 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:06:14.259564 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:06:14.259765 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:06:14.261292 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:06:14.304315 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:06:14.305230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:14.306707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:06:14.306707 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:14.306707 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:06:14.306707 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:06:14.306707 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:14.306707 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:06:14.306707 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:14.306707 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:06:14.307756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.047 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:14.307756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:06:14.307756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:14.307756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:06:14.307756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:06:14.307756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:06:14.307756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:06:14.307756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:14.307756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:14.307756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:14.307756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:06:14.307756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:14.308341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:14.308341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:14.308341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:06:14.308341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:14.308341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:06:14.308341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:06:14.308341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:14.308341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:06:14.308341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:14.308341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:06:14.308341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:06:14.308341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:14.308341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:06:14.308938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:14.308938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:06:14.308938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:14.308938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:06:14.308938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:06:14.308938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:06:14.308938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:06:14.308938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:06:14.308938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:14.308938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:06:14.308938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:06:14.308938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:06:14.308938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:06:14.308938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:06:14.308938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:14.309610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:14.309610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:06:14.309610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:06:14.309610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:06:14.309610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:06:14.309610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:14.309610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:14.309610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:14.309610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:06:14.309610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:14.309610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:14.309610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:14.309610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:06:14.309610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:14.309610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:06:14.309610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:06:14.309610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:14.309610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:06:14.309610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:14.310452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:06:14.310452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:06:14.310452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:14.310452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:06:14.310452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:14.310452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:06:14.942021 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:06:14.942579 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:06:14.942689 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:06:14.943607 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:06:14.943956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:14.944870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:06:14.944870 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:14.944870 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:06:14.944870 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:06:14.944870 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:14.944870 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:06:14.944870 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:14.944870 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:06:14.946087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:14.946087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:06:14.946087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:14.946087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:06:14.946087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:06:14.946087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:06:14.946087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:06:14.946087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:14.946087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:14.946087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:14.946087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:06:14.946087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:14.946970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:14.946970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:14.946970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:06:14.946970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:14.946970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:06:14.946970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:06:14.946970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:14.946970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:06:14.946970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:14.946970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:06:14.946970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:06:14.946970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:14.946970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:06:14.948018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:14.948018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:06:14.948018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:14.948018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:06:14.948018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:06:14.948018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:06:14.948018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:06:14.948018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:06:14.948018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:14.948018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:06:14.948018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:06:14.948018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:06:14.948018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:06:14.948018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:06:14.948018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:14.949130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:14.949130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:06:14.949130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:06:14.949130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:06:14.949130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:06:14.949130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:14.949130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:14.949130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:14.949130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:06:14.949130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:14.949130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:14.949130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:14.949130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:06:14.949130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:14.949130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:06:14.949130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:06:14.949130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:14.949130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:06:14.949130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:14.950471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:06:14.950471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:06:14.950471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:14.950471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:06:14.950471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:14.950471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:06:14.951257 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:06:14.951257 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:06:14.951257 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:06:14.951257 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:06:14.951642 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:06:14.951822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:14.952465 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:06:14.952465 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:14.952465 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:06:14.952465 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:06:14.952465 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:14.952465 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:06:14.952465 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:14.952465 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:06:14.953228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:14.953228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:06:14.953228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:06:14.953228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:06:14.953228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:14.953228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:06:14.953228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:14.953228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:06:14.953228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:14.953228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:06:14.953228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:14.953228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:06:14.953228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:14.953228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:06:14.953228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:06:14.953642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:14.953642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:06:14.953642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:14.953642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:06:14.953642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:06:14.953642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:14.953642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:06:14.953642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:14.953642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:06:14.954063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:14.954063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:06:14.954063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:14.954063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:06:14.954063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:06:14.954063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:06:14.954063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:06:14.954063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:06:14.954063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:06:14.954063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:06:14.954063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:14.954063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:14.954424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:14.954424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:06:14.954424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:14.954424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:14.954424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:14.954424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:06:14.954424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:14.954424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:06:14.954424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:06:14.954424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:14.954424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:06:14.954424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:14.954424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:06:14.954424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:06:14.954892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:14.954892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:06:14.954892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:14.954892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:06:14.955064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:14.955064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:06:14.955064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:06:14.955064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:06:14.955064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:06:14.955064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:06:14.955064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:14.955064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:06:14.955064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:06:14.955064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:06:14.955064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:06:14.955064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:06:14.955064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:14.955064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:14.955064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:06:14.955064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:06:14.955609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:06:14.955609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:06:14.955609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:06:14.955609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:06:14.955609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:06:14.955609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:14.955609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:14.955609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:14.955609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:06:14.955609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:14.955609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:14.955609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:14.955609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:06:14.955609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:14.955609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:06:14.955609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:06:14.955609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:14.955609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:06:14.956205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:14.956205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:06:14.956205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:06:14.956205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:14.956205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:06:14.956205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:14.956205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:06:14.965019 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:06:14.965324 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:06:14.965455 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:06:14.965573 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:06:14.966055 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:06:14.966482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:14.967250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:06:14.967250 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:14.967250 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:06:14.967250 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:06:14.967250 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:14.967250 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:06:14.967250 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:14.967250 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:06:14.968451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:14.968451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:06:14.968451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:06:14.968451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:06:14.968451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:14.968451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:06:14.968451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:14.968451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:06:14.968451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:14.968451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:06:14.968451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:14.968451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:06:14.968451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:14.968451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:06:14.968451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:06:14.969066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:14.969066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:06:14.969066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:14.969066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:06:14.969066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:06:14.969066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:14.969066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:06:14.969066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:14.969066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:06:14.969870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:14.969870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:06:14.969870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:14.969870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:06:14.969870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:06:14.969870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:06:14.969870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:06:14.969870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:06:14.969870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:06:14.969870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:14.969870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:14.970570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:14.970570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:06:14.970570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:14.970570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:14.970570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:14.970570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:06:14.970570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:14.970570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:06:14.970570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:06:14.970570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:14.970570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:06:14.970570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:14.970570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:06:14.970570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:06:14.971480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:14.971480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:06:14.971480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:14.971480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:06:14.971786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:14.971786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:06:14.971786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:06:14.971786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:06:14.971786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:06:14.971786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:06:14.971786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:14.971786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:06:14.971786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:06:14.971786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:06:14.971786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:06:14.971786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:06:14.971786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:14.971786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:14.971786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:06:14.971786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:06:14.971786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:06:14.972834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:06:14.972834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:06:14.972834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:06:14.972834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:14.972834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:14.972834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:14.972834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:06:14.972834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:14.972834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:14.972834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:14.972834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:06:14.972834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:14.972834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:06:14.972834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:06:14.972834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:14.972834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:06:14.972834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:14.972834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:06:14.972834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:06:14.973959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:14.973959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:06:14.973959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:14.973959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:06:14.975785 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:06:14.976180 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:06:14.976303 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:06:14.976473 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:06:14.976787 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:06:14.977020 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:06:14.977149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:14.977688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:06:14.977688 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:14.977688 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:06:14.977688 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:06:14.977688 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:14.977688 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:06:14.977688 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:14.977688 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:06:14.978485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:14.978485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:06:14.978485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:14.978485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:06:14.978485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:06:14.978485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:06:14.978485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:06:14.978485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:14.978485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:14.978485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:14.978485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:06:14.978485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:14.978939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:14.978939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:14.978939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:06:14.978939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:14.978939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:06:14.978939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:06:14.978939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:14.978939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:06:14.978939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:14.978939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:06:14.978939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:06:14.978939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:14.978939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:06:14.979449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:14.979449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:06:14.979449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:14.979449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:06:14.979449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:06:14.979449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:06:14.979449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:06:14.979449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:06:14.979449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:14.979449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:06:14.979449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:06:14.979449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:06:14.979449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:06:14.979449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:06:14.979449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:14.980032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:14.980032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:06:14.980032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:06:14.980032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:06:14.980032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:06:14.980032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:14.980032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:14.980032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:14.980032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:06:14.980032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:14.980032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:14.980032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:14.980032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:06:14.980032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:14.980032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:06:14.980032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:06:14.980032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:14.980032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:06:14.980032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:14.980920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:06:14.980920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:06:14.980920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:14.980920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:06:14.980920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:14.980920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:06:19.259388 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:06:19.260038 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:06:19.260038 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:06:19.261301 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:06:19.296564 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:06:19.297040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:19.297969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:06:19.297969 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:19.297969 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:06:19.297969 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:06:19.297969 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:19.297969 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:06:19.297969 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:19.297969 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:06:19.298906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:19.298906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:06:19.298906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:19.298906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:06:19.298906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:06:19.298906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:06:19.298906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:06:19.298906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:19.298906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:19.298906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:19.298906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:06:19.298906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:19.299566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:19.299566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:19.299566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:06:19.299566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:19.299566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:06:19.299566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:06:19.299566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:19.299566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:06:19.299566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:19.299566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:06:19.299566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:06:19.299566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:19.299566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:06:19.300132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:19.300132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:06:19.300132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:19.300132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:06:19.300132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:06:19.300132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:06:19.300132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:06:19.300132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:06:19.300132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:19.300132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:06:19.300132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:06:19.300132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:06:19.300132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:06:19.300132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:06:19.300132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:19.300866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:19.300866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:06:19.300866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:06:19.300866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:06:19.300866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:06:19.300866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:19.300866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:19.300866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:19.300866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:06:19.300866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:19.300866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:19.300866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:19.300866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:06:19.300866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:19.300866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:06:19.300866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:06:19.300866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:19.300866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:06:19.300866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:19.301730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:06:19.301730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:06:19.301730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:19.301730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:06:19.301730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:19.301730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:06:19.942604 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:06:19.943177 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:06:19.943372 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:06:19.944650 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:06:19.944966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:19.945909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:06:19.945909 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:19.945909 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:06:19.945909 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:06:19.945909 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:19.945909 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:06:19.945909 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:19.945909 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:06:19.946907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:19.946907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:06:19.946907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:19.946907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:06:19.946907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:06:19.946907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:06:19.946907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:06:19.946907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:19.946907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:19.946907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:19.946907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:06:19.946907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:19.947504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:19.947504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:19.947504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:06:19.947504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:19.947504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:06:19.947504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:06:19.947504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:19.947504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:06:19.947504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:19.947504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:06:19.947504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:06:19.947504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:19.947504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:06:19.947999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:19.947999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:06:19.947999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:19.947999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:06:19.947999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:06:19.947999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:06:19.947999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:06:19.947999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:06:19.947999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:19.947999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:06:19.947999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:06:19.947999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:06:19.947999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:06:19.947999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:06:19.947999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:19.948643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:19.948643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:06:19.948643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:06:19.948643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:06:19.948643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:06:19.948643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:19.948643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:19.948643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:19.948643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:06:19.948643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:19.948643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:19.948643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:19.948643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:06:19.948643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:19.948643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:06:19.948643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:06:19.948643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:19.948643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:06:19.948643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:19.949519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:06:19.949519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:06:19.949519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:19.949519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:06:19.949519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:19.949519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:06:19.951216 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:06:19.951536 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:06:19.951623 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:06:19.951755 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:06:19.952167 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:06:19.952483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:19.953145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:06:19.953145 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:19.953145 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:06:19.953145 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:06:19.953145 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:19.953145 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:06:19.953145 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:19.953145 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:06:19.953945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:19.953945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:06:19.953945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:06:19.953945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:06:19.953945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:19.953945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:06:19.953945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:19.953945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:06:19.953945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:19.953945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:06:19.953945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:19.953945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:06:19.953945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:19.953945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:06:19.953945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:06:19.954531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:19.954531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:06:19.954531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:19.954531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:06:19.954531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:06:19.954531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:19.954531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:06:19.954531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:19.954531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:06:19.954793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:19.954793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:06:19.954793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:19.954793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:06:19.954793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:06:19.954793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:06:19.954793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:06:19.954793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:06:19.954793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:06:19.954793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:06:19.954793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:19.954793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:19.955124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:19.955124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:06:19.955124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:19.955124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:19.955124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:19.955124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:06:19.955124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:19.955124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:06:19.955124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:06:19.955124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:19.955124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:06:19.955124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:19.955124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:06:19.955124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:06:19.955511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:19.955511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:06:19.955511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:19.955511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:06:19.955696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:19.955696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:06:19.955696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:06:19.955696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:06:19.955696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:06:19.955696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:06:19.955696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:19.955696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:06:19.955696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:06:19.955696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:06:19.955696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:06:19.955696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:06:19.955696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:19.955696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:19.955696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:06:19.955696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:06:19.956130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:06:19.956130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:06:19.956130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:06:19.956130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:06:19.956130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:06:19.956130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:19.956130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:19.956130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:19.956130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:06:19.956130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:19.956130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:19.956130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:19.956130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:06:19.956130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:19.956130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:06:19.956130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:06:19.956130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:19.956130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:06:19.956639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:19.956639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:06:19.956639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:06:19.956639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:19.956639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:06:19.956639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:19.956639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:06:19.965564 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:06:19.965989 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:06:19.966169 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:06:19.966374 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:06:19.966859 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:06:19.967209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:19.967978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:06:19.967978 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:19.967978 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:06:19.967978 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:06:19.967978 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:19.967978 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:06:19.967978 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:19.967978 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:06:19.969177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:19.969177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:06:19.969177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:06:19.969177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:06:19.969177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:19.969177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:06:19.969177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:19.969177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:06:19.969177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:19.969177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:06:19.969177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:19.969177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:06:19.969177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:19.969177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:06:19.969177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:06:19.969895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:19.969895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:06:19.969895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:19.969895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:06:19.969895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:06:19.969895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:19.969895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:06:19.969895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:19.969895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:06:19.970499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:19.970499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:06:19.970499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:19.970499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:06:19.970499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:06:19.970499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:06:19.970499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:06:19.970499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:06:19.970499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:06:19.970499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:19.970499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:19.971034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:19.971034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:06:19.971034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:19.971034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:19.971034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:19.971034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:06:19.971034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:19.971034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:06:19.971034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:06:19.971034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:19.971034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:06:19.971034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:19.971034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:06:19.971034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:06:19.971641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:19.971641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:06:19.971641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:19.971641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:06:19.971641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:19.971641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:06:19.971641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:06:19.971641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:06:19.971641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:06:19.971641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:06:19.971641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:19.971641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:06:19.971641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:06:19.971641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:06:19.972431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:06:19.972431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:06:19.972431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:19.972431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:19.972431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:06:19.972431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:06:19.972431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:06:19.972431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:06:19.972431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:06:19.972431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:06:19.972431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:19.972431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:19.972431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:19.972431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:06:19.972431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:19.972431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:19.972431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:19.972431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:06:19.972431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:19.973244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:06:19.973244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:06:19.973244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:19.973244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:06:19.973244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:19.973244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:06:19.973244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:06:19.973244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:19.973244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:06:19.973244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:19.973244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:06:19.976270 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:06:19.976788 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:06:19.977079 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:06:19.977420 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:06:19.978058 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:06:19.978419 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:06:19.978742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:19.979573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:06:19.979573 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:19.979573 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:06:19.979573 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:06:19.979573 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:19.979573 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:06:19.979573 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:19.979573 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:06:19.981026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:19.981026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:06:19.981026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:19.981026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:06:19.981026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:06:19.981026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:06:19.981026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:06:19.981026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:19.981026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:19.981026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:19.981026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:06:19.981026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:19.981379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:19.981379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:19.981379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:06:19.981379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:19.981379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:06:19.981379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:06:19.981379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:19.981379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:06:19.981379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:19.981379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:06:19.981379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:06:19.981379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:19.981379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:06:19.981747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:19.981747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:06:19.982619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:19.982619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:06:19.982619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:06:19.982619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:06:19.982619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:06:19.982619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:06:19.982619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:19.982619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:06:19.982619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:06:19.982619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:06:19.982619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:06:19.982619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:06:19.982619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:19.982619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:19.982619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:06:19.982619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:06:19.983351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:06:19.983351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:06:19.983351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:19.983351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:19.983351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:19.983351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:06:19.983351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:19.983351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:19.983351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:19.983351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:06:19.983351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:19.983351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:06:19.983351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:06:19.983351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:19.983351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:06:19.983351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:19.983351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:06:19.983351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:06:19.983351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:19.984194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:06:19.984194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:19.984194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:06:24.260080 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:06:24.260704 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:06:24.260899 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:06:24.262072 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:06:24.263927 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG dbcounter [-] [105581] Writing DB stats neutron:SELECT=240 {{(pid=105581) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 14:06:24.299488 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:06:24.299488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:24.300325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:06:24.300325 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:24.300325 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:06:24.300325 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:06:24.300325 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:24.300325 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:06:24.300325 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:24.300325 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:06:24.301105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:24.301105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:06:24.301105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:24.301105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:06:24.301105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:06:24.301105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:06:24.301105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:06:24.301105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:24.301105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:24.301105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:24.301105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:06:24.301105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:24.301644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:24.301644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:24.301644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:06:24.301644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:24.301644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:06:24.301644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:06:24.301644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:24.301644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:06:24.301644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:24.301644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:06:24.301644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:06:24.301644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:24.301644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:06:24.302163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:24.302163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:06:24.302163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:24.302163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:06:24.302163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:06:24.302163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:06:24.302163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:06:24.302163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:06:24.302163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:24.302163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:06:24.302163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:06:24.302163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:06:24.302163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:06:24.302163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:06:24.302163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:24.302832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:24.302832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:06:24.302832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:06:24.302832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:06:24.302832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:06:24.302832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:24.302832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:24.302832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:24.302832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:06:24.302832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:24.302832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:24.302832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:24.302832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:06:24.302832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:24.302832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:06:24.302832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:06:24.302832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:24.302832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:06:24.302832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:24.303674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:06:24.303674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:06:24.303674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:24.303674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:06:24.303674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:24.303674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:06:24.943209 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:06:24.943758 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:06:24.943934 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:06:24.944898 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:06:24.946044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:24.946346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:06:24.946346 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:24.946346 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:06:24.946346 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:06:24.946346 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:24.946346 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:06:24.946346 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:24.946346 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:06:24.947921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:24.947921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:06:24.947921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:24.947921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:06:24.947921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:06:24.947921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:06:24.947921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:06:24.947921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:24.947921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:24.947921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:24.947921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:06:24.947921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:24.948437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:24.948437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:24.948437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:06:24.948437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:24.948437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:06:24.948437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:06:24.948437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:24.948437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:06:24.948437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:24.948437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:06:24.948437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:06:24.948437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:24.948437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:06:24.948971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:24.948971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:06:24.950281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:24.950281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:06:24.950281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:06:24.950281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:06:24.950281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:06:24.950281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:06:24.950281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:24.950281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:06:24.950281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:06:24.950281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:06:24.950281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:06:24.950281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:06:24.950281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:24.950281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:24.950281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:06:24.950281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:06:24.951021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:06:24.951021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:06:24.951021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:24.951021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:24.951021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:24.951021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:06:24.951021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:24.951021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:24.951021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:24.951021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:06:24.951021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:24.951021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:06:24.951021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:06:24.951021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:24.951021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:06:24.951021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:24.951021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:06:24.951021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:06:24.951021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:24.951901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:06:24.951901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:24.951901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:06:24.951901 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:06:24.951901 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:06:24.952117 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:06:24.952117 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:06:24.953117 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:06:24.953117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:24.953318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:06:24.953318 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:24.953318 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:06:24.953318 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:06:24.953318 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:24.953318 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:06:24.953318 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:24.953318 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:06:24.954112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:24.954112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:06:24.954112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:06:24.954112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:06:24.954112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:24.954112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:06:24.954112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:24.954112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:06:24.954112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:24.954112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:06:24.954112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:24.954112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:06:24.954112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:24.954112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:06:24.954112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:06:24.954515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:24.954515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:06:24.954515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:24.954515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:06:24.954515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:06:24.954515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:24.954515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:06:24.954515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:24.954515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:06:24.954981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:24.954981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:06:24.954981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:24.954981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:06:24.954981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:06:24.954981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:06:24.954981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:06:24.954981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:06:24.954981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:06:24.954981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:06:24.954981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:24.954981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:24.955289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:24.955289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:06:24.955289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:24.955289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:24.955289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:24.955289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:06:24.955289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:24.955289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:06:24.955289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:06:24.955289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:24.955289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:06:24.955289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:24.955289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:06:24.955289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:06:24.955662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:24.955662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:06:24.955662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:24.955662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:06:24.956063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:24.956063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:06:24.956063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:06:24.956063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:06:24.956063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:06:24.956063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:06:24.956063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:24.956063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:06:24.956063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:06:24.956063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:06:24.956063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:06:24.956063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:06:24.956063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:24.956063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:24.956063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:06:24.956063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:06:24.956466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:06:24.956466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:06:24.956466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:06:24.956466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:06:24.956466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:06:24.956466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:24.956466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:24.956466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:24.956466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:06:24.956466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:24.956466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:24.956466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:24.956466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:06:24.956466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:24.956466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:06:24.956466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:06:24.956466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:24.956466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:06:24.956910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:24.956910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:06:24.956910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:06:24.956910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:24.956910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:06:24.956910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:24.956910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:06:24.965920 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:06:24.966158 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:06:24.966280 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:06:24.966422 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:06:24.966866 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:06:24.967367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:24.968376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:06:24.968376 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:24.968376 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:06:24.968376 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:06:24.968376 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:24.968376 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:06:24.968376 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:24.968376 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:06:24.969645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:24.969645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:06:24.969645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:06:24.969645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:06:24.969645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:24.969645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:06:24.969645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:24.969645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:06:24.969645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:24.969645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:06:24.969645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:24.969645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:06:24.969645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:24.969645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:06:24.969645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:06:24.970233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:24.970233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:06:24.970233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:24.970233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:06:24.970233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:06:24.970233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:24.970233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:06:24.970233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:24.970233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:06:24.971037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:24.971037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:06:24.971037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:24.971037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:06:24.971037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:06:24.971037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:06:24.971037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:06:24.971037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:06:24.971037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:06:24.971037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:24.971037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:24.971540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:24.971540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:06:24.971540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:24.971540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:24.971540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:24.971540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:06:24.971540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:24.971540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:06:24.971540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:06:24.971540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:24.971540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:06:24.971540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:24.971540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:06:24.971540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:06:24.972117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:24.972117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:06:24.972117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:24.972117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:06:24.973937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:24.973937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:06:24.973937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:06:24.973937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:06:24.973937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:06:24.973937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:06:24.973937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:24.973937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:06:24.973937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:06:24.973937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:06:24.973937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:06:24.973937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:06:24.973937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:24.973937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:24.973937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:06:24.973937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:06:24.973937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:06:24.974577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:06:24.974577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:06:24.974577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:06:24.974577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:24.974577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:24.974577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:24.974577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:06:24.974577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:24.974577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:24.974577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:24.974577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:06:24.974577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:24.974577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:06:24.974577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:06:24.974577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:24.974577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:06:24.974577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:24.974577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:06:24.974577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:06:24.975050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:24.975050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:06:24.975050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:24.975050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:06:24.976720 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:06:24.977124 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:06:24.977318 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:06:24.977542 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:06:24.977839 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:06:24.978152 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:06:24.978379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:24.978995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:06:24.978995 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:24.978995 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:06:24.978995 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:06:24.978995 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:24.978995 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:06:24.978995 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:24.978995 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:06:24.979958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:24.979958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:06:24.979958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:24.979958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:06:24.979958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:06:24.979958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:06:24.979958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:06:24.979958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:24.979958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:24.979958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:24.979958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:06:24.979958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:24.980280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:24.980280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:24.980280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:06:24.980280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:24.980280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:06:24.980280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:06:24.980280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:24.980280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:06:24.980280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:24.980280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:06:24.980280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:06:24.980280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:24.980280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:06:24.980623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:24.980623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:06:24.981003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:24.981003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:06:24.981003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:06:24.981003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:06:24.981003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:06:24.981003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:06:24.981003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:24.981003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:06:24.981003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:06:24.981003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:06:24.981003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:06:24.981003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:06:24.981003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:24.981003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:24.981003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:06:24.981003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:06:24.981435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:06:24.981435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:06:24.981435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:24.981435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:24.981435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:24.981435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:06:24.981435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:24.981435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:24.981435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:24.981435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:06:24.981435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:24.981435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:06:24.981435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:06:24.981435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:24.981435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:06:24.981435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:24.981435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:06:24.981435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:06:24.981435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:24.981907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:06:24.981907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:24.981907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:06:29.260622 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:06:29.262036 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:06:29.262227 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:06:29.263420 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:06:29.295628 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:06:29.296055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:29.296858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:06:29.296858 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:29.296858 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:06:29.296858 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:06:29.296858 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:29.296858 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:06:29.296858 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:29.296858 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:06:29.297639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:29.297639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:06:29.297639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:29.297639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:06:29.297639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:06:29.297639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:06:29.297639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:06:29.297639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:29.297639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:29.297639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:29.297639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:06:29.297639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:29.297998 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:29.297998 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:29.297998 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:06:29.297998 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:29.297998 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:06:29.297998 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:06:29.297998 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:29.297998 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:06:29.297998 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:29.297998 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:06:29.297998 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:06:29.297998 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:29.297998 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:06:29.298366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:29.298366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:06:29.298366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:29.298366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:06:29.298366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:06:29.298366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:06:29.298366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:06:29.298366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:06:29.298366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:29.298366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:06:29.298366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:06:29.298366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:06:29.298366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:06:29.298366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:06:29.298366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:29.298801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:29.298801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:06:29.298801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:06:29.298801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:06:29.298801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:06:29.298801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:29.298801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:29.298801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:29.298801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:06:29.298801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:29.298801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:29.298801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:29.298801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:06:29.298801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:29.298801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:06:29.298801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:06:29.298801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:29.298801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:06:29.298801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:29.299336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:06:29.299336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:06:29.299336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:29.299336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:06:29.299336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:29.299336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:06:29.943893 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:06:29.944438 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:06:29.944438 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:06:29.945268 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:06:29.945675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:29.946432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:06:29.946432 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:29.946432 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:06:29.946432 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:06:29.946432 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:29.946432 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:06:29.946432 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:29.946432 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:06:29.947156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:29.947156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:06:29.947156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:29.947156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:06:29.947156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:06:29.947156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:06:29.947156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:06:29.947156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:29.947156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:29.947156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:29.947156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:06:29.947156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:29.947734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:29.947734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:29.947734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:06:29.947734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:29.947734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:06:29.947734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:06:29.947734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:29.947734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:06:29.947734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:29.947734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:06:29.947734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:06:29.947734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:29.947734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:06:29.948271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:29.948271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:06:29.948271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:29.948271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:06:29.948271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:06:29.948271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:06:29.948271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:06:29.948271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:06:29.948271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:29.948271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:06:29.948271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:06:29.948271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:06:29.948271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:06:29.948271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:06:29.948271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:29.948949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:29.948949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:06:29.948949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:06:29.948949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:06:29.948949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:06:29.948949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:29.948949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:29.948949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:29.948949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:06:29.948949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:29.948949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:29.948949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:29.948949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:06:29.948949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:29.948949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:06:29.948949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:06:29.948949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:29.948949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:06:29.948949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:29.949796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:06:29.949796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:06:29.949796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:29.949796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:06:29.949796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:29.949796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:06:29.951995 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:06:29.952192 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:06:29.952277 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:06:29.952348 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:06:29.952774 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:06:29.953087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:29.953716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:06:29.953716 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:29.953716 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:06:29.953716 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:06:29.953716 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:29.953716 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:06:29.953716 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:29.953716 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:06:29.954255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:29.954255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:06:29.954255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:06:29.954255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:06:29.954255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:29.954255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:06:29.954255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:29.954255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:06:29.954255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:29.954255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:06:29.954255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:29.954255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:06:29.954255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:29.954255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:06:29.954255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:06:29.954712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:29.954712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:06:29.954712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:29.954712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:06:29.954712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:06:29.954712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:29.954712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:06:29.954712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:29.954712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:06:29.954981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:29.954981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:06:29.954981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:29.954981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:06:29.954981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:06:29.954981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:06:29.954981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:06:29.954981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:06:29.954981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:06:29.954981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:06:29.954981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:29.954981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:29.955324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:29.955324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:06:29.955324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:29.955324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:29.955324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:29.955324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:06:29.955324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:29.955324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:06:29.955324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:06:29.955324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:29.955324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:06:29.955324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:29.955324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:06:29.955324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:06:29.955718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:29.955718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:06:29.955718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:29.955718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:06:29.955718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:29.955718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:06:29.955718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:06:29.955718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:06:29.955718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:06:29.955718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:06:29.955718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:29.955718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:06:29.955718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:06:29.955718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:06:29.956120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:06:29.956120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:06:29.956120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:29.956120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:29.956120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:06:29.956120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:06:29.956120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:06:29.956120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:06:29.956120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:06:29.956120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:06:29.956120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:06:29.956120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:29.956120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:29.956120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:29.956120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:06:29.956120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:29.956120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:29.956120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:29.956120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:06:29.956663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:29.956663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:06:29.956663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:06:29.956663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:29.956663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:06:29.956663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:29.956663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:06:29.956663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:06:29.956663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:29.956663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:06:29.956663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:29.956663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:06:29.966368 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:06:29.966643 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:06:29.966723 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:06:29.966817 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:06:29.967262 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:06:29.967649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:29.968358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:06:29.968358 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:29.968358 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:06:29.968358 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:06:29.968358 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:29.968358 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:06:29.968358 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:29.968358 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:06:29.968907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:29.968907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:06:29.968907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:06:29.968907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:06:29.968907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:29.968907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:06:29.968907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:29.968907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:06:29.968907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:29.968907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:06:29.968907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:29.968907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:06:29.968907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:29.968907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:06:29.968907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:06:29.969221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:29.969221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:06:29.969221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:29.969221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:06:29.969221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:06:29.969221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:29.969221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:06:29.969221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:29.969221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:06:29.969544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:29.969544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:06:29.969544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:29.969544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:06:29.969544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:06:29.969544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:06:29.969544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:06:29.969544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:06:29.969544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:06:29.969544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:29.969544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:29.969776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:29.969776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:06:29.969776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:29.969776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:29.969776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:29.969776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:06:29.969776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:29.969776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:06:29.969776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:06:29.969776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:29.969776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:06:29.969776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:29.969776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:06:29.969776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:06:29.970011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:29.970011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:06:29.970011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:29.970011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:06:29.970275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:29.970275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:06:29.970275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:06:29.970275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:06:29.970275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:06:29.970275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:06:29.970275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:29.970275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:06:29.970275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:06:29.970275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:06:29.970275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:06:29.970275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:06:29.970275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:29.970275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:29.970275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:06:29.970275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:06:29.970275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:06:29.970612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:06:29.970612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:06:29.970612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:06:29.970612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:29.970612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:29.970612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:29.970612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:06:29.970612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:29.970612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:29.970612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:29.970612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:06:29.970612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:29.970612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:06:29.970612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:06:29.970612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:29.970612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:06:29.970612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:29.970612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:06:29.970612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:06:29.970961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:29.970961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:06:29.970961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:29.970961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:06:29.977248 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:06:29.977645 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:06:29.977808 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:06:29.978027 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:06:29.978509 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:06:29.978852 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:06:29.979020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:29.979762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:06:29.979762 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:29.979762 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:06:29.979762 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:06:29.979762 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:29.979762 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:06:29.979762 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:29.979762 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:06:29.981172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:29.981172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:06:29.981172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:29.981172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:06:29.981172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:06:29.981172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:06:29.981172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:06:29.981172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:29.981172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:29.981172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:29.981172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:06:29.981172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:29.981794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:29.981794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:29.981794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:06:29.981794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:29.981794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:06:29.981794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:06:29.981794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:29.981794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:06:29.981794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:29.981794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:06:29.981794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:06:29.981794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:29.981794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:06:29.983046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:29.983046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:06:29.983046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:29.983046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:06:29.983046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:06:29.983046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:06:29.983046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:06:29.983046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:06:29.983046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:29.983046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:06:29.983046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:06:29.983046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:06:29.983046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:06:29.983046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:06:29.983046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:29.983753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:29.983753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:06:29.983753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:06:29.983753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:06:29.983753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:06:29.983753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:29.983753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:29.983753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:29.983753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:06:29.983753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:29.983753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:29.983753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:29.983753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:06:29.983753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:29.983753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:06:29.983753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:06:29.983753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:29.983753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:06:29.983753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:29.984649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:06:29.984649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:06:29.984649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:29.984649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:06:29.984649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:29.984649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:06:34.262269 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:06:34.263095 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:06:34.263179 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:06:34.264305 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:06:34.295603 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:06:34.296032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:34.296926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:06:34.296926 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:34.296926 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:06:34.296926 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:06:34.296926 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:34.296926 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:06:34.296926 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:34.296926 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:06:34.297865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:34.297865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:06:34.297865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:34.297865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:06:34.297865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:06:34.297865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:06:34.297865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:06:34.297865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:34.297865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:34.297865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:34.297865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:06:34.297865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:34.298460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:34.298460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:34.298460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:06:34.298460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:34.298460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:06:34.298460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:06:34.298460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:34.298460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:06:34.298460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:34.298460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:06:34.298460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:06:34.298460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:34.298460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:06:34.299057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:34.299057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:06:34.299057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:34.299057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:06:34.299057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:06:34.299057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:06:34.299057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:06:34.299057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:06:34.299057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:34.299057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:06:34.299057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:06:34.299057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:06:34.299057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:06:34.299057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:06:34.299057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:34.299716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:34.299716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:06:34.299716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:06:34.299716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:06:34.299716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:06:34.299716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:34.299716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:34.299716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:34.299716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:06:34.299716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:34.299716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:34.299716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:34.299716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:06:34.299716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:34.299716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:06:34.299716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:06:34.299716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:34.299716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:06:34.299716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:34.300563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:06:34.300563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:06:34.300563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:34.300563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:06:34.300563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:34.300563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:06:34.944570 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:06:34.945205 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:06:34.945430 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:06:34.946498 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:06:34.946919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:34.949307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:06:34.949307 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:34.949307 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:06:34.949307 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:06:34.949307 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:34.949307 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:06:34.949307 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:34.949307 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:06:34.950909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:34.950909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:06:34.950909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:34.950909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:06:34.950909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:06:34.950909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:06:34.950909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:06:34.950909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:34.950909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:34.950909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:34.950909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:06:34.950909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:34.951519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:34.951519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:34.951519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:06:34.951519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:34.951519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:06:34.951519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:06:34.951519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:34.951519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:06:34.951519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:34.951519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:06:34.951519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:06:34.951519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:34.951519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:06:34.952212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:34.952212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:06:34.952314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:34.952314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:06:34.952314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:06:34.952314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:06:34.952314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:06:34.952314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:06:34.952314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:34.952314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:06:34.952314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:06:34.952314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:06:34.952314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:06:34.952314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:06:34.952314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:34.952314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:34.952314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:06:34.952314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:06:34.952980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:06:34.952980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:06:34.952980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:34.952980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:34.952980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:34.952980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:06:34.952980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:34.952980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:34.952980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:34.952980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:06:34.952980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:34.952980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:06:34.952980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:06:34.952980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:34.952980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:06:34.952980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:34.952980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:06:34.952980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:06:34.952980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:34.953776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:06:34.953776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:34.953776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:06:34.953776 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:06:34.953776 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:06:34.953776 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:06:34.953776 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:06:34.953776 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:06:34.954185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:34.954374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:06:34.954374 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:34.954374 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:06:34.954374 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:06:34.954374 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:34.954374 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:06:34.954374 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:34.954374 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:06:34.955145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:34.955145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:06:34.955145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:06:34.955145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:06:34.955145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:34.955145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:06:34.955145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:34.955145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:06:34.955145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:34.955145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:06:34.955145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:34.955145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:06:34.955145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:34.955145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:06:34.955145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:06:34.955866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:34.955866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:06:34.955866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:34.955866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:06:34.955866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:06:34.955866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:34.955866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:06:34.955866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:34.955866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:06:34.957010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:34.957010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:06:34.957010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:34.957010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:06:34.957010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:06:34.957010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:06:34.957010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:06:34.957010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:06:34.957010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:06:34.957010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:06:34.957010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:34.957010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:34.957612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:34.957612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:06:34.957612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:34.957612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:34.957612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:34.957612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:06:34.957612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:34.957612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:06:34.957612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:06:34.957612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:34.957612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:06:34.957612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:34.957612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:06:34.957612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:06:34.958181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:34.958181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:06:34.958181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:34.958181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:06:34.958181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:34.958181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:06:34.958181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:06:34.958181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:06:34.958181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:06:34.958181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:06:34.958181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:34.958181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:06:34.958181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:06:34.958181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:06:34.960189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:06:34.960189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:06:34.960189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:34.960189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:34.960189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:06:34.960189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:06:34.960189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:06:34.960189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:06:34.960189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:06:34.960189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:06:34.960189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:06:34.960189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:34.960189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:34.960189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:34.960189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:06:34.960189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:34.960189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:34.960189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:34.960189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:06:34.961137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:34.961137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:06:34.961137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:06:34.961137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:34.961137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:06:34.961137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:34.961137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:06:34.961137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:06:34.961137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:34.961137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:06:34.961137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:34.961137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:06:34.966953 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:06:34.967218 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:06:34.967357 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:06:34.967357 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:06:34.967927 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:06:34.968362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:34.969111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:06:34.969111 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:34.969111 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:06:34.969111 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:06:34.969111 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:34.969111 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:06:34.969111 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:34.969111 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:06:34.970221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:34.970221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:06:34.970221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:06:34.970221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:06:34.970221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:34.970221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:06:34.970221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:34.970221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:06:34.970221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:34.970221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:06:34.970221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:34.970221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:06:34.970221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:34.970221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:06:34.970221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:06:34.970970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:34.970970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:06:34.970970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:34.970970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:06:34.970970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:06:34.970970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:34.970970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:06:34.970970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:34.970970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:06:34.971602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:34.971602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:06:34.971602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:34.971602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:06:34.971602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:06:34.971602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:06:34.971602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:06:34.971602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:06:34.971602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:06:34.971602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:34.971602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:34.972181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:34.972181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:06:34.972181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:34.972181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:34.972181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:34.972181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:06:34.972181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:34.972181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:06:34.972181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:06:34.972181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:34.972181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:06:34.972181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:34.972181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:06:34.972181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:06:34.972799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:34.972799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:06:34.972799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:34.972799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:06:34.973158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:34.973158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:06:34.973158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:06:34.973158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:06:34.973158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:06:34.973158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:06:34.973158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:34.973158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:06:34.973158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:06:34.973158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:06:34.973158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:06:34.973158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:06:34.973158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:34.973158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:34.973158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:06:34.973158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:06:34.973158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:06:34.973922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:06:34.973922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:06:34.973922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:06:34.973922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:34.973922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:34.973922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:34.973922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:06:34.973922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:34.973922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:34.973922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:34.973922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:06:34.973922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:34.973922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:06:34.973922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:06:34.973922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:34.973922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:06:34.973922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:34.973922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:06:34.973922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:06:34.974855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:34.974855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:06:34.974855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:34.974855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:06:34.977713 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:06:34.978092 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:06:34.978387 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:06:34.978716 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:06:34.979229 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:06:34.979595 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:06:34.979781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:34.980536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:06:34.980536 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:34.980536 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:06:34.980536 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:06:34.980536 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:34.980536 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:06:34.980536 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:34.980536 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:06:34.981471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:34.981471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:06:34.981471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:34.981471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:06:34.981471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:06:34.981471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:06:34.981471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:06:34.981471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:34.981471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:34.981471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:34.981471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:06:34.981471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:34.981832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:34.981832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:34.981832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:06:34.981832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:34.981832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:06:34.981832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:06:34.981832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:34.981832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:06:34.981832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:34.981832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:06:34.981832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:06:34.981832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:34.981832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:06:34.982158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:34.982158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:06:34.982400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:34.982400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:06:34.982400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:06:34.982400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:06:34.982400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:06:34.982400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:06:34.982400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:34.982400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:06:34.982400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:06:34.982400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:06:34.982400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:06:34.982400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:06:34.982400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:34.982400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:34.982400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:06:34.982400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:06:34.982868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:06:34.982868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:06:34.982868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:34.982868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:34.982868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:34.982868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:06:34.982868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:34.982868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:34.982868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:34.982868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:06:34.982868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:34.982868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:06:34.982868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:06:34.982868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:34.982868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:06:34.982868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:34.982868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:06:34.982868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:06:34.982868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:34.983362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:06:34.983362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:34.983362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:06:39.262924 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:06:39.263780 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:06:39.263780 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:06:39.264732 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:06:39.294145 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:06:39.294500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:39.295285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:06:39.295285 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:39.295285 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:06:39.295285 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:06:39.295285 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:39.295285 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:06:39.295285 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:39.295285 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:06:39.296028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:39.296028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:06:39.296028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:39.296028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:06:39.296028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:06:39.296028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:06:39.296028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:06:39.296028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:39.296028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:39.296028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:39.296028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:06:39.296028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:39.296618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:39.296618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:39.296618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:06:39.296618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:39.296618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:06:39.296618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:06:39.296618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:39.296618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:06:39.296618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:39.296618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:06:39.296618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:06:39.296618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:39.296618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:06:39.297201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:39.297201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:06:39.297201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:39.297201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:06:39.297201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:06:39.297201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:06:39.297201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:06:39.297201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:06:39.297201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:39.297201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:06:39.297201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:06:39.297201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:06:39.297201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:06:39.297201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:06:39.297201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:39.297896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:39.297896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:06:39.297896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:06:39.297896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:06:39.297896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:06:39.297896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:39.297896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:39.297896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:39.297896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:06:39.297896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:39.297896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:39.297896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:39.297896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:06:39.297896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:39.297896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:06:39.297896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:06:39.297896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:39.297896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:06:39.297896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:39.298700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:06:39.298700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:06:39.298700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:39.298700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:06:39.298700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:39.298700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:06:39.945290 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:06:39.946016 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:06:39.946211 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:06:39.947116 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:06:39.947439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:39.948366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:06:39.948366 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:39.948366 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:06:39.948366 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:06:39.948366 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:39.948366 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:06:39.948366 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:39.948366 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:06:39.949844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:39.949844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:06:39.949844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:39.949844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:06:39.949844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:06:39.949844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:06:39.949844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:06:39.949844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:39.949844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:39.949844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:39.949844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:06:39.949844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:39.950963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:39.950963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:39.950963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:06:39.950963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:39.950963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:06:39.950963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:06:39.950963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:39.950963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:06:39.950963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:39.950963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:06:39.950963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:06:39.950963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:39.950963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:06:39.952814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:39.952814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:06:39.952814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:39.952814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:06:39.952814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:06:39.952814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:06:39.952814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:06:39.952814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:06:39.952814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:39.952814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:06:39.952814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:06:39.952814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:06:39.952814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:06:39.952814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:06:39.952814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:39.953215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:39.953215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:06:39.953215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:06:39.953215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:06:39.953215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:06:39.953215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:39.953215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:39.953215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:39.953215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:06:39.953215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:39.953215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:39.953215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:39.953215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:06:39.953215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:39.953215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:06:39.953215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:06:39.953215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:39.953215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:06:39.953215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:39.953684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:06:39.953684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:06:39.953684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:39.953684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:06:39.953684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:39.953684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:06:39.953684 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:06:39.953684 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:06:39.953684 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:06:39.953684 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:06:39.953955 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:06:39.953955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:39.954354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:06:39.954354 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:39.954354 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:06:39.954354 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:06:39.954354 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:39.954354 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:06:39.954354 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:39.954354 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:06:39.955172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:39.955172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:06:39.955172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:06:39.955172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:06:39.955172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:39.955172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:06:39.955172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:39.955172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:06:39.955172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:39.955172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:06:39.955172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:39.955172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:06:39.955172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:39.955172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:06:39.955172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:06:39.955991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:39.955991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:06:39.955991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:39.955991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:06:39.955991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:06:39.955991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:39.955991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:06:39.955991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:39.955991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:06:39.955991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:39.955991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:06:39.955991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:39.955991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:06:39.955991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:06:39.956795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:06:39.956795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:06:39.956795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:06:39.956795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:06:39.956795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:06:39.956795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:39.956795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:39.956795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:39.956795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:06:39.956795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:39.956795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:39.956795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:39.956795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:06:39.957541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:39.957541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:06:39.957541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:06:39.957541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:39.957541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:06:39.957541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:39.957541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:06:39.957541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:06:39.957541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:39.957541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:06:39.957541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:39.957541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:06:39.958172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:39.958172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:06:39.958172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:06:39.958172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:06:39.958172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:06:39.958172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:06:39.958172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:39.958172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:06:39.958172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:06:39.958172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:06:39.958172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:06:39.958172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:06:39.958172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:39.958172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:39.958172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:06:39.958172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:06:39.958939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:06:39.958939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:06:39.958939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:06:39.958939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:06:39.958939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:06:39.958939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:39.958939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:39.958939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:39.958939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:06:39.958939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:39.958939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:39.958939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:39.958939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:06:39.958939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:39.958939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:06:39.958939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:06:39.958939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:39.958939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:06:39.959374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:39.959374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:06:39.959374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:06:39.959374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:39.959374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:06:39.959374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:39.959374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:06:39.967267 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:06:39.967575 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:06:39.967686 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:06:39.967812 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:06:39.968249 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:06:39.968660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:39.969421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:06:39.969421 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:39.969421 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:06:39.969421 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:06:39.969421 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:39.969421 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:06:39.969421 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:39.969421 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:06:39.970620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:39.970620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:06:39.970620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:06:39.970620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:06:39.970620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:39.970620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:06:39.970620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:39.970620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:06:39.970620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:39.970620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:06:39.970620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:39.970620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:06:39.970620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:39.970620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:06:39.970620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:06:39.971385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:39.971385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:06:39.971385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:39.971385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:06:39.971385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:06:39.971385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:39.971385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:06:39.971385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:39.971385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:06:39.972218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:39.972218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:06:39.972218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:39.972218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:06:39.972218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:06:39.972218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:06:39.972218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:06:39.972218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:06:39.972218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:06:39.972218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:39.972218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:39.973190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:39.973190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:06:39.973190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:39.973190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:39.973190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:39.973190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:06:39.973190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:39.973190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:06:39.973190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:06:39.973190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:39.973190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:06:39.973190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:39.973190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:06:39.973190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:06:39.974382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:39.974382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:06:39.974382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:39.974382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:06:39.974382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:39.974382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:06:39.974382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:06:39.974382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:06:39.974382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:06:39.974382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:06:39.974382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:39.974382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:06:39.974382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:06:39.974382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:06:39.975345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:06:39.975345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:06:39.975345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:39.975345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:39.975345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:06:39.975345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:06:39.975345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:06:39.975345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:06:39.975345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:06:39.975345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:06:39.975345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:39.975345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:39.975345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:39.975345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:06:39.975345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:39.975345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:39.975345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:39.975345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:06:39.975345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:39.976091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:06:39.976091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:06:39.976091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:39.976091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:06:39.976091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:39.976091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:06:39.976091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:06:39.976091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:39.976091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:06:39.976091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:39.976091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:06:39.978188 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:06:39.978607 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:06:39.978780 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:06:39.978961 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:06:39.979351 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:06:39.979756 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:06:39.979936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:39.980665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:06:39.980665 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:39.980665 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:06:39.980665 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:06:39.980665 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:39.980665 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:06:39.980665 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:39.980665 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:06:39.981416 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:39.981416 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:06:39.981416 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:39.981416 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:06:39.981416 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:06:39.981416 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:06:39.981416 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:06:39.981416 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:39.981416 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:39.981416 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:39.981416 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:06:39.981416 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:39.981871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:39.981871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:39.981871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:06:39.981871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:39.981871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:06:39.981871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:06:39.981871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:39.981871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:06:39.981871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:39.981871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:06:39.981871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:06:39.981871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:39.981871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:06:39.982276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:39.982276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:06:39.982329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:39.982329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:06:39.982329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:06:39.982329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:06:39.982329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:06:39.982329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:06:39.982329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:39.982329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:06:39.982329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:06:39.982329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:06:39.982329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:06:39.982329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:06:39.982329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:39.982329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:39.982329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:06:39.982329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:06:39.982612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:06:39.982612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:06:39.982612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:39.982612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:39.982612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:39.982612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:06:39.982612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:39.982612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:39.982612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:39.982612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:06:39.982612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:39.982612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:06:39.982612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:06:39.982612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:39.982612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:06:39.982612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:39.982612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:06:39.982612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:06:39.982612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:39.983017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:06:39.983017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:39.983017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:06:44.263620 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:06:44.264364 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:06:44.264610 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:06:44.265885 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:06:44.297835 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:06:44.298233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:44.299137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:06:44.299137 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:44.299137 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:06:44.299137 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:06:44.299137 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:44.299137 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:06:44.299137 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:44.299137 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:06:44.300134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:44.300134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:06:44.300134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:44.300134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:06:44.300134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:06:44.300134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:06:44.300134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:06:44.300134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:44.300134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:44.300134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:44.300134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:06:44.300134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:44.301120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:44.301120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:44.301120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:06:44.301120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:44.301120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:06:44.301120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:06:44.301120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:44.301120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:06:44.301120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:44.301120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:06:44.301120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:06:44.301120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:44.301120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:06:44.301647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:44.301647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:06:44.301647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:44.301647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:06:44.301647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:06:44.301647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:06:44.301647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:06:44.301647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:06:44.301647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:44.301647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:06:44.301647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:06:44.301647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:06:44.301647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:06:44.301647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:06:44.301647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:44.302283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:44.302283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:06:44.302283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:06:44.302283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:06:44.302283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:06:44.302283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:44.302283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:44.302283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:44.302283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:06:44.302283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:44.302283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:44.302283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:44.302283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:06:44.302283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:44.302283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:06:44.302283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:06:44.302283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:44.302283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:06:44.302283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:44.303152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:06:44.303152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:06:44.303152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:44.303152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:06:44.303152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:44.303152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:06:44.946081 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:06:44.946838 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:06:44.947017 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:06:44.948035 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:06:44.948390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:44.949367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:06:44.949367 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:44.949367 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:06:44.949367 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:06:44.949367 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:44.949367 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:06:44.949367 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:44.949367 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:06:44.951010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:44.951010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:06:44.951010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:44.951010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:06:44.951010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:06:44.951010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:06:44.951010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:06:44.951010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:44.951010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:44.951010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:44.951010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:06:44.951010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:44.952167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:44.952167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:44.952167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:06:44.952167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:44.952167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:06:44.952167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:06:44.952167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:44.952167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:06:44.952167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:44.952167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:06:44.952167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:06:44.952167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:44.952167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:06:44.952549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:44.952549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:06:44.952620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:44.952620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:06:44.952620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:06:44.952620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:06:44.952620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:06:44.952620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:06:44.952620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:44.952620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:06:44.952620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:06:44.952620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:06:44.952620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:06:44.952620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:06:44.952620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:44.952620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:44.952620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:06:44.952620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:06:44.953046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:06:44.953046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:06:44.953046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:44.953046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:44.953046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:44.953046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:06:44.953046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:44.953046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:44.953046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:44.953046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:06:44.953046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:44.953046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:06:44.953046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:06:44.953046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:44.953046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:06:44.953046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:44.953046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:06:44.953046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:06:44.953046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:44.953540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:06:44.953540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:44.953540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:06:44.953540 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:06:44.954126 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:06:44.954126 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:06:44.954257 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:06:44.954641 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:06:44.954841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:44.955530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:06:44.955530 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:44.955530 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:06:44.955530 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:06:44.955530 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:44.955530 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:06:44.955530 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:44.955530 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:06:44.956713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:44.956713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:06:44.956713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:06:44.956713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:06:44.956713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:44.956713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:06:44.956713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:44.956713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:06:44.956713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:44.956713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:06:44.956713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:44.956713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:06:44.956713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:44.956713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:06:44.956713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:06:44.957576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:44.957576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:06:44.957576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:44.957576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:06:44.957576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:06:44.957576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:44.957576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:06:44.957576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:44.957576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:06:44.958065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:44.958065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:06:44.958065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:44.958065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:06:44.958065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:06:44.958065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:06:44.958065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:06:44.958065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:06:44.958065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:06:44.958065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:06:44.958065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:44.958065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:44.958758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:44.958758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:06:44.958758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:44.958758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:44.958758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:44.958758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:06:44.958758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:44.958758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:06:44.958758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:06:44.958758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:44.958758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:06:44.958758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:44.958758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:06:44.958758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:06:44.959345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:44.959345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:06:44.959345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:44.959345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:06:44.959467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:44.959467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:06:44.959467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:06:44.959467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:06:44.959467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:06:44.959467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:06:44.959467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:44.959467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:06:44.959467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:06:44.959467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:06:44.959467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:06:44.959467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:06:44.959467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:44.959467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:44.959467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:06:44.959467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:06:44.959913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:06:44.959913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:06:44.959913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:06:44.959913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:06:44.959913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:06:44.959913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:44.959913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:44.959913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:44.959913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:06:44.959913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:44.959913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:44.959913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:44.959913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:06:44.959913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:44.959913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:06:44.959913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:06:44.959913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:44.959913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:06:44.960341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:44.960341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:06:44.960341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:06:44.960341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:44.960341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:06:44.960341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:44.960341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:06:44.967754 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:06:44.968029 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:06:44.968166 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:06:44.968356 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:06:44.968875 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:06:44.969230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:44.970128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:06:44.970128 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:44.970128 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:06:44.970128 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:06:44.970128 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:44.970128 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:06:44.970128 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:44.970128 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:06:44.971235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:44.971235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:06:44.971235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:06:44.971235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:06:44.971235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:44.971235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:06:44.971235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:44.971235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:06:44.971235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:44.971235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:06:44.971235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:44.971235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:06:44.971235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:44.971235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:06:44.971235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:06:44.972479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:44.972479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:06:44.972479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:44.972479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:06:44.972479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:06:44.972479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:44.972479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:06:44.972479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:44.972479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:06:44.972824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:44.972824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:06:44.972824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:44.972824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:06:44.972824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:06:44.972824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:06:44.972824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:06:44.972824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:06:44.972824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:06:44.972824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:44.972824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:44.973266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:44.973266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:06:44.973266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:44.973266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:44.973266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:44.973266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:06:44.973266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:44.973266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:06:44.973266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:06:44.973266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:44.973266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:06:44.973266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:44.973266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:06:44.973266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:06:44.973809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:44.973809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:06:44.973809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:44.973809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:06:44.974603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:44.974603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:06:44.974603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:06:44.974603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:06:44.974603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:06:44.974603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:06:44.974603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:44.974603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:06:44.974603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:06:44.974603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:06:44.974603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:06:44.974603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:06:44.974603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:44.974603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:44.974603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:06:44.974603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:06:44.974603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:06:44.975989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:06:44.975989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:06:44.975989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:06:44.975989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:44.975989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:44.975989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:44.975989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:06:44.975989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:44.975989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:44.975989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:44.975989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:06:44.975989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:44.975989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:06:44.975989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:06:44.975989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:44.975989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:06:44.975989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:44.975989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:06:44.975989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:06:44.977576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:44.977576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:06:44.977576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:44.977576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:06:44.978702 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:06:44.979051 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:06:44.979222 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:06:44.979368 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:06:44.979787 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:06:44.980125 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:06:44.980304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:44.981014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:06:44.981014 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:44.981014 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:06:44.981014 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:06:44.981014 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:44.981014 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:06:44.981014 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:44.981014 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:06:44.981827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:44.981827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:06:44.981827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:44.981827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:06:44.981827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:06:44.981827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:06:44.981827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:06:44.981827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:44.981827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:44.981827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:44.981827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:06:44.981827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:44.982077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:44.982077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:44.982077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:06:44.982077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:44.982077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:06:44.982077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:06:44.982077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:44.982077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:06:44.982077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:44.982077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:06:44.982077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:06:44.982077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:44.982077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:06:44.982301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:44.982301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:06:44.982779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:44.982779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:06:44.982779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:06:44.982779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:06:44.982779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:06:44.982779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:06:44.982779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:44.982779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:06:44.982779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:06:44.982779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:06:44.982779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:06:44.982779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:06:44.982779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:44.982779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:44.982779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:06:44.982779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:06:44.983135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:06:44.983135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:06:44.983135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:44.983135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:44.983135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:44.983135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:06:44.983135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:44.983135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:44.983135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:44.983135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:06:44.983135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:44.983135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:06:44.983135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:06:44.983135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:44.983135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:06:44.983135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:44.983135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:06:44.983135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:06:44.983135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:44.983481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:06:44.983481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:44.983481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:06:49.264451 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:06:49.265238 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:06:49.265238 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:06:49.266446 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:06:49.299985 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:06:49.300290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:49.301273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:06:49.301273 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:49.301273 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:06:49.301273 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:06:49.301273 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:49.301273 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:06:49.301273 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:49.301273 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:06:49.302373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:49.302373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:06:49.302373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:49.302373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:06:49.302373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:06:49.302373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:06:49.302373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:06:49.302373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:49.302373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:49.302373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:49.302373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:06:49.302373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:49.303033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:49.303033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:49.303033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:06:49.303033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:49.303033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:06:49.303033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:06:49.303033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:49.303033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:06:49.303033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:49.303033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:06:49.303033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:06:49.303033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:49.303033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:06:49.303631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:49.303631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:06:49.303631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:49.303631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:06:49.303631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:06:49.303631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:06:49.303631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:06:49.303631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:06:49.303631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:49.303631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:06:49.303631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:06:49.303631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:06:49.303631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:06:49.303631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:06:49.303631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:49.304345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:49.304345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:06:49.304345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:06:49.304345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:06:49.304345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:06:49.304345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:49.304345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:49.304345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:49.304345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:06:49.304345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:49.304345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:49.304345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:49.304345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:06:49.304345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:49.304345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:06:49.304345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:06:49.304345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:49.304345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:06:49.304345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:49.305338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:06:49.305338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:06:49.305338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:49.305338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:06:49.305338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:49.305338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:06:49.946913 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:06:49.947280 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:06:49.947522 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:06:49.948632 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:06:49.948947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:49.949837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:06:49.949837 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:49.949837 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:06:49.949837 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:06:49.949837 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:49.949837 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:06:49.949837 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:49.949837 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:06:49.951599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:49.951599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:06:49.951599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:49.951599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:06:49.951599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:06:49.951599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:06:49.951599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:06:49.951599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:49.951599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:49.951599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:49.951599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:06:49.951599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:49.952172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:49.952172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:49.952172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:06:49.952172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:49.952172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:06:49.952172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:06:49.952172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:49.952172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:06:49.952172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:49.952172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:06:49.952172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:06:49.952172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:49.952172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:06:49.952772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:49.952772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:06:49.953335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:49.953335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:06:49.953335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:06:49.953335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:06:49.953335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:06:49.953335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:06:49.953335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:49.953335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:06:49.953335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:06:49.953335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:06:49.953335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:06:49.953335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:06:49.953335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:49.953335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:49.953335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:06:49.953335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:06:49.953791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:06:49.953791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:06:49.953791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:49.953791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:49.953791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:49.953791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:06:49.953791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:49.953791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:49.953791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:49.953791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:06:49.953791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:49.953791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:06:49.953791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:06:49.953791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:49.953791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:06:49.953791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:49.953791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:06:49.953791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:06:49.953791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:49.954317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:06:49.954317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:49.954317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:06:49.954317 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:06:49.954317 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:06:49.954507 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:06:49.954549 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:06:49.955055 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:06:49.955338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:49.956105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:06:49.956105 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:49.956105 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:06:49.956105 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:06:49.956105 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:49.956105 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:06:49.956105 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:49.956105 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:06:49.957339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:49.957339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:06:49.957339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:06:49.957339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:06:49.957339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:49.957339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:06:49.957339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:49.957339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:06:49.957339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:49.957339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:06:49.957339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:49.957339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:06:49.957339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:49.957339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:06:49.957339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:06:49.957819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:49.957819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:06:49.957819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:49.957819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:06:49.957819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:06:49.957819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:49.957819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:06:49.957819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:49.957819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:06:49.958769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:49.958769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:06:49.958769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:49.958769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:06:49.958769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:06:49.958769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:06:49.958769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:06:49.958769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:06:49.958769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:06:49.958769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:06:49.958769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:49.958769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:49.959182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:49.959182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:06:49.959182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:49.959182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:49.959182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:49.959182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:06:49.959182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:49.959182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:06:49.959182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:06:49.959182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:49.959182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:06:49.959182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:49.959182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:06:49.959182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:06:49.959600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:49.959600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:06:49.959600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:49.959600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:06:49.960425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:49.960425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:06:49.960425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:06:49.960425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:06:49.960425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:06:49.960425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:06:49.960425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:49.960425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:06:49.960425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:06:49.960425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:06:49.960425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:06:49.960425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:06:49.960425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:49.960425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:49.960425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:06:49.960425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:06:49.960860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:06:49.960860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:06:49.960860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:06:49.960860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:06:49.960860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:06:49.960860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:49.960860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:49.960860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:49.960860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:06:49.960860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:49.960860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:49.960860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:49.960860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:06:49.960860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:49.960860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:06:49.960860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:06:49.960860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:49.960860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:06:49.961376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:49.961376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:06:49.961376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:06:49.961376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:49.961376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:06:49.961376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:49.961376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:06:49.968063 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:06:49.968340 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:06:49.968576 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:06:49.968773 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:06:49.969213 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:06:49.969601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:49.970287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:06:49.970287 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:49.970287 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:06:49.970287 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:06:49.970287 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:49.970287 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:06:49.970287 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:49.970287 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:06:49.971456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:49.971456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:06:49.971456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:06:49.971456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:06:49.971456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:49.971456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:06:49.971456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:49.971456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:06:49.971456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:49.971456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:06:49.971456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:49.971456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:06:49.971456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:49.971456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:06:49.971456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:06:49.972244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:49.972244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:06:49.972244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:49.972244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:06:49.972244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:06:49.972244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:49.972244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:06:49.972244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:49.972244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:06:49.972926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:49.972926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:06:49.972926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:49.972926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:06:49.972926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:06:49.972926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:06:49.972926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:06:49.972926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:06:49.972926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:06:49.972926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:49.972926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:49.973477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:49.973477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:06:49.973477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:49.973477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:49.973477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:49.973477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:06:49.973477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:49.973477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:06:49.973477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:06:49.973477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:49.973477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:06:49.973477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:49.973477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:06:49.973477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:06:49.974080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:49.974080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:06:49.974080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:49.974080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:06:49.974591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:49.974591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:06:49.974591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:06:49.974591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:06:49.974591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:06:49.974591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:06:49.974591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:49.974591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:06:49.974591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:06:49.974591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:06:49.974591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:06:49.974591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:06:49.974591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:49.974591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:49.974591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:06:49.974591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:06:49.974591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:06:49.975395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:06:49.975395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:06:49.975395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:06:49.975395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:49.975395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:49.975395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:49.975395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:06:49.975395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:49.975395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:49.975395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:49.975395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:06:49.975395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:49.975395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:06:49.975395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:06:49.975395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:49.975395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:06:49.975395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:49.975395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:06:49.975395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:06:49.976331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:49.976331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:06:49.976331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:49.976331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:06:49.979065 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:06:49.979258 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:06:49.979369 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:06:49.979568 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:06:49.979966 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:06:49.980290 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:06:49.980517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:49.981190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:06:49.981190 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:49.981190 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:06:49.981190 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:06:49.981190 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:49.981190 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:06:49.981190 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:49.981190 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:06:49.981999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:49.981999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:06:49.981999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:49.981999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:06:49.981999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:06:49.981999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:06:49.981999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:06:49.981999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:49.981999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:49.981999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:49.981999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:06:49.981999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:49.982376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:49.982376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:49.982376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:06:49.982376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:49.982376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:06:49.982376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:06:49.982376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:49.982376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:06:49.982376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:49.982376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:06:49.982376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:06:49.982376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:49.982376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:06:49.982758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:49.982758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:06:49.982933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:49.982933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:06:49.982933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:06:49.982933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:06:49.982933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:06:49.982933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:06:49.982933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:49.982933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:06:49.982933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:06:49.982933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:06:49.982933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:06:49.982933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:06:49.982933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:49.982933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:49.982933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:06:49.982933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:06:49.983371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:06:49.983371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:06:49.983371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:49.983371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:49.983371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:49.983371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:06:49.983371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:49.983371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:49.983371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:49.983371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:06:49.983371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:49.983371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:06:49.983371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:06:49.983371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:49.983371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:06:49.983371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:49.983371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:06:49.983371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:06:49.983371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:49.983993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:06:49.983993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:49.983993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:06:54.265118 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:06:54.265931 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:06:54.265931 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:06:54.267078 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:06:54.296798 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:06:54.297140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:54.297950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:06:54.297950 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:54.297950 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:06:54.297950 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:06:54.297950 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:54.297950 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:06:54.297950 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:54.297950 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:06:54.298836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:54.298836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:06:54.298836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:54.298836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:06:54.298836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:06:54.298836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:06:54.298836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:06:54.298836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:54.298836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:54.298836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:54.298836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:06:54.298836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:54.299819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:54.299819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:54.299819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:06:54.299819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:54.299819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:06:54.299819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:06:54.299819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:54.299819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:06:54.299819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:54.299819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:06:54.299819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:06:54.299819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:54.299819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:06:54.300388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:54.300388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:06:54.300388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:54.300388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:06:54.300388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:06:54.300388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:06:54.300388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:06:54.300388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:06:54.300388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:54.300388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:06:54.300388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:06:54.300388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:06:54.300388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:06:54.300388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:06:54.300388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:54.301099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:54.301099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:06:54.301099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:06:54.301099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:06:54.301099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:06:54.301099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:54.301099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:54.301099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:54.301099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:06:54.301099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:54.301099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:54.301099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:54.301099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:06:54.301099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:54.301099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:06:54.301099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:06:54.301099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:54.301099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:06:54.301099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:54.302050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:06:54.302050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:06:54.302050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:54.302050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:06:54.302050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:54.302050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:06:54.947634 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:06:54.948233 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:06:54.948434 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:06:54.949555 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:06:54.949884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:54.950874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:06:54.950874 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:54.950874 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:06:54.950874 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:06:54.950874 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:54.950874 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:06:54.950874 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:54.950874 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:06:54.952516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:54.952516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:06:54.952516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:54.952516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:06:54.952516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:06:54.952516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:06:54.952516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:06:54.952516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:54.952516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:54.952516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:54.952516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:06:54.952516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:54.953131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:54.953131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:54.953131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:06:54.953131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:54.953131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:06:54.953131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:06:54.953131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:54.953131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:06:54.953131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:54.953131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:06:54.953131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:06:54.953131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:54.953131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:06:54.953758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:54.953758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:06:54.953976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:54.953976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:06:54.953976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:06:54.953976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:06:54.953976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:06:54.953976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:06:54.953976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:54.953976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:06:54.953976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:06:54.953976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:06:54.953976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:06:54.953976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:06:54.953976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:54.953976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:54.953976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:06:54.953976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:06:54.954751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:06:54.954751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:06:54.954751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:54.954751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:54.954751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:54.954751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:06:54.954751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:54.954751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:54.954751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:54.954751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:06:54.954751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:54.954751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:06:54.954751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:06:54.954751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:54.954751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:06:54.954751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:54.954751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:06:54.954751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:06:54.954751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:54.955681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:06:54.955681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:54.955681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:06:54.955681 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:06:54.955681 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:06:54.955681 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:06:54.955681 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:06:54.955681 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:06:54.956117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:54.956233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:06:54.956233 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:54.956233 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:06:54.956233 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:06:54.956233 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:54.956233 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:06:54.956233 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:54.956233 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:06:54.956987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:54.956987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:06:54.956987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:06:54.956987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:06:54.956987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:54.956987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:06:54.956987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:54.956987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:06:54.956987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:54.956987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:06:54.956987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:54.956987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:06:54.956987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:54.956987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:06:54.956987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:06:54.957858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:54.957858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:06:54.957858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:54.957858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:06:54.957858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:06:54.957858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:54.957858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:06:54.957858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:54.957858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:06:54.957858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:54.957858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:06:54.957858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:54.957858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:06:54.957858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:06:54.958664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:06:54.958664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:06:54.958664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:06:54.958664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:06:54.958664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:06:54.958664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:54.958664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:54.958664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:54.958664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:06:54.958664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:54.958664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:54.958664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:54.958664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:06:54.959337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:54.959337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:06:54.959337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:06:54.959337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:54.959337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:06:54.959337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:54.959337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:06:54.959337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:06:54.959337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:54.959337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:06:54.959337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:54.959337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:06:54.960099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:54.960099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:06:54.960099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:06:54.960099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:06:54.960099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:06:54.960099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:06:54.960099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:54.960099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:06:54.960099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:06:54.960099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:06:54.960099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:06:54.960099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:06:54.960099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:54.960099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:54.960099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:06:54.960099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:06:54.961274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:06:54.961274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:06:54.961274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:06:54.961274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:06:54.961274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:06:54.961274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:54.961274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:54.961274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:54.961274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:06:54.961274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:54.961274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:54.961274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:54.961274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:06:54.961274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:54.961274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:06:54.961274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:06:54.961274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:54.961274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:06:54.961849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:54.961849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:06:54.961849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:06:54.961849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:54.961849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:06:54.961849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:54.961849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:06:54.968488 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:06:54.968789 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:06:54.968899 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:06:54.969024 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:06:54.969500 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:06:54.969831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:54.970543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:06:54.970543 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:54.970543 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:06:54.970543 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:06:54.970543 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:54.970543 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:06:54.970543 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:54.970543 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:06:54.971862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:54.971862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:06:54.971862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:06:54.971862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:06:54.971862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:54.971862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:06:54.971862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:54.971862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:06:54.971862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:54.971862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:06:54.971862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:54.971862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:06:54.971862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:54.971862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:06:54.971862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:06:54.972723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:54.972723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:06:54.972723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:54.972723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:06:54.972723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:06:54.972723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:54.972723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:06:54.972723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:54.972723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:06:54.973230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:54.973230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:06:54.973230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:54.973230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:06:54.973230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:06:54.973230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:06:54.973230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:06:54.973230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:06:54.973230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:06:54.973230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:54.973230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:54.973770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:54.973770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:06:54.973770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:54.973770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:54.973770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:54.973770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:06:54.973770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:54.973770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:06:54.973770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:06:54.973770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:54.973770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:06:54.973770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:54.973770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:06:54.973770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:06:54.974374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:54.974374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:06:54.974374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:54.974374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:06:54.974923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:54.974923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:06:54.974923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:06:54.974923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:06:54.974923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:06:54.974923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:06:54.974923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:54.974923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:06:54.974923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:06:54.974923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:06:54.974923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:06:54.974923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:06:54.974923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:54.974923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:54.974923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:06:54.974923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:06:54.974923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:06:54.975699 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:06:54.975699 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:06:54.975699 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:06:54.975699 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:54.975699 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:54.975699 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:54.975699 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:06:54.975699 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:54.975699 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:54.975699 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:54.975699 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:06:54.975699 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:54.975699 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:06:54.975699 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:06:54.975699 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:54.975699 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:06:54.975699 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:54.975699 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:06:54.975699 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:06:54.976539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:54.976539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:06:54.976539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:54.976539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:06:54.979432 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:06:54.979733 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:06:54.979832 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:06:54.979979 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:06:54.980390 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:06:54.980759 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:06:54.980936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:54.982012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:06:54.982012 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:54.982012 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:06:54.982012 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:06:54.982012 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:54.982012 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:06:54.982012 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:54.982012 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:06:54.982835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:54.982835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:06:54.982835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:54.982835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:06:54.982835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:06:54.982835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:06:54.982835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:06:54.982835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:54.982835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:54.982835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:54.982835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:06:54.982835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:54.983173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:54.983173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:54.983173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:06:54.983173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:54.983173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:06:54.983173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:06:54.983173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:54.983173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:06:54.983173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:54.983173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:06:54.983173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:06:54.983173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:54.983173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:06:54.983549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:54.983549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:06:54.983772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:54.983772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:06:54.983772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:06:54.983772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:06:54.983772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:06:54.983772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:06:54.983772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:54.983772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:06:54.983772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:06:54.983772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:06:54.983772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:06:54.983772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:06:54.983772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:54.983772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:54.983772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:06:54.983772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:06:54.984199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:06:54.984199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:06:54.984199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:54.984199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:54.984199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:54.984199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:06:54.984199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:54.984199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:54.984199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:54.984199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:06:54.984199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:54.984199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:06:54.984199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:06:54.984199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:54.984199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:06:54.984199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:54.984199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:06:54.984199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:06:54.984199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:54.984736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:06:54.984736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:54.984736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:06:59.265742 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:06:59.266675 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:06:59.266802 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:06:59.267599 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:06:59.293945 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:06:59.294364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:59.295179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:06:59.295179 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:59.295179 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:06:59.295179 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:06:59.295179 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:59.295179 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:06:59.295179 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:59.295179 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:06:59.296056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.028 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:59.296056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:06:59.296056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:59.296056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:06:59.296056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:06:59.296056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:06:59.296056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:06:59.296056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:59.296056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:59.296056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:59.296056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:06:59.296056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:59.296662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:59.296662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:59.296662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:06:59.296662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:59.296662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:06:59.296662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:06:59.296662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:59.296662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:06:59.296662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:59.296662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:06:59.296662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:06:59.296662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:59.296662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:06:59.297159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:59.297159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:06:59.297159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:59.297159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:06:59.297159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:06:59.297159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:06:59.297159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:06:59.297159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:06:59.297159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:59.297159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:06:59.297159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:06:59.297159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:06:59.297159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:06:59.297159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:06:59.297159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:59.297778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:59.297778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:06:59.297778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:06:59.297778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:06:59.297778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:06:59.297778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:59.297778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:59.297778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:59.297778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:06:59.297778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:59.297778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:59.297778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:59.297778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:06:59.297778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:59.297778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:06:59.297778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:06:59.297778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:59.297778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:06:59.297778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:59.298890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:06:59.298890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:06:59.298890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:59.298890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:06:59.298890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:59.298890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:06:59.948307 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:06:59.949059 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:06:59.949244 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:06:59.950128 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:06:59.950308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:59.951165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:06:59.951165 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:59.951165 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:06:59.951165 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:06:59.951165 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:59.951165 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:06:59.951165 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:59.951165 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:06:59.952671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:59.952671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:06:59.952671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:59.952671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:06:59.952671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:06:59.952671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:06:59.952671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:06:59.952671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:59.952671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:59.952671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:59.952671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:06:59.952671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:59.953193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:59.953193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:59.953193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:06:59.953193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:59.953193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:06:59.953193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:06:59.953193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:59.953193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:06:59.953193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:59.953193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:06:59.953193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:06:59.953193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:59.953193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:06:59.953707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:59.953707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:06:59.954309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:59.954309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:06:59.954309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:06:59.954309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:06:59.954309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:06:59.954309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:06:59.954309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:59.954309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:06:59.954309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:06:59.954309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:06:59.954309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:06:59.954309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:06:59.954309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:59.954309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:59.954309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:06:59.954309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:06:59.955791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:06:59.955791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:06:59.955791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:59.955791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:59.955791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:59.955791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:06:59.955791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:59.955791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:59.955791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:59.955791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:06:59.955791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:59.955791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:06:59.955791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:06:59.955791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:59.955791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:06:59.955791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:59.955791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:06:59.955791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:06:59.955791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:59.957181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:06:59.957181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:59.957181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:06:59.957181 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:06:59.957181 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:06:59.957181 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:06:59.957181 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:06:59.957181 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:06:59.957570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:59.957570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:06:59.957570 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:59.957570 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:06:59.957570 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:06:59.957570 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:59.957570 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:06:59.957570 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:59.957570 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:06:59.957941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:59.957941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:06:59.957941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:06:59.957941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:06:59.957941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:59.957941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:06:59.957941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:59.957941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:06:59.957941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:59.957941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:06:59.957941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:59.957941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:06:59.957941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:59.957941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:06:59.957941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:06:59.959378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:59.959378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:06:59.959378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:59.959378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:06:59.959378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:06:59.959378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:59.959378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:06:59.959378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:59.959378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:06:59.959378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:59.959378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:06:59.959378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:59.959378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:06:59.959378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:06:59.960725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:06:59.960725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:06:59.960725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:06:59.960725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:06:59.960725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:06:59.960725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:59.960725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:59.960725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:59.960725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:06:59.960725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:59.960725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:59.960725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:59.960725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:06:59.961656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:59.961656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:06:59.961656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:06:59.961656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:59.961656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:06:59.961656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:59.961656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:06:59.961656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:06:59.961656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:59.961656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:06:59.961656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:59.961656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:06:59.962154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:59.962154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:06:59.962154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:06:59.962154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:06:59.962154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:06:59.962154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:06:59.962154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:59.962154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:06:59.962154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:06:59.962154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:06:59.962154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:06:59.962154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:06:59.962154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:59.962154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:59.962154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:06:59.962154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:06:59.962889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:06:59.962889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:06:59.962889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:06:59.962889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:06:59.962889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:06:59.962889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:59.962889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:59.962889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:59.962889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:06:59.962889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:59.962889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:59.962889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:59.962889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:06:59.962889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:59.962889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:06:59.962889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:06:59.962889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:59.962889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:06:59.963672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:59.963672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:06:59.963672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:06:59.963672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:59.963672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:06:59.963672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:59.963672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:06:59.968932 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:06:59.969205 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:06:59.969318 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:06:59.969462 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:06:59.969828 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:06:59.970160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:59.970836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:06:59.970836 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:59.970836 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:06:59.970836 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:06:59.970836 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:59.970836 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:06:59.970836 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:59.970836 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:06:59.971555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:59.971555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:06:59.971555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:06:59.971555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:06:59.971555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:59.971555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:06:59.971555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:59.971555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:06:59.971555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:59.971555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:06:59.971555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:59.971555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:06:59.971555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:59.971555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:06:59.971555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:06:59.971853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:59.971853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:06:59.971853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:59.971853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:06:59.971853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:06:59.971853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:59.971853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:06:59.971853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:59.971853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:06:59.972417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:59.972417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:06:59.972417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:59.972417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:06:59.972417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:06:59.972417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:06:59.972417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:06:59.972417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:06:59.972417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:06:59.972417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:59.972417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:59.972859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:59.972859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:06:59.972859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:59.972859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:59.972859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:59.972859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:06:59.972859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:59.972859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:06:59.972859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:06:59.972859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:59.972859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:06:59.972859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:59.972859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:06:59.972859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:06:59.973526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:59.973526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:06:59.973526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:59.973526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:06:59.973526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:59.973526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:06:59.973526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:06:59.973526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:06:59.973526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:06:59.973526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:06:59.973526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:59.973526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:06:59.973526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:06:59.973526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:06:59.974186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:06:59.974186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:06:59.974186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:59.974186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:59.974186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:06:59.974186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:06:59.974186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:06:59.974186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:06:59.974186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:06:59.974186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:06:59.974186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:59.974186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:59.974186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:59.974186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:06:59.974186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:59.974186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:59.974186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:59.974186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:06:59.974186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:59.975116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:06:59.975116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:06:59.975116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:59.975116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:06:59.975116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:59.975116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:06:59.975116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:06:59.975116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:59.975116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:06:59.975116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:59.975116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:06:59.979912 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:06:59.980182 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:06:59.980253 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:06:59.980433 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:06:59.980824 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:06:59.981178 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:06:59.981359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:59.982645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:06:59.982645 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:59.982645 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:06:59.982645 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:06:59.982645 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:59.982645 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:06:59.982645 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:59.982645 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:06:59.983421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:59.983421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:06:59.983421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:59.983421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:06:59.983421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:06:59.983421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:06:59.983421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:06:59.983421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:59.983421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:59.983421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:59.983421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:06:59.983421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:59.983751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:06:59.983751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:59.983751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:06:59.983751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:59.983751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:06:59.983751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:06:59.983751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:59.983751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:06:59.983751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:59.983751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:06:59.983751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:06:59.983751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:59.983751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:06:59.984199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:59.984199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:06:59.984937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:59.984937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:06:59.984937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:06:59.984937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:06:59.984937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:06:59.984937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:06:59.984937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:59.984937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:06:59.984937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:06:59.984937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:06:59.984937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:06:59.984937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:06:59.984937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:06:59.984937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:06:59.984937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:06:59.984937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:06:59.985381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:06:59.985381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:06:59.985381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:59.985381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:59.985381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:06:59.985381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:06:59.985381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:06:59.985381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:06:59.985381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:06:59.985381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:06:59.985381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:06:59.985381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:06:59.985381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:06:59.985381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:06:59.985381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:06:59.985381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:06:59.985381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:06:59.985381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:06:59.985381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:06:59.985910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:06:59.985910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:06:59.985910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:07:04.267024 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:07:04.268188 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:07:04.268188 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:07:04.269114 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:07:04.299923 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:07:04.300341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:04.301163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:07:04.301163 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:04.301163 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:07:04.301163 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:07:04.301163 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:04.301163 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:07:04.301163 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:04.301163 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:07:04.301940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:04.301940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:07:04.301940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:04.301940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:07:04.301940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:07:04.301940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:07:04.301940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:07:04.301940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:04.301940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:04.301940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:04.301940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:07:04.301940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:04.302491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:04.302491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:04.302491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:07:04.302491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:04.302491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:07:04.302491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:07:04.302491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:04.302491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:07:04.302491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:04.302491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:07:04.302491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:07:04.302491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:04.302491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:07:04.303071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:04.303071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:07:04.303071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:04.303071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:07:04.303071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:07:04.303071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:07:04.303071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:07:04.303071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:07:04.303071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:04.303071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:07:04.303071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:07:04.303071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:07:04.303071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:07:04.303071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:07:04.303071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:04.303729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:04.303729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:07:04.303729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:07:04.303729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:07:04.303729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:07:04.303729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:04.303729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:04.303729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:04.303729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:07:04.303729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:04.303729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:04.303729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:04.303729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:07:04.303729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:04.303729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:07:04.303729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:07:04.303729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:04.303729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:07:04.303729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:04.305779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:07:04.305779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:07:04.305779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:04.305779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:07:04.305779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:04.305779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:07:04.949156 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:07:04.949829 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:07:04.950052 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:07:04.951086 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:07:04.951426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:04.952343 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:07:04.952343 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:04.952343 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:07:04.952343 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:07:04.952343 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:04.952343 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:07:04.952343 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:04.952343 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:07:04.954053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:04.954053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:07:04.954053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:04.954053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:07:04.954053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:07:04.954053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:07:04.954053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:07:04.954053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:04.954053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:04.954053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:04.954053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:07:04.954053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:04.954671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:04.954671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:04.954671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:07:04.954671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:04.954671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:07:04.954671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:07:04.954671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:04.954671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:07:04.954671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:04.954671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:07:04.954671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:07:04.954671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:04.954671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:07:04.955174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:04.955174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:07:04.955752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:04.955752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:07:04.955752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:07:04.955752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:07:04.955752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:07:04.955752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:07:04.955752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:04.955752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:07:04.955752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:07:04.955752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:07:04.955752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:07:04.955752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:07:04.955752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:04.955752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:04.955752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:07:04.955752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:07:04.957456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:07:04.957456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:07:04.957456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:04.957456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:04.957456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:04.957456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:07:04.957456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:04.957456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:04.957456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:04.957456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:07:04.957456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:04.957456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:07:04.957456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:07:04.957456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:04.957456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:07:04.957456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:04.957456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:07:04.957456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:07:04.957456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:04.959212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:07:04.959212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:04.959212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:07:04.959212 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:07:04.959212 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:07:04.959212 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:07:04.959212 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:07:04.959212 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:07:04.959632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:04.959632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:07:04.959632 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:04.959632 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:07:04.959632 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:07:04.959632 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:04.959632 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:07:04.959632 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:04.959632 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:07:04.959632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:04.959632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:07:04.959632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:07:04.959632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:07:04.959632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:04.959632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:07:04.959632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:04.959632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:07:04.960843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:04.960843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:07:04.960843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:04.960843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:07:04.960843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:04.960843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:07:04.960843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:07:04.960843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:04.960843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:07:04.960843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:04.960843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:07:04.960843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:07:04.960843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:04.960843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:07:04.960843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:04.960843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:07:04.961588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:04.961588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:07:04.961588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:04.961588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:07:04.961588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:07:04.961588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:07:04.961588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:07:04.961588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:07:04.961588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:07:04.961588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:07:04.961588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:04.961588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:04.962141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:04.962141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:07:04.962141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:04.962141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:04.962141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:04.962141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:07:04.962141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:04.962141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:07:04.962141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:07:04.962141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:04.962141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:07:04.962141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:04.962141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:07:04.962141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:07:04.962767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:04.962767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:07:04.962767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:04.962767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:07:04.962767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:04.962767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:07:04.962767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:07:04.962767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:07:04.962767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:07:04.962767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:07:04.962767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:04.962767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:07:04.962767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:07:04.962767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:07:04.963096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:07:04.963096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:07:04.963096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:04.963096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:04.963096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:07:04.963096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:07:04.963096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:07:04.963096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:07:04.963096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:07:04.963096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:07:04.963096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:07:04.963096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:04.963096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:04.963096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:04.963096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:07:04.963096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:04.963096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:04.963096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:04.963096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:07:04.963477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:04.963477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:07:04.963477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:07:04.963477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:04.963477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:07:04.963477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:04.963477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:07:04.963477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:07:04.963477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:04.963477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:07:04.963477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:04.963477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:07:04.969340 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:07:04.969607 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:07:04.969675 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:07:04.969756 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:07:04.970156 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:07:04.970547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:04.971146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:07:04.971146 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:04.971146 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:07:04.971146 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:07:04.971146 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:04.971146 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:07:04.971146 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:04.971146 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:07:04.971698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:04.971698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:07:04.971698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:07:04.971698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:07:04.971698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:04.971698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:07:04.971698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:04.971698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:07:04.971698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:04.971698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:07:04.971698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:04.971698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:07:04.971698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:04.971698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:07:04.971698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:07:04.972131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:04.972131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:07:04.972131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:04.972131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:07:04.972131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:07:04.972131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:04.972131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:07:04.972131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:04.972131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:07:04.973113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:04.973113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:07:04.973113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:04.973113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:07:04.973113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:07:04.973113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:07:04.973113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:07:04.973113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:07:04.973113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:07:04.973113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:04.973113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:04.973504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:04.973504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:07:04.973504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:04.973504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:04.973504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:04.973504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:07:04.973504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:04.973504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:07:04.973504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:07:04.973504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:04.973504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:07:04.973504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:04.973504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:07:04.973504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:07:04.973837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:04.973837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:07:04.973837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:04.973837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:07:04.974848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:04.974848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:07:04.974848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:07:04.974848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:07:04.974848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:07:04.974848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:07:04.974848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:04.974848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:07:04.974848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:07:04.974848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:07:04.974848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:07:04.974848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:07:04.974848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:04.974848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:04.974848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:07:04.974848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:07:04.974848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:07:04.975310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:07:04.975310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:07:04.975310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:07:04.975310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:04.975310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:04.975310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:04.975310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:07:04.975310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:04.975310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:04.975310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:04.975310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:07:04.975310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:04.975310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:07:04.975310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:07:04.975310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:04.975310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:07:04.975310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:04.975310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:07:04.975310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:07:04.976290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:04.976290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:07:04.976290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:04.976290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:07:04.980389 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:07:04.980755 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:07:04.980865 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:07:04.981012 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:07:04.981392 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:07:04.981750 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:07:04.981918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:04.982568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:07:04.982568 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:04.982568 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:07:04.982568 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:07:04.982568 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:04.982568 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:07:04.982568 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:04.982568 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:07:04.983344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:04.983344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:07:04.983344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:04.983344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:07:04.983344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:07:04.983344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:07:04.983344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:07:04.983344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:04.983344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:04.983344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:04.983344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:07:04.983344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:04.983758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:04.983758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:04.983758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:07:04.983758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:04.983758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:07:04.983758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:07:04.983758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:04.983758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:07:04.983758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:04.983758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:07:04.983758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:07:04.983758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:04.983758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:07:04.984114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:04.984114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:07:04.984282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:04.984282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:07:04.984282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:07:04.984282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:07:04.984282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:07:04.984282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:07:04.984282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:04.984282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:07:04.984282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:07:04.984282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:07:04.984282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:07:04.984282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:07:04.984282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:04.984282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:04.984282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:07:04.984282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:07:04.984857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:07:04.984857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:07:04.984857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:04.984857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:04.984857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:04.984857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:07:04.984857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:04.984857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:04.984857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:04.984857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:07:04.984857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:04.984857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:07:04.984857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:07:04.984857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:04.984857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:07:04.984857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:04.984857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:07:04.984857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:07:04.984857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:04.985380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:07:04.985380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:04.985380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:07:09.267647 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:07:09.270231 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:07:09.270231 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:07:09.270231 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:07:09.302371 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:07:09.302807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:09.303665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:07:09.303665 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:09.303665 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:07:09.303665 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:07:09.303665 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:09.303665 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:07:09.303665 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:09.303665 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:07:09.304438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:09.304438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:07:09.304438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:09.304438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:07:09.304438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:07:09.304438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:07:09.304438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:07:09.304438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:09.304438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:09.304438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:09.304438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:07:09.304438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:09.305049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:09.305049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:09.305049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:07:09.305049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:09.305049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:07:09.305049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:07:09.305049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:09.305049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:07:09.305049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:09.305049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:07:09.305049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:07:09.305049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:09.305049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:07:09.305621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:09.305621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:07:09.305621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:09.305621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:07:09.305621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:07:09.305621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:07:09.305621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:07:09.305621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:07:09.305621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:09.305621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:07:09.305621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:07:09.305621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:07:09.305621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:07:09.305621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:07:09.305621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:09.306456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:09.306456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:07:09.306456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:07:09.306456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:07:09.306456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:07:09.306456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:09.306456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:09.306456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:09.306456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:07:09.306456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:09.306456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:09.306456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:09.306456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:07:09.306456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:09.306456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:07:09.306456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:07:09.306456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:09.306456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:07:09.306456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:09.307270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:07:09.307270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:07:09.307270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:09.307270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:07:09.307270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:09.307270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:07:09.949808 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:07:09.950324 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:07:09.950621 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:07:09.951702 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:07:09.952065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:09.952978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:07:09.952978 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:09.952978 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:07:09.952978 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:07:09.952978 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:09.952978 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:07:09.952978 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:09.952978 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:07:09.954602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:09.954602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:07:09.954602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:09.954602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:07:09.954602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:07:09.954602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:07:09.954602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:07:09.954602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:09.954602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:09.954602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:09.954602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:07:09.954602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:09.955018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:09.955018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:09.955018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:07:09.955018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:09.955018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:07:09.955018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:07:09.955018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:09.955018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:07:09.955018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:09.955018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:07:09.955018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:07:09.955018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:09.955018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:07:09.955427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:09.955427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:07:09.956201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:09.956201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:07:09.956201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:07:09.956201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:07:09.956201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:07:09.956201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:07:09.956201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:09.956201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:07:09.956201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:07:09.956201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:07:09.956201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:07:09.956201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:07:09.956201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:09.956201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:09.956201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:07:09.956201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:07:09.956612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:07:09.956612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:07:09.956612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:09.956612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:09.956612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:09.956612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:07:09.956612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:09.956612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:09.956612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:09.956612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:07:09.956612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:09.956612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:07:09.956612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:07:09.956612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:09.956612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:07:09.956612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:09.956612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:07:09.956612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:07:09.956612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:09.957148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:07:09.957148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:09.957148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:07:09.957148 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:07:09.957148 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:07:09.957148 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:07:09.957359 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:07:09.957595 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:07:09.957797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:09.958389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:07:09.958389 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:09.958389 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:07:09.958389 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:07:09.958389 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:09.958389 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:07:09.958389 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:09.958389 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:07:09.959549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:09.959549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:07:09.959549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:07:09.959549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:07:09.959549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:09.959549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:07:09.959549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:09.959549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:07:09.959549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:09.959549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:07:09.959549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:09.959549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:07:09.959549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:09.959549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:07:09.959549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:07:09.959946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:09.959946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:07:09.959946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:09.959946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:07:09.959946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:07:09.959946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:09.959946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:07:09.959946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:09.959946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:07:09.960942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:09.960942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:07:09.960942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:09.960942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:07:09.960942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:07:09.960942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:07:09.960942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:07:09.960942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:07:09.960942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:07:09.960942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:07:09.960942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:09.960942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:09.961338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:09.961338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:07:09.961338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:09.961338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:09.961338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:09.961338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:07:09.961338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:09.961338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:07:09.961338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:07:09.961338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:09.961338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:07:09.961338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:09.961338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:07:09.961338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:07:09.961707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:09.961707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:07:09.961707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:09.961707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:07:09.962682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:09.962682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:07:09.962682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:07:09.962682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:07:09.962682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:07:09.962682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:07:09.962682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:09.962682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:07:09.962682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:07:09.962682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:07:09.962682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:07:09.962682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:07:09.962682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:09.962682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:09.962682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:07:09.962682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:07:09.963193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:07:09.963193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:07:09.963193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:07:09.963193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:07:09.963193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:07:09.963193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:09.963193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:09.963193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:09.963193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:07:09.963193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:09.963193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:09.963193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:09.963193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:07:09.963193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:09.963193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:07:09.963193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:07:09.963193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:09.963193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:07:09.963706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:09.963706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:07:09.963706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:07:09.963706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:09.963706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:07:09.963706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:09.963706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:07:09.969962 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:07:09.970221 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:07:09.970288 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:07:09.970393 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:07:09.970867 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:07:09.971258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:09.972191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:07:09.972191 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:09.972191 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:07:09.972191 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:07:09.972191 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:09.972191 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:07:09.972191 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:09.972191 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:07:09.973508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:09.973508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:07:09.973508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:07:09.973508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:07:09.973508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:09.973508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:07:09.973508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:09.973508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:07:09.973508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:09.973508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:07:09.973508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:09.973508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:07:09.973508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:09.973508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:07:09.973508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:07:09.974203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:09.974203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:07:09.974203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:09.974203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:07:09.974203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:07:09.974203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:09.974203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:07:09.974203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:09.974203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:07:09.974885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:09.974885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:07:09.974885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:09.974885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:07:09.974885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:07:09.974885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:07:09.974885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:07:09.974885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:07:09.974885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:07:09.974885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:09.974885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:09.975361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:09.975361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:07:09.975361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:09.975361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:09.975361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:09.975361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:07:09.975361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:09.975361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:07:09.975361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:07:09.975361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:09.975361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:07:09.975361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:09.975361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:07:09.975361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:07:09.976071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:09.976071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:07:09.976071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:09.976071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:07:09.976616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:09.976616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:07:09.976616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:07:09.976616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:07:09.976616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:07:09.976616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:07:09.976616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:09.976616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:07:09.976616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:07:09.976616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:07:09.976616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:07:09.976616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:07:09.976616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:09.976616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:09.976616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:07:09.976616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:07:09.976616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:07:09.977263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:07:09.977263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:07:09.977263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:07:09.977263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:09.977263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:09.977263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:09.977263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:07:09.977263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:09.977263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:09.977263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:09.977263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:07:09.977263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:09.977263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:07:09.977263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:07:09.977263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:09.977263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:07:09.977263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:09.977263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:07:09.977263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:07:09.977751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:09.977751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:07:09.977751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:09.977751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:07:09.980875 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:07:09.981235 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:07:09.981420 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:07:09.981652 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:07:09.981998 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:07:09.982208 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:07:09.982324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:09.982802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:07:09.982802 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:09.982802 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:07:09.982802 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:07:09.982802 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:09.982802 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:07:09.982802 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:09.982802 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:07:09.983599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:09.983599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:07:09.983599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:09.983599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:07:09.983599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:07:09.983599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:07:09.983599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:07:09.983599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:09.983599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:09.983599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:09.983599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:07:09.983599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:09.983980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:09.983980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:09.983980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:07:09.983980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:09.983980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:07:09.983980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:07:09.983980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:09.983980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:07:09.983980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:09.983980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:07:09.983980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:07:09.983980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:09.983980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:07:09.984531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:09.984531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:07:09.984531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:09.984531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:07:09.984531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:07:09.984531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:07:09.984531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:07:09.984531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:07:09.984531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:09.984531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:07:09.984531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:07:09.984531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:07:09.984531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:07:09.984531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:07:09.984531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:09.985075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:09.985075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:07:09.985075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:07:09.985075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:07:09.985075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:07:09.985075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:09.985075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:09.985075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:09.985075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:07:09.985075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:09.985075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:09.985075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:09.985075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:07:09.985075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:09.985075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:07:09.985075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:07:09.985075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:09.985075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:07:09.985075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:09.985629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:07:09.985629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:07:09.985629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:09.985629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:07:09.985629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:09.985629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:07:14.268587 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:07:14.269759 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:07:14.269759 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:07:14.270563 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:07:14.308859 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:07:14.309254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:14.310140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:07:14.310140 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:14.310140 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:07:14.310140 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:07:14.310140 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:14.310140 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:07:14.310140 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:14.310140 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:07:14.311186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:14.311186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:07:14.311186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:14.311186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:07:14.311186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:07:14.311186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:07:14.311186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:07:14.311186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:14.311186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:14.311186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:14.311186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:07:14.311186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:14.311840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:14.311840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:14.311840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:07:14.311840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:14.311840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:07:14.311840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:07:14.311840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:14.311840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:07:14.311840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:14.311840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:07:14.311840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:07:14.311840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:14.311840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:07:14.312501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:14.312501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:07:14.312501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:14.312501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:07:14.312501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:07:14.312501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:07:14.312501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:07:14.312501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:07:14.312501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:14.312501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:07:14.312501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:07:14.312501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:07:14.312501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:07:14.312501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:07:14.312501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:14.313224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:14.313224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:07:14.313224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:07:14.313224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:07:14.313224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:07:14.313224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:14.313224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:14.313224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:14.313224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:07:14.313224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:14.313224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:14.313224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:14.313224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:07:14.313224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:14.313224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:07:14.313224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:07:14.313224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:14.313224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:07:14.313224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:14.314174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:07:14.314174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:07:14.314174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:14.314174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:07:14.314174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:14.314174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:07:14.953871 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:07:14.954636 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:07:14.954875 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:07:14.956025 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:07:14.956382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:14.957199 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:07:14.957589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:07:14.957589 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:14.957589 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:07:14.957589 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:07:14.957589 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:14.957589 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:07:14.957589 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:14.957589 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:07:14.959097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:14.959097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:07:14.959097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:14.959097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:07:14.959097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:07:14.959097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:07:14.959097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:07:14.959097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:14.959097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:14.959097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:14.959097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:07:14.959097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:14.959682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:14.959682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:14.959682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:07:14.959682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:14.959682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:07:14.959682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:07:14.959682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:14.959682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:07:14.959682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:14.959682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:07:14.959682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:07:14.959682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:14.959682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:07:14.960556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:14.960556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:07:14.960738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:14.960738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:07:14.960738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:07:14.960738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:07:14.960738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:07:14.960738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:07:14.960738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:14.960738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:07:14.960738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:07:14.960738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:07:14.960738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:07:14.960738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:07:14.960738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:14.960738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:14.960738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:07:14.960738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:07:14.961508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:07:14.961508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:07:14.961508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:14.961508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:14.961508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:14.961508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:07:14.961508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:14.961508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:14.961508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:14.961508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:07:14.961508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:14.961508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:07:14.961508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:07:14.961508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:14.961508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:07:14.961508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:14.961508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:07:14.961508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:07:14.961508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:14.962252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:07:14.962252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:14.962252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:07:14.962252 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:07:14.962252 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:07:14.962252 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:07:14.962252 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:07:14.962252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:14.963456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:07:14.963456 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:14.963456 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:07:14.963456 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:07:14.963456 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:14.963456 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:07:14.963456 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:14.963456 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:07:14.964139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:14.964139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:07:14.964139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:07:14.964139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:07:14.964139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:14.964139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:07:14.964139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:14.964139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:07:14.964139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:14.964139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:07:14.964139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:14.964139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:07:14.964139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:14.964139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:07:14.964139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:07:14.965654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:14.965654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:07:14.965654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:14.965654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:07:14.965654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:07:14.965654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:14.965654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:07:14.965654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:14.965654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:07:14.965654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:14.965654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:07:14.965654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:14.965654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:07:14.965654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:07:14.966223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:07:14.966223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:07:14.966223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:07:14.966223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:07:14.966223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:07:14.966223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:14.966223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:14.966223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:14.966223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:07:14.966223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:14.966223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:14.966223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:14.966223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:07:14.966774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:14.966774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:07:14.966774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:07:14.966774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:14.966774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:07:14.966774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:14.966774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:07:14.966774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:07:14.966774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:14.966774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:07:14.966774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:14.966774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:07:14.967258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:14.967258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:07:14.967258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:07:14.967258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:07:14.967258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:07:14.967258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:07:14.967258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:14.967258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:07:14.967258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:07:14.967258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:07:14.967258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:07:14.967258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:07:14.967258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:14.967258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:14.967258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:07:14.967258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:07:14.968344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:07:14.968344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:07:14.968344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:07:14.968344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:07:14.968344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:07:14.968344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:14.968344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:14.968344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:14.968344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:07:14.968344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:14.968344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:14.968344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:14.968344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:07:14.968344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:14.968344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:07:14.968344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:07:14.968344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:14.968344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:07:14.969097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:14.969097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:07:14.969097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:07:14.969097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:14.969097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:07:14.969097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:14.969097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:07:14.970318 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:07:14.970587 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:07:14.970702 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:07:14.970832 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:07:14.971151 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:07:14.971387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:14.971923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:07:14.971923 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:14.971923 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:07:14.971923 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:07:14.971923 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:14.971923 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:07:14.971923 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:14.971923 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:07:14.972729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:14.972729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:07:14.972729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:07:14.972729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:07:14.972729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:14.972729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:07:14.972729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:14.972729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:07:14.972729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:14.972729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:07:14.972729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:14.972729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:07:14.972729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:14.972729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:07:14.972729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:07:14.973304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:14.973304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:07:14.973304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:14.973304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:07:14.973304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:07:14.973304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:14.973304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:07:14.973304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:14.973304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:07:14.973659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:14.973659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:07:14.973659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:14.973659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:07:14.973659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:07:14.973659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:07:14.973659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:07:14.973659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:07:14.973659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:07:14.973659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:14.973659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:14.974055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:14.974055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:07:14.974055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:14.974055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:14.974055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:14.974055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:07:14.974055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:14.974055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:07:14.974055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:07:14.974055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:14.974055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:07:14.974055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:14.974055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:07:14.974055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:07:14.974294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:14.974294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:07:14.974294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:14.974294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:07:14.974735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:14.974735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:07:14.974735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:07:14.974735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:07:14.974735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:07:14.974735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:07:14.974735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:14.974735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:07:14.974735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:07:14.974735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:07:14.974735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:07:14.974735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:07:14.974735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:14.974735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:14.974735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:07:14.974735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:07:14.974735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:07:14.975052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:07:14.975052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:07:14.975052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:07:14.975052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:14.975052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:14.975052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:14.975052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:07:14.975052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:14.975052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:14.975052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:14.975052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:07:14.975052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:14.975052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:07:14.975052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:07:14.975052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:14.975052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:07:14.975052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:14.975052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:07:14.975052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:07:14.975380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:14.975380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:07:14.975380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:14.975380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:07:14.981214 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:07:14.981502 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:07:14.981630 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:07:14.981783 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:07:14.982182 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:07:14.982583 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:07:14.982770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:14.983478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:07:14.983478 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:14.983478 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:07:14.983478 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:07:14.983478 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:14.983478 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:07:14.983478 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:14.983478 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:07:14.984490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:14.984490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:07:14.984490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:14.984490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:07:14.984490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:07:14.984490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:07:14.984490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:07:14.984490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:14.984490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:14.984490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:14.984490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:07:14.984490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:14.984801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:14.984801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:14.984801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:07:14.984801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:14.984801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:07:14.984801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:07:14.984801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:14.984801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:07:14.984801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:14.984801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:07:14.984801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:07:14.984801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:14.984801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:07:14.985103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:14.985103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:07:14.985995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:14.985995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:07:14.985995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:07:14.985995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:07:14.985995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:07:14.985995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:07:14.985995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:14.985995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:07:14.985995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:07:14.985995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:07:14.985995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:07:14.985995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:07:14.985995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:14.985995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:14.985995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:07:14.985995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:07:14.986399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:07:14.986399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:07:14.986399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:14.986399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:14.986399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:14.986399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:07:14.986399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:14.986399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:14.986399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:14.986399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:07:14.986399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:14.986399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:07:14.986399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:07:14.986399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:14.986399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:07:14.986399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:14.986399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:07:14.986399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:07:14.986399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:14.986873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:07:14.986873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:14.986873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:07:19.269187 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:07:19.272959 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:07:19.272959 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:07:19.272959 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:07:19.305717 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:07:19.306180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:19.309144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:07:19.309144 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:19.309144 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:07:19.309144 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:07:19.309144 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:19.309144 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:07:19.309144 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:19.309144 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:07:19.309144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:19.309144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:07:19.309144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:19.309144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:07:19.309144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:07:19.309144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:07:19.309144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:07:19.310102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:19.310102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:19.310102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:19.310102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:07:19.310102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:19.310102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:19.310102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:19.310102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:07:19.310102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:19.310102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:07:19.310102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:07:19.310102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:19.310102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:07:19.310102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:19.311050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:07:19.311050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:07:19.311050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:19.311050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:07:19.311050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:19.311050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:07:19.311050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:19.311050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:07:19.311050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:07:19.311050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:07:19.311050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:07:19.311050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:07:19.311050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:19.311050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:07:19.311862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:07:19.311862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:07:19.311862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:07:19.311862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:07:19.311862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:19.311862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:19.311862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:07:19.311862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:07:19.311862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:07:19.311862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:07:19.311862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:19.311862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:19.311862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:19.311862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:07:19.311862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:19.311862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:19.311862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:19.311862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:07:19.311862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:19.311862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:07:19.311862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:07:19.313627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:19.313627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:07:19.313627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:19.313627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:07:19.313627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:07:19.313627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:19.313627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:07:19.313627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:19.313627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:07:19.954479 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:07:19.955859 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:07:19.955859 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:07:19.956171 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:07:19.956550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:19.960441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:07:19.960441 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:19.960441 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:07:19.960441 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:07:19.960441 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:19.960441 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:07:19.960441 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:19.960441 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:07:19.960441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:19.960441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:07:19.960441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:19.960441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:07:19.960441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:07:19.960441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:07:19.960441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:07:19.961151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:19.961151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:19.961151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:19.961151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:07:19.961151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:19.961151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:19.961151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:19.961151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:07:19.961151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:19.961151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:07:19.961151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:07:19.961151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:19.961151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:07:19.961151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:19.961750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:07:19.961750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:07:19.961750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:19.961750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:07:19.961750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:19.961750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:07:19.961750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:19.961750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:07:19.961750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:07:19.961750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:07:19.961750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:07:19.961750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:07:19.961750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:19.961750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:07:19.962333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:07:19.962333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:07:19.962333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:07:19.962333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:07:19.962333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:19.962333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:19.962333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:07:19.962333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:07:19.962333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:07:19.962333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:07:19.962333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:19.962333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:19.962333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:19.962333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:07:19.962333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:19.962333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:19.962333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:19.962333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:07:19.962333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:19.962333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:07:19.963163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:07:19.963163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:19.963163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:07:19.963163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:19.963163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:07:19.963163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:07:19.963163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:19.963163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:07:19.963163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:19.963163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:07:19.965548 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:07:19.966313 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:07:19.966313 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:07:19.966313 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:07:19.966685 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:07:19.966964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:19.969327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:07:19.969327 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:19.969327 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:07:19.969327 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:07:19.969327 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:19.969327 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:07:19.969327 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:19.969327 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:07:19.969327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:19.969327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:07:19.969327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:07:19.969327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:07:19.969327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:19.969327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:07:19.969327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:19.969327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:07:19.969327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:19.969327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:07:19.970098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:19.970098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:07:19.970098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:19.970098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:07:19.970098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:07:19.970098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:19.970098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:07:19.970098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:19.970098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:07:19.970098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:07:19.970098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:19.970098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:07:19.970098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:19.970098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:07:19.970098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:19.970730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:07:19.970730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:19.970730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:07:19.970730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:07:19.970730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:07:19.970730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:07:19.970730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:07:19.970730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:07:19.970730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:07:19.970730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:19.970730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:19.970730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:19.970730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:07:19.970977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:19.970977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:19.970977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:19.970977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:07:19.970977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:19.970977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:07:19.970977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:07:19.970977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:19.970977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:07:19.970977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:19.970977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:07:19.970977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:07:19.970977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:19.970977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:07:19.971210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:19.971210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:07:19.971210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:19.971210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:07:19.971210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:07:19.971210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:07:19.971210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:07:19.971210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:07:19.971210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:19.971210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:07:19.971210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:07:19.971210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:07:19.971210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:07:19.971210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:07:19.971210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:19.971490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:19.971490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:07:19.971490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:07:19.971490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:07:19.971490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:07:19.971490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:07:19.971490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:07:19.971490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:07:19.971490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:19.971490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:19.971490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:19.971490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:07:19.971490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:19.971490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:19.971490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:19.971490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:07:19.971490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:19.971490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:07:19.971490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:07:19.971826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:19.971826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:07:19.971826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:19.971826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:07:19.971826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:07:19.971826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:19.971826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:07:19.971826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:19.971826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:07:19.971826 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:07:19.973113 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:07:19.973113 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:07:19.973113 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:07:19.973507 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:07:19.973890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:19.974496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:07:19.974496 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:19.974496 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:07:19.974496 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:07:19.974496 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:19.974496 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:07:19.974496 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:19.974496 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:07:19.976585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:19.976585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:07:19.976585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:07:19.976585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:07:19.976585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:19.976585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:07:19.976585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:19.976585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:07:19.976585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:19.976585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:07:19.976585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:19.976585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:07:19.976585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:19.976585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:07:19.976585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:07:19.976851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:19.976851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:07:19.976851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:19.976851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:07:19.976851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:07:19.976851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:19.976851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:07:19.976851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:19.976851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:07:19.976851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:19.976851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:07:19.976851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:19.976851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:07:19.976851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:07:19.977097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:07:19.977097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:07:19.977097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:07:19.977097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:07:19.977097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:19.977097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:19.977097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:19.977097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:07:19.977097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:19.977097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:19.977097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:19.977097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:07:19.977097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:19.977341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:07:19.977341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:07:19.977341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:19.977341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:07:19.977341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:19.977341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:07:19.977341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:07:19.977341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:19.977341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:07:19.977341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:19.977341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:07:19.977341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:19.977570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:07:19.977570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:07:19.977570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:07:19.977570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:07:19.977570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:07:19.977570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:19.977570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:07:19.977570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:07:19.977570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:07:19.977570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:07:19.977570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:07:19.977570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:19.977570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:19.977570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:07:19.977570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:07:19.977570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:07:19.977570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:07:19.977570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:07:19.977570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:07:19.977895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:19.977895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:19.977895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:19.977895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:07:19.977895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:19.977895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:19.977895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:19.977895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:07:19.977895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:19.977895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:07:19.977895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:07:19.977895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:19.977895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:07:19.977895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:19.977895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:07:19.977895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:07:19.977895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:19.977895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:07:19.977895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:19.977895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:07:19.981714 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:07:19.982190 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:07:19.982190 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:07:19.982190 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:07:19.982550 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:07:19.982870 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:07:19.983091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:19.985530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:07:19.985530 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:19.985530 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:07:19.985530 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:07:19.985530 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:19.985530 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:07:19.985530 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:19.985530 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:07:19.985530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:19.985530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:07:19.985530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:19.985530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:07:19.985530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:07:19.985530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:07:19.985530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:07:19.986084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:19.986084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:19.986084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:19.986084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:07:19.986084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:19.986084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:19.986084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:19.986084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:07:19.986084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:19.986084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:07:19.986084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:07:19.986084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:19.986084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:07:19.986084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:19.986429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:07:19.986429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:07:19.986429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:19.986429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:07:19.986429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:19.986429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:07:19.986429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:19.986429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:07:19.986429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:07:19.986429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:07:19.986429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:07:19.986429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:07:19.986429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:19.986429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:07:19.986779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:07:19.986779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:07:19.986779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:07:19.986779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:07:19.986779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:19.986779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:19.986779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:07:19.986779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:07:19.986779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:07:19.986779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:07:19.986779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:19.986779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:19.986779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:19.986779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:07:19.986779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:19.986779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:19.986779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:19.986779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:07:19.986779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:19.986779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:07:19.987238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:07:19.987238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:19.987238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:07:19.987238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:19.987238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:07:19.987238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:07:19.987238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:19.987238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:07:19.987238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:19.987238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:07:24.270519 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:07:24.271351 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:07:24.271351 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:07:24.272632 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:07:24.275506 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG dbcounter [-] [105581] Writing DB stats neutron:SELECT=240 {{(pid=105581) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 14:07:24.303467 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:07:24.303734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:24.304349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:07:24.304349 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:24.304349 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:07:24.304349 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:07:24.304349 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:24.304349 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:07:24.304349 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:24.304349 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:07:24.305093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:24.305093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:07:24.305093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:24.305093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:07:24.305093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:07:24.305093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:07:24.305093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:07:24.305093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:24.305093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:24.305093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:24.305093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:07:24.305093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:24.305342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:24.305342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:24.305342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:07:24.305342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:24.305342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:07:24.305342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:07:24.305342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:24.305342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:07:24.305342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:24.305342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:07:24.305342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:07:24.305342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:24.305342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:07:24.305584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:24.305584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:07:24.305846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:24.305846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:07:24.305846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:07:24.305846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:07:24.305846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:07:24.305846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:07:24.305846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:24.305846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:07:24.305846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:07:24.305846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:07:24.305846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:07:24.305846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:07:24.305846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:24.305846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:24.305846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:07:24.305846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:07:24.306152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:07:24.306152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:07:24.306152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:24.306152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:24.306152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:24.306152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:07:24.306152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:24.306152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:24.306152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:24.306152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:07:24.306152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:24.306152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:07:24.306152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:07:24.306152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:24.306152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:07:24.306152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:24.306152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:07:24.306152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:07:24.306152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:24.306520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:07:24.306520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:24.306520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:07:24.955186 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:07:24.955878 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:07:24.956030 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:07:24.957073 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:07:24.957427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:24.958432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:07:24.958432 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:24.958432 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:07:24.958432 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:07:24.958432 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:24.958432 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:07:24.958432 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:24.958432 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:07:24.960107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:24.960107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:07:24.960107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:24.960107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:07:24.960107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:07:24.960107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:07:24.960107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:07:24.960107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:24.960107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:24.960107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:24.960107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:07:24.960107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:24.960713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:24.960713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:24.960713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:07:24.960713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:24.960713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:07:24.960713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:07:24.960713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:24.960713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:07:24.960713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:24.960713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:07:24.960713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:07:24.960713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:24.960713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:07:24.961215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:24.961215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:07:24.961932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:24.961932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:07:24.961932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:07:24.961932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:07:24.961932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:07:24.961932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:07:24.961932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:24.961932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:07:24.961932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:07:24.961932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:07:24.961932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:07:24.961932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:07:24.961932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:24.961932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:24.961932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:07:24.961932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:07:24.962989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:07:24.962989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:07:24.962989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:24.962989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:24.962989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:24.962989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:07:24.962989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:24.962989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:24.962989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:24.962989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:07:24.962989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:24.962989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:07:24.962989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:07:24.962989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:24.962989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:07:24.962989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:24.962989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:07:24.962989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:07:24.962989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:24.964150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:07:24.964150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:24.964150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:07:24.966034 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:07:24.966417 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:07:24.966581 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:07:24.966757 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:07:24.967293 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:07:24.967661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:24.968338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:07:24.968338 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:24.968338 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:07:24.968338 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:07:24.968338 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:24.968338 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:07:24.968338 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:24.968338 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:07:24.969157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:24.969157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:07:24.969157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:07:24.969157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:07:24.969157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:24.969157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:07:24.969157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:24.969157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:07:24.969157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:24.969157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:07:24.969157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:24.969157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:07:24.969157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:24.969157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:07:24.969157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:07:24.969685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:24.969685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:07:24.969685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:24.969685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:07:24.969685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:07:24.969685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:24.969685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:07:24.969685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:24.969685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:07:24.970151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:24.970151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:07:24.970151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:24.970151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:07:24.970151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:07:24.970151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:07:24.970151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:07:24.970151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:07:24.970151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:07:24.970151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:07:24.970151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:24.970151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:24.970873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:24.970873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:07:24.970873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:24.970873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:24.970873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:24.970873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:07:24.970873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:24.970873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:07:24.970873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:07:24.970873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:24.970873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:07:24.970873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:24.970873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:07:24.970873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:07:24.971321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:24.971321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:07:24.971321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:24.971321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:07:24.971321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:24.971321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:07:24.971321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:07:24.971321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:07:24.971321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:07:24.971321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:07:24.971321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:24.971321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:07:24.971321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:07:24.971321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:07:24.971688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:07:24.971688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:07:24.971688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:24.971688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:24.971688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:07:24.971688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:07:24.971688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:07:24.971688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:07:24.971688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:07:24.971688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:07:24.971688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:07:24.971688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:24.971688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:24.971688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:24.971688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:07:24.971688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:24.971688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:24.971688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:24.971688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:07:24.972126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:24.972126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:07:24.972126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:07:24.972126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:24.972126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:07:24.972126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:24.972126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:07:24.972126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:07:24.972126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:24.972126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:07:24.972126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:24.972126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:07:24.972633 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:07:24.973069 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:07:24.973069 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:07:24.973069 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:07:24.973476 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:07:24.973864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:24.974678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:07:24.974678 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:24.974678 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:07:24.974678 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:07:24.974678 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:24.974678 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:07:24.974678 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:24.974678 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:07:24.975575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:24.975575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:07:24.975575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:07:24.975575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:07:24.975575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:24.975575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:07:24.975575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:24.975575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:07:24.975575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:24.975575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:07:24.975575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:24.975575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:07:24.975575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:24.975575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:07:24.975575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:07:24.976428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:24.976428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:07:24.976428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:24.976428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:07:24.976428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:07:24.976428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:24.976428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:07:24.976428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:24.976428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:07:24.976428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:24.976428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:07:24.976428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:24.976428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:07:24.976428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:07:24.976967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:07:24.976967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:07:24.976967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:07:24.976967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:07:24.976967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:24.976967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:24.976967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:24.976967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:07:24.976967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:24.976967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:24.976967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:24.976967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:07:24.976967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:24.977281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:07:24.977281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:07:24.977281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:24.977281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:07:24.977281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:24.977281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:07:24.977281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:07:24.977281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:24.977281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:07:24.977281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:24.977281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:07:24.977581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:24.977581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:07:24.977581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:07:24.977581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:07:24.977581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:07:24.977581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:07:24.977581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:24.977581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:07:24.977581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:07:24.977581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:07:24.977581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:07:24.977581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:07:24.977581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:24.977581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:24.977581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:07:24.977581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:07:24.977581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:07:24.977981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:07:24.977981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:07:24.977981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:07:24.977981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:24.977981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:24.977981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:24.977981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:07:24.977981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:24.977981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:24.977981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:24.977981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:07:24.977981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:24.977981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:07:24.977981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:07:24.977981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:24.977981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:07:24.977981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:24.977981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:07:24.977981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:07:24.978453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:24.978453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:07:24.978453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:24.978453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:07:24.982140 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:07:24.982417 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:07:24.982542 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:07:24.982683 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:07:24.983203 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:07:24.983707 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:07:24.983942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:24.984890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:07:24.984890 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:24.984890 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:07:24.984890 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:07:24.984890 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:24.984890 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:07:24.984890 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:24.984890 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:07:24.985767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:24.985767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:07:24.985767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:24.985767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:07:24.985767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:07:24.985767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:07:24.985767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:07:24.985767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:24.985767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:24.985767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:24.985767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:07:24.985767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:24.986435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:24.986435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:24.986435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:07:24.986435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:24.986435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:07:24.986435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:07:24.986435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:24.986435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:07:24.986435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:24.986435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:07:24.986435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:07:24.986435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:24.986435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:07:24.986776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:24.986776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:07:24.986776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:24.986776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:07:24.986776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:07:24.986776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:07:24.986776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:07:24.986776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:07:24.986776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:24.986776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:07:24.986776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:07:24.986776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:07:24.986776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:07:24.986776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:07:24.986776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:24.987134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:24.987134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:07:24.987134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:07:24.987134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:07:24.987134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:07:24.987134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:24.987134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:24.987134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:24.987134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:07:24.987134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:24.987134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:24.987134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:24.987134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:07:24.987134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:24.987134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:07:24.987134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:07:24.987134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:24.987134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:07:24.987134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:24.987622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:07:24.987622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:07:24.987622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:24.987622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:07:24.987622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:24.987622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:07:29.271322 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:07:29.272036 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:07:29.272133 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:07:29.273505 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:07:29.311540 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:07:29.311930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:29.312836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:07:29.312836 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:29.312836 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:07:29.312836 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:07:29.312836 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:29.312836 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:07:29.312836 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:29.312836 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:07:29.313965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:29.313965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:07:29.313965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:29.313965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:07:29.313965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:07:29.313965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:07:29.313965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:07:29.313965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:29.313965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:29.313965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:29.313965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:07:29.313965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:29.314493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:29.314493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:29.314493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:07:29.314493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:29.314493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:07:29.314493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:07:29.314493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:29.314493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:07:29.314493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:29.314493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:07:29.314493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:07:29.314493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:29.314493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:07:29.315009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:29.315009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:07:29.315365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:29.315365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:07:29.315365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:07:29.315365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:07:29.315365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:07:29.315365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:07:29.315365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:29.315365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:07:29.315365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:07:29.315365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:07:29.315365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:07:29.315365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:07:29.315365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:29.315365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:29.315365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:07:29.315365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:07:29.315851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:07:29.315851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:07:29.315851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:29.315851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:29.315851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:29.315851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:07:29.315851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:29.315851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:29.315851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:29.315851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:07:29.315851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:29.315851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:07:29.315851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:07:29.315851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:29.315851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:07:29.315851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:29.315851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:07:29.315851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:07:29.315851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:29.316336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:07:29.316336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:29.316336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:07:29.955763 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:07:29.956330 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:07:29.956605 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:07:29.957460 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:07:29.957793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:29.958721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:07:29.958721 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:29.958721 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:07:29.958721 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:07:29.958721 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:29.958721 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:07:29.958721 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:29.958721 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:07:29.960271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:29.960271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:07:29.960271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:29.960271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:07:29.960271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:07:29.960271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:07:29.960271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:07:29.960271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:29.960271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:29.960271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:29.960271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:07:29.960271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:29.961205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:29.961205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:29.961205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:07:29.961205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:29.961205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:07:29.961205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:07:29.961205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:29.961205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:07:29.961205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:29.961205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:07:29.961205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:07:29.961205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:29.961205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:07:29.962060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:29.962060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:07:29.962060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:29.962060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:07:29.962060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:07:29.962060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:07:29.962060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:07:29.962060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:07:29.962060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:29.962060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:07:29.962060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:07:29.962060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:07:29.962060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:07:29.962060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:07:29.962060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:29.963094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:29.963094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:07:29.963094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:07:29.963094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:07:29.963094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:07:29.963094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:29.963094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:29.963094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:29.963094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:07:29.963094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:29.963094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:29.963094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:29.963094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:07:29.963094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:29.963094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:07:29.963094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:07:29.963094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:29.963094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:07:29.963094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:29.964303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:07:29.964303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:07:29.964303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:29.964303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:07:29.964303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:29.964303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:07:29.966507 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:07:29.966951 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:07:29.967161 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:07:29.967350 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:07:29.967862 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:07:29.968165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:29.968920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:07:29.968920 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:29.968920 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:07:29.968920 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:07:29.968920 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:29.968920 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:07:29.968920 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:29.968920 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:07:29.969665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:29.969665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:07:29.969665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:07:29.969665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:07:29.969665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:29.969665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:07:29.969665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:29.969665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:07:29.969665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:29.969665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:07:29.969665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:29.969665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:07:29.969665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:29.969665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:07:29.969665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:07:29.970275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:29.970275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:07:29.970275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:29.970275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:07:29.970275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:07:29.970275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:29.970275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:07:29.970275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:29.970275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:07:29.970670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:29.970670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:07:29.970670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:29.970670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:07:29.970670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:07:29.970670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:07:29.970670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:07:29.970670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:07:29.970670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:07:29.970670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:07:29.970670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:29.970670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:29.971183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:29.971183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:07:29.971183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:29.971183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:29.971183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:29.971183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:07:29.971183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:29.971183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:07:29.971183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:07:29.971183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:29.971183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:07:29.971183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:29.971183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:07:29.971183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:07:29.971726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:29.971726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:07:29.971726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:29.971726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:07:29.971726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:29.971726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:07:29.971726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:07:29.971726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:07:29.971726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:07:29.971726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:07:29.971726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:29.971726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:07:29.971726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:07:29.971726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:07:29.972289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:07:29.972289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:07:29.972289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:29.972289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:29.972289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:07:29.972289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:07:29.972289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:07:29.972289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:07:29.972289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:07:29.972289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:07:29.972289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:07:29.972289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:29.972289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:29.972289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:29.972289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:07:29.972289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:29.972289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:29.972289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:29.972289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:07:29.973073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:29.973073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:07:29.973073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:07:29.973073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:29.973073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:07:29.973073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:29.973073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:07:29.973073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:07:29.973073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:29.973073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:07:29.973073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:29.973073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:07:29.973073 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:07:29.973632 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:07:29.973632 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:07:29.973632 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:07:29.973779 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:07:29.974153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:29.974848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:07:29.974848 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:29.974848 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:07:29.974848 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:07:29.974848 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:29.974848 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:07:29.974848 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:29.974848 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:07:29.975571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:29.975571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:07:29.975571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:07:29.975571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:07:29.975571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:29.975571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:07:29.975571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:29.975571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:07:29.975571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:29.975571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:07:29.975571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:29.975571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:07:29.975571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:29.975571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:07:29.975571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:07:29.976095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:29.976095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:07:29.976095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:29.976095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:07:29.976095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:07:29.976095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:29.976095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:07:29.976095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:29.976095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:07:29.976538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:29.976538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:07:29.976538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:29.976538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:07:29.976538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:07:29.976538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:07:29.976538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:07:29.976538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:07:29.976538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:07:29.976538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:29.976538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:29.976965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:29.976965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:07:29.976965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:29.976965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:29.976965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:29.976965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:07:29.976965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:29.976965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:07:29.976965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:07:29.976965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:29.976965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:07:29.976965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:29.976965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:07:29.976965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:07:29.977498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:29.977498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:07:29.977498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:29.977498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:07:29.977498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:29.977498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:07:29.977498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:07:29.977498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:07:29.977498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:07:29.977498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:07:29.977498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:29.977498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:07:29.977498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:07:29.977498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:07:29.978024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:07:29.978024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:07:29.978024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:29.978024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:29.978024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:07:29.978024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:07:29.978024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:07:29.978024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:07:29.978024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:07:29.978024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:07:29.978024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:29.978024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:29.978024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:29.978024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:07:29.978024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:29.978024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:29.978024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:29.978024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:07:29.978024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:29.978756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:07:29.978756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:07:29.978756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:29.978756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:07:29.978756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:29.978756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:07:29.978756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:07:29.978756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:29.978756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:07:29.978756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:29.978756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:07:29.982583 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:07:29.982834 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:07:29.982961 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:07:29.983106 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:07:29.983546 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:07:29.983892 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:07:29.984072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:29.984837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:07:29.984837 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:29.984837 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:07:29.984837 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:07:29.984837 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:29.984837 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:07:29.984837 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:29.984837 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:07:29.985686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:29.985686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:07:29.985686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:29.985686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:07:29.985686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:07:29.985686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:07:29.985686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:07:29.985686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:29.985686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:29.985686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:29.985686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:07:29.985686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:29.986183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:29.986183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:29.986183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:07:29.986183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:29.986183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:07:29.986183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:07:29.986183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:29.986183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:07:29.986183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:29.986183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:07:29.986183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:07:29.986183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:29.986183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:07:29.986716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:29.986716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:07:29.986716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:29.986716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:07:29.986716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:07:29.986716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:07:29.986716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:07:29.986716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:07:29.986716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:29.986716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:07:29.986716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:07:29.986716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:07:29.986716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:07:29.986716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:07:29.986716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:29.987704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:29.987704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:07:29.987704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:07:29.987704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:07:29.987704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:07:29.987704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:29.987704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:29.987704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:29.987704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:07:29.987704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:29.987704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:29.987704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:29.987704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:07:29.987704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:29.987704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:07:29.987704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:07:29.987704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:29.987704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:07:29.987704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:29.988798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:07:29.988798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:07:29.988798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:29.988798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:07:29.988798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:29.988798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:07:34.272042 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:07:34.273020 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:07:34.273020 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:07:34.273927 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:07:34.301948 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:07:34.302536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:34.303442 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:07:34.303442 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:34.303442 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:07:34.303442 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:07:34.303442 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:34.303442 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:07:34.303442 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:34.303442 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:07:34.304353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.031 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:34.304353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:07:34.304353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:34.304353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:07:34.304353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:07:34.304353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:07:34.304353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:07:34.304353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:34.304353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:34.304353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:34.304353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:07:34.304353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:34.304912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:34.304912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:34.304912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:07:34.304912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:34.304912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:07:34.304912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:07:34.304912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:34.304912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:07:34.304912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:34.304912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:07:34.304912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:07:34.304912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:34.304912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:07:34.305449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:34.305449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:07:34.305449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:34.305449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:07:34.305449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:07:34.305449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:07:34.305449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:07:34.305449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:07:34.305449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:34.305449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:07:34.305449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:07:34.305449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:07:34.305449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:07:34.305449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:07:34.305449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:34.306060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:34.306060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:07:34.306060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:07:34.306060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:07:34.306060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:07:34.306060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:34.306060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:34.306060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:34.306060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:07:34.306060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:34.306060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:34.306060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:34.306060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:07:34.306060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:34.306060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:07:34.306060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:07:34.306060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:34.306060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:07:34.306060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:34.306821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:07:34.306821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:07:34.306821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:34.306821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:07:34.306821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:34.306821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:07:34.956503 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:07:34.957090 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:07:34.957285 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:07:34.958360 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:07:34.958795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:34.959761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:07:34.959761 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:34.959761 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:07:34.959761 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:07:34.959761 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:34.959761 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:07:34.959761 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:34.959761 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:07:34.961330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:34.961330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:07:34.961330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:34.961330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:07:34.961330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:07:34.961330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:07:34.961330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:07:34.961330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:34.961330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:34.961330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:34.961330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:07:34.961330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:34.961853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:34.961853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:34.961853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:07:34.961853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:34.961853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:07:34.961853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:07:34.961853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:34.961853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:07:34.961853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:34.961853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:07:34.961853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:07:34.961853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:34.961853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:07:34.962374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:34.962374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:07:34.963133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:34.963133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:07:34.963133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:07:34.963133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:07:34.963133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:07:34.963133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:07:34.963133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:34.963133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:07:34.963133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:07:34.963133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:07:34.963133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:07:34.963133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:07:34.963133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:34.963133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:34.963133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:07:34.963133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:07:34.963825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:07:34.963825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:07:34.963825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:34.963825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:34.963825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:34.963825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:07:34.963825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:34.963825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:34.963825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:34.963825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:07:34.963825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:34.963825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:07:34.963825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:07:34.963825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:34.963825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:07:34.963825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:34.963825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:07:34.963825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:07:34.963825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:34.964551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:07:34.964551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:34.964551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:07:34.966956 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:07:34.967329 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:07:34.967522 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:07:34.967800 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:07:34.968307 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:07:34.968620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:34.969287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:07:34.969287 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:34.969287 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:07:34.969287 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:07:34.969287 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:34.969287 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:07:34.969287 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:34.969287 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:07:34.970028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:34.970028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:07:34.970028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:07:34.970028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:07:34.970028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:34.970028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:07:34.970028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:34.970028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:07:34.970028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:34.970028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:07:34.970028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:34.970028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:07:34.970028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:34.970028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:07:34.970028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:07:34.970430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:34.970430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:07:34.970430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:34.970430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:07:34.970430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:07:34.970430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:34.970430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:07:34.970430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:34.970430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:07:34.970878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:34.970878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:07:34.970878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:34.970878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:07:34.970878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:07:34.970878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:07:34.970878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:07:34.970878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:07:34.970878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:07:34.970878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:07:34.970878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:34.970878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:34.971661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:34.971661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:07:34.971661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:34.971661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:34.971661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:34.971661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:07:34.971661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:34.971661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:07:34.971661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:07:34.971661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:34.971661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:07:34.971661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:34.971661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:07:34.971661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:07:34.972291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:34.972291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:07:34.972291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:34.972291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:07:34.972291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:34.972291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:07:34.972291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:07:34.972291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:07:34.972291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:07:34.972291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:07:34.972291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:34.972291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:07:34.972291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:07:34.972291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:07:34.972643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:07:34.972643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:07:34.972643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:34.972643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:34.972643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:07:34.972643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:07:34.972643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:07:34.972643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:07:34.972643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:07:34.972643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:07:34.972643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:07:34.972643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:34.972643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:34.972643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:34.972643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:07:34.972643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:34.972643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:34.972643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:34.972643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:07:34.973094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:34.973094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:07:34.973094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:07:34.973094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:34.973094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:07:34.973094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:34.973094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:07:34.973094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:07:34.973094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:34.973094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:07:34.973094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:34.973094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:07:34.973376 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:07:34.973851 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:07:34.973851 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:07:34.973851 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:07:34.974164 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:07:34.974536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:34.975193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:07:34.975193 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:34.975193 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:07:34.975193 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:07:34.975193 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:34.975193 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:07:34.975193 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:34.975193 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:07:34.976005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:34.976005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:07:34.976005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:07:34.976005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:07:34.976005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:34.976005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:07:34.976005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:34.976005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:07:34.976005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:34.976005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:07:34.976005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:34.976005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:07:34.976005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:34.976005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:07:34.976005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:07:34.976918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:34.976918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:07:34.976918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:34.976918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:07:34.976918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:07:34.976918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:34.976918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:07:34.976918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:34.976918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:07:34.976918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:34.976918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:07:34.976918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:34.976918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:07:34.976918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:07:34.977492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:07:34.977492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:07:34.977492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:07:34.977492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:07:34.977492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:34.977492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:34.977492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:34.977492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:07:34.977492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:34.977492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:34.977492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:34.977492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:07:34.977492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:34.977808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:07:34.977808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:07:34.977808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:34.977808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:07:34.977808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:34.977808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:07:34.977808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:07:34.977808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:34.977808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:07:34.977808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:34.977808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:07:34.978089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:34.978089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:07:34.978089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:07:34.978089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:07:34.978089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:07:34.978089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:07:34.978089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:34.978089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:07:34.978089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:07:34.978089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:07:34.978089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:07:34.978089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:07:34.978089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:34.978089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:34.978089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:07:34.978089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:07:34.978089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:07:34.978507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:07:34.978507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:07:34.978507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:07:34.978507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:34.978507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:34.978507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:34.978507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:07:34.978507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:34.978507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:34.978507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:34.978507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:07:34.978507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:34.978507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:07:34.978507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:07:34.978507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:34.978507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:07:34.978507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:34.978507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:07:34.978507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:07:34.978939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:34.978939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:07:34.978939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:34.978939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:07:34.982989 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:07:34.983252 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:07:34.983358 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:07:34.983548 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:07:34.984046 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:07:34.984397 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:07:34.984609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:34.985288 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:07:34.985288 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:34.985288 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:07:34.985288 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:07:34.985288 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:34.985288 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:07:34.985288 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:34.985288 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:07:34.986143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:34.986143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:07:34.986143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:34.986143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:07:34.986143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:07:34.986143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:07:34.986143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:07:34.986143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:34.986143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:34.986143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:34.986143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:07:34.986143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:34.986460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:34.986460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:34.986460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:07:34.986460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:34.986460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:07:34.986460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:07:34.986460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:34.986460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:07:34.986460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:34.986460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:07:34.986460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:07:34.986460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:34.986460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:07:34.986774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:34.986774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:07:34.987098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:34.987098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:07:34.987098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:07:34.987098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:07:34.987098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:07:34.987098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:07:34.987098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:34.987098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:07:34.987098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:07:34.987098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:07:34.987098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:07:34.987098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:07:34.987098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:34.987098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:34.987098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:07:34.987098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:07:34.987520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:07:34.987520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:07:34.987520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:34.987520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:34.987520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:34.987520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:07:34.987520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:34.987520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:34.987520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:34.987520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:07:34.987520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:34.987520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:07:34.987520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:07:34.987520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:34.987520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:07:34.987520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:34.987520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:07:34.987520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:07:34.987520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:34.987988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:07:34.987988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:34.987988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:07:39.272964 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:07:39.273828 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:07:39.273828 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:07:39.275105 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:07:39.302640 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:07:39.302850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:39.303769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:07:39.303769 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:39.303769 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:07:39.303769 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:07:39.303769 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:39.303769 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:07:39.303769 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:39.303769 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:07:39.304539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.030 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:39.304539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:07:39.304539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:39.304539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:07:39.304539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:07:39.304539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:07:39.304539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:07:39.304539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:39.304539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:39.304539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:39.304539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:07:39.304539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:39.305236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:39.305236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:39.305236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:07:39.305236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:39.305236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:07:39.305236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:07:39.305236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:39.305236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:07:39.305236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:39.305236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:07:39.305236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:07:39.305236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:39.305236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:07:39.306532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:39.306532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:07:39.306532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:39.306532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:07:39.306532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:07:39.306532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:07:39.306532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:07:39.306532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:07:39.306532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:39.306532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:07:39.306532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:07:39.306532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:07:39.306532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:07:39.306532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:07:39.306532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:39.307740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:39.307740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:07:39.307740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:07:39.307740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:07:39.307740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:07:39.307740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:39.307740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:39.307740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:39.307740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:07:39.307740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:39.307740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:39.307740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:39.307740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:07:39.307740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:39.307740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:07:39.307740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:07:39.307740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:39.307740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:07:39.307740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:39.308996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:07:39.308996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:07:39.308996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:39.308996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:07:39.308996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:39.308996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:07:39.957255 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:07:39.957882 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:07:39.958052 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:07:39.959213 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:07:39.959597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:39.960608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:07:39.960608 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:39.960608 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:07:39.960608 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:07:39.960608 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:39.960608 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:07:39.960608 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:39.960608 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:07:39.962200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:39.962200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:07:39.962200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:39.962200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:07:39.962200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:07:39.962200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:07:39.962200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:07:39.962200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:39.962200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:39.962200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:39.962200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:07:39.962200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:39.962774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:39.962774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:39.962774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:07:39.962774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:39.962774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:07:39.962774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:07:39.962774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:39.962774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:07:39.962774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:39.962774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:07:39.962774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:07:39.962774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:39.962774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:07:39.963793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:39.963793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:07:39.963895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:39.963895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:07:39.963895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:07:39.963895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:07:39.963895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:07:39.963895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:07:39.963895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:39.963895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:07:39.963895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:07:39.963895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:07:39.963895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:07:39.963895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:07:39.963895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:39.963895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:39.963895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:07:39.963895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:07:39.964536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:07:39.964536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:07:39.964536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:39.964536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:39.964536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:39.964536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:07:39.964536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:39.964536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:39.964536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:39.964536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:07:39.964536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:39.964536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:07:39.964536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:07:39.964536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:39.964536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:07:39.964536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:39.964536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:07:39.964536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:07:39.964536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:39.965272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:07:39.965272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:39.965272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:07:39.967645 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:07:39.967947 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:07:39.968046 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:07:39.968174 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:07:39.968962 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:07:39.969537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:39.970439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:07:39.970439 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:39.970439 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:07:39.970439 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:07:39.970439 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:39.970439 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:07:39.970439 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:39.970439 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:07:39.971329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:39.971329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:07:39.971329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:07:39.971329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:07:39.971329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:39.971329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:07:39.971329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:39.971329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:07:39.971329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:39.971329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:07:39.971329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:39.971329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:07:39.971329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:39.971329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:07:39.971329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:07:39.971783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:39.971783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:07:39.971783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:39.971783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:07:39.971783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:07:39.971783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:39.971783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:07:39.971783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:39.971783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:07:39.972318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:39.972318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:07:39.972318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:39.972318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:07:39.972318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:07:39.972318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:07:39.972318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:07:39.972318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:07:39.972318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:07:39.972318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:07:39.972318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:39.972318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:39.973218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:39.973218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:07:39.973218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:39.973218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:39.973218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:39.973218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:07:39.973218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:39.973218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:07:39.973218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:07:39.973218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:39.973218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:07:39.973218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:39.973218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:07:39.973218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:07:39.974039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:39.974039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:07:39.974039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:39.974039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:07:39.974039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:39.974039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:07:39.974039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:07:39.974039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:07:39.974039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:07:39.974039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:07:39.974039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:39.974039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:07:39.974039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:07:39.974039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:07:39.974396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:07:39.974396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:07:39.974396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:39.974396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:39.974396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:07:39.974396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:07:39.974396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:07:39.974396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:07:39.974396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:07:39.974396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:07:39.974396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:07:39.974396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:39.974396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:39.974396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:39.974396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:07:39.974396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:39.974396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:39.974396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:39.974396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:07:39.974954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:39.974954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:07:39.974954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:07:39.974954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:39.974954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:07:39.974954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:39.974954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:07:39.974954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:07:39.974954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:39.974954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:07:39.974954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:39.974954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:07:39.974954 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:07:39.974954 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:07:39.975306 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:07:39.975306 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:07:39.975306 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:07:39.975306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:39.975306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:07:39.975306 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:39.975306 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:07:39.975306 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:07:39.975306 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:39.975306 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:07:39.975306 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:39.975306 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:07:39.976211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:39.976211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:07:39.976211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:07:39.976211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:07:39.976211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:39.976211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:07:39.976211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:39.976211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:07:39.976211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:39.976211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:07:39.976211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:39.976211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:07:39.976211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:39.976211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:07:39.976211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:07:39.977185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:39.977185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:07:39.977185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:39.977185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:07:39.977185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:07:39.977185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:39.977185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:07:39.977185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:39.977185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:07:39.977185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:39.977185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:07:39.977185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:39.977185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:07:39.977185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:07:39.978117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:07:39.978117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:07:39.978117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:07:39.978117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:07:39.978117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:39.978117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:39.978117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:39.978117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:07:39.978117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:39.978117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:39.978117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:39.978117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:07:39.978117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:39.978972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:07:39.978972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:07:39.978972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:39.978972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:07:39.978972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:39.978972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:07:39.978972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:07:39.978972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:39.978972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:07:39.978972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:39.978972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:07:39.978972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:39.979377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:07:39.979377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:07:39.979377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:07:39.979377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:07:39.979377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:07:39.979377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:39.979377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:07:39.979377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:07:39.979377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:07:39.979377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:07:39.979377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:07:39.979377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:39.979377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:39.979377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:07:39.979377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:07:39.979377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:07:39.979377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:07:39.979377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:07:39.979377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:07:39.979877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:39.979877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:39.979877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:39.979877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:07:39.979877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:39.979877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:39.979877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:39.979877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:07:39.979877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:39.979877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:07:39.979877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:07:39.979877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:39.979877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:07:39.979877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:39.979877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:07:39.979877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:07:39.979877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:39.979877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:07:39.979877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:39.979877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:07:39.983317 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:07:39.983583 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:07:39.983677 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:07:39.983848 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:07:39.984252 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:07:39.984720 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:07:39.984964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:39.985769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:07:39.985769 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:39.985769 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:07:39.985769 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:07:39.985769 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:39.985769 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:07:39.985769 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:39.985769 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:07:39.986591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:39.986591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:07:39.986591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:39.986591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:07:39.986591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:07:39.986591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:07:39.986591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:07:39.986591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:39.986591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:39.986591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:39.986591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:07:39.986591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:39.987395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:39.987395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:39.987395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:07:39.987395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:39.987395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:07:39.987395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:07:39.987395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:39.987395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:07:39.987395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:39.987395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:07:39.987395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:07:39.987395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:39.987395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:07:39.987818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:39.987818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:07:39.987818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:39.987818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:07:39.987818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:07:39.987818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:07:39.987818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:07:39.987818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:07:39.987818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:39.987818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:07:39.987818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:07:39.987818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:07:39.987818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:07:39.987818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:07:39.987818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:39.988209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:39.988209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:07:39.988209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:07:39.988209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:07:39.988209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:07:39.988209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:39.988209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:39.988209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:39.988209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:07:39.988209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:39.988209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:39.988209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:39.988209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:07:39.988209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:39.988209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:07:39.988209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:07:39.988209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:39.988209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:07:39.988209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:39.988712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:07:39.988712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:07:39.988712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:39.988712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:07:39.988712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:39.988712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:07:44.273730 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:07:44.274312 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:07:44.274595 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:07:44.275837 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:07:44.304720 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:07:44.305103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:44.305928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:07:44.305928 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:44.305928 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:07:44.305928 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:07:44.305928 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:44.305928 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:07:44.305928 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:44.305928 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:07:44.307606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.031 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:44.307606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:07:44.307606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:44.307606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:07:44.307606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:07:44.307606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:07:44.307606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:07:44.307606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:44.307606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:44.307606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:44.307606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:07:44.307606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:44.308159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:44.308159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:44.308159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:07:44.308159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:44.308159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:07:44.308159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:07:44.308159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:44.308159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:07:44.308159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:44.308159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:07:44.308159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:07:44.308159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:44.308159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:07:44.308713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:44.308713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:07:44.309236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:44.309236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:07:44.309236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:07:44.309236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:07:44.309236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:07:44.309236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:07:44.309236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:44.309236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:07:44.309236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:07:44.309236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:07:44.309236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:07:44.309236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:07:44.309236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:44.309236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:44.309236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:07:44.309236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:07:44.309952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:07:44.309952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:07:44.309952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:44.309952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:44.309952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:44.309952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:07:44.309952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:44.309952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:44.309952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:44.309952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:07:44.309952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:44.309952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:07:44.309952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:07:44.309952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:44.309952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:07:44.309952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:44.309952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:07:44.309952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:07:44.309952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:44.311014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:07:44.311014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:44.311014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:07:44.957919 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:07:44.959392 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:07:44.959618 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:07:44.960431 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:07:44.960692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:44.961582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:07:44.961582 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:44.961582 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:07:44.961582 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:07:44.961582 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:44.961582 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:07:44.961582 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:44.961582 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:07:44.963043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:44.963043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:07:44.963043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:44.963043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:07:44.963043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:07:44.963043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:07:44.963043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:07:44.963043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:44.963043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:44.963043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:44.963043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:07:44.963043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:44.963479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:44.963479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:44.963479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:07:44.963479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:44.963479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:07:44.963479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:07:44.963479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:44.963479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:07:44.963479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:44.963479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:07:44.963479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:07:44.963479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:44.963479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:07:44.963891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:44.963891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:07:44.964714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:44.964714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:07:44.964714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:07:44.964714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:07:44.964714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:07:44.964714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:07:44.964714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:44.964714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:07:44.964714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:07:44.964714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:07:44.964714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:07:44.964714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:07:44.964714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:44.964714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:44.964714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:07:44.964714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:07:44.965574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:07:44.965574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:07:44.965574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:44.965574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:44.965574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:44.965574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:07:44.965574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:44.965574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:44.965574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:44.965574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:07:44.965574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:44.965574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:07:44.965574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:07:44.965574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:44.965574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:07:44.965574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:44.965574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:07:44.965574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:07:44.965574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:44.966263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:07:44.966263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:44.966263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:07:44.968049 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:07:44.968227 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:07:44.968355 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:07:44.968480 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:07:44.968704 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:07:44.969037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:44.969544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:07:44.969544 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:44.969544 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:07:44.969544 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:07:44.969544 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:44.969544 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:07:44.969544 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:44.969544 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:07:44.970371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:44.970371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:07:44.970371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:07:44.970371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:07:44.970371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:44.970371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:07:44.970371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:44.970371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:07:44.970371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:44.970371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:07:44.970371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:44.970371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:07:44.970371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:44.970371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:07:44.970371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:07:44.970839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:44.970839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:07:44.970839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:44.970839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:07:44.970839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:07:44.970839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:44.970839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:07:44.970839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:44.970839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:07:44.971469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:44.971469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:07:44.971469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:44.971469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:07:44.971469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:07:44.971469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:07:44.971469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:07:44.971469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:07:44.971469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:07:44.971469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:07:44.971469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:44.971469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:44.971867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:44.971867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:07:44.971867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:44.971867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:44.971867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:44.971867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:07:44.971867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:44.971867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:07:44.971867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:07:44.971867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:44.971867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:07:44.971867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:44.971867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:07:44.971867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:07:44.972301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:44.972301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:07:44.972301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:44.972301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:07:44.972622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:44.972622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:07:44.972622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:07:44.972622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:07:44.972622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:07:44.972622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:07:44.972622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:44.972622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:07:44.972622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:07:44.972622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:07:44.972622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:07:44.972622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:07:44.972622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:44.972622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:44.972622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:07:44.972622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:07:44.973186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:07:44.973186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:07:44.973186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:07:44.973186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:07:44.973186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:07:44.973186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:44.973186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:44.973186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:44.973186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:07:44.973186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:44.973186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:44.973186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:44.973186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:07:44.973186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:44.973186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:07:44.973186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:07:44.973186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:44.973186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:07:44.973720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:44.973720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:07:44.973720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:07:44.973720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:44.973720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:07:44.973720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:44.973720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:07:44.974243 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:07:44.974470 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:07:44.974571 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:07:44.974695 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:07:44.974909 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:07:44.975113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:44.975533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:07:44.975533 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:44.975533 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:07:44.975533 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:07:44.975533 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:44.975533 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:07:44.975533 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:44.975533 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:07:44.976259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:44.976259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:07:44.976259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:07:44.976259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:07:44.976259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:44.976259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:07:44.976259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:44.976259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:07:44.976259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:44.976259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:07:44.976259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:44.976259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:07:44.976259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:44.976259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:07:44.976259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:07:44.976730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:44.976730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:07:44.976730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:44.976730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:07:44.976730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:07:44.976730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:44.976730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:07:44.976730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:44.976730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:07:44.977105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:44.977105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:07:44.977105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:44.977105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:07:44.977105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:07:44.977105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:07:44.977105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:07:44.977105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:07:44.977105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:07:44.977105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:44.977105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:44.977478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:44.977478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:07:44.977478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:44.977478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:44.977478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:44.977478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:07:44.977478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:44.977478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:07:44.977478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:07:44.977478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:44.977478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:07:44.977478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:44.977478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:07:44.977478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:07:44.977866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:44.977866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:07:44.977866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:44.977866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:07:44.978174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:44.978174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:07:44.978174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:07:44.978174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:07:44.978174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:07:44.978174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:07:44.978174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:44.978174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:07:44.978174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:07:44.978174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:07:44.978174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:07:44.978174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:07:44.978174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:44.978174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:44.978174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:07:44.978174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:07:44.978174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:07:44.978758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:07:44.978758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:07:44.978758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:07:44.978758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:44.978758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:44.978758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:44.978758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:07:44.978758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:44.978758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:44.978758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:44.978758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:07:44.978758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:44.978758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:07:44.978758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:07:44.978758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:44.978758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:07:44.978758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:44.978758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:07:44.978758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:07:44.979275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:44.979275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:07:44.979275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:44.979275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:07:44.983666 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:07:44.983897 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:07:44.984005 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:07:44.984154 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:07:44.984369 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:07:44.984596 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:07:44.984694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:44.985116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:07:44.985116 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:44.985116 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:07:44.985116 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:07:44.985116 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:44.985116 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:07:44.985116 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:44.985116 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:07:44.985874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:44.985874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:07:44.985874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:44.985874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:07:44.985874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:07:44.985874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:07:44.985874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:07:44.985874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:44.985874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:44.985874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:44.985874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:07:44.985874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:44.986249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:44.986249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:44.986249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:07:44.986249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:44.986249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:07:44.986249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:07:44.986249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:44.986249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:07:44.986249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:44.986249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:07:44.986249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:07:44.986249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:44.986249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:07:44.986632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:44.986632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:07:44.986867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:44.986867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:07:44.986867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:07:44.986867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:07:44.986867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:07:44.986867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:07:44.986867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:44.986867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:07:44.986867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:07:44.986867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:07:44.986867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:07:44.986867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:07:44.986867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:44.986867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:44.986867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:07:44.986867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:07:44.987382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:07:44.987382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:07:44.987382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:44.987382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:44.987382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:44.987382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:07:44.987382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:44.987382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:44.987382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:44.987382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:07:44.987382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:44.987382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:07:44.987382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:07:44.987382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:44.987382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:07:44.987382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:44.987382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:07:44.987382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:07:44.987382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:44.987947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:07:44.987947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:44.987947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:07:49.274361 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:07:49.275052 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:07:49.275052 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:07:49.275628 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:07:49.309444 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:07:49.309762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:49.310226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:07:49.310226 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:49.310226 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:07:49.310226 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:07:49.310226 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:49.310226 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:07:49.310226 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:49.310226 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:07:49.311274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:49.311274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:07:49.311274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:49.311274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:07:49.311274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:07:49.311274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:07:49.311274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:07:49.311274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:49.311274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:49.311274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:49.311274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:07:49.311274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:49.312186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:49.312186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:49.312186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:07:49.312186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:49.312186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:07:49.312186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:07:49.312186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:49.312186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:07:49.312186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:49.312186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:07:49.312186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:07:49.312186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:49.312186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:07:49.312947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:49.312947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:07:49.312947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:49.312947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:07:49.312947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:07:49.312947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:07:49.312947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:07:49.312947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:07:49.312947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:49.312947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:07:49.312947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:07:49.312947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:07:49.312947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:07:49.312947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:07:49.312947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:49.313686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:49.313686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:07:49.313686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:07:49.313686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:07:49.313686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:07:49.313686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:49.313686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:49.313686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:49.313686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:07:49.313686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:49.313686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:49.313686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:49.313686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:07:49.313686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:49.313686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:07:49.313686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:07:49.313686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:49.313686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:07:49.313686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:49.314213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:07:49.314213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:07:49.314213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:49.314213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:07:49.314213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:49.314213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:07:49.959119 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:07:49.959775 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:07:49.959965 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:07:49.960968 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:07:49.961250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:49.962340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:07:49.962340 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:49.962340 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:07:49.962340 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:07:49.962340 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:49.962340 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:07:49.962340 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:49.962340 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:07:49.963338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:49.963338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:07:49.963338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:49.963338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:07:49.963338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:07:49.963338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:07:49.963338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:07:49.963338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:49.963338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:49.963338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:49.963338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:07:49.963338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:49.963870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:49.963870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:49.963870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:07:49.963870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:49.963870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:07:49.963870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:07:49.963870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:49.963870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:07:49.963870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:49.963870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:07:49.963870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:07:49.963870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:49.963870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:07:49.964393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:49.964393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:07:49.964393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:49.964393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:07:49.964393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:07:49.964393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:07:49.964393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:07:49.964393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:07:49.964393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:49.964393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:07:49.964393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:07:49.964393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:07:49.964393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:07:49.964393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:07:49.964393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:49.965019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:49.965019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:07:49.965019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:07:49.965019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:07:49.965019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:07:49.965019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:49.965019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:49.965019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:49.965019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:07:49.965019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:49.965019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:49.965019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:49.965019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:07:49.965019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:49.965019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:07:49.965019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:07:49.965019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:49.965019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:07:49.965019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:49.965923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:07:49.965923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:07:49.965923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:49.965923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:07:49.965923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:49.965923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:07:49.968466 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:07:49.968757 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:07:49.968869 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:07:49.968989 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:07:49.969360 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:07:49.969585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:49.970116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:07:49.970116 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:49.970116 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:07:49.970116 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:07:49.970116 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:49.970116 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:07:49.970116 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:49.970116 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:07:49.971721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:49.971721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:07:49.971721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:07:49.971721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:07:49.971721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:49.971721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:07:49.971721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:49.971721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:07:49.971721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:49.971721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:07:49.971721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:49.971721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:07:49.971721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:49.971721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:07:49.971721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:07:49.972117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:49.972117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:07:49.972117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:49.972117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:07:49.972117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:07:49.972117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:49.972117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:07:49.972117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:49.972117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:07:49.972576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:49.972576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:07:49.972576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:49.972576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:07:49.972576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:07:49.972576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:07:49.972576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:07:49.972576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:07:49.972576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:07:49.972576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:07:49.972576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:49.972576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:49.973473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:49.973473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:07:49.973473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:49.973473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:49.973473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:49.973473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:07:49.973473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:49.973473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:07:49.973473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:07:49.973473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:49.973473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:07:49.973473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:49.973473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:07:49.973473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:07:49.974162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:49.974162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:07:49.974162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:49.974162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:07:49.974162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:49.974162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:07:49.974162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:07:49.974162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:07:49.974162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:07:49.974162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:07:49.974162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:49.974162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:07:49.974162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:07:49.974162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:07:49.974571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:07:49.974571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:07:49.974571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:49.974571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:49.974571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:07:49.974571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:07:49.974571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:07:49.974571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:07:49.974571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:07:49.974571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:07:49.974571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:07:49.974571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:49.974571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:49.974571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:49.974571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:07:49.974571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:49.974571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:49.974571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:49.974571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:07:49.975015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:49.975015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:07:49.975015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:07:49.975015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:49.975015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:07:49.975015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:49.975015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:07:49.975015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:07:49.975015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:49.975015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:07:49.975015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:49.975015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:07:49.975015 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:07:49.975015 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:07:49.975357 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:07:49.975357 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:07:49.975450 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:07:49.975891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:49.976188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:07:49.976188 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:49.976188 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:07:49.976188 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:07:49.976188 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:49.976188 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:07:49.976188 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:49.976188 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:07:49.976952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:49.976952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:07:49.976952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:07:49.976952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:07:49.976952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:49.976952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:07:49.976952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:49.976952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:07:49.976952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:49.976952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:07:49.976952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:49.976952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:07:49.976952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:49.976952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:07:49.976952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:07:49.977712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:49.977712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:07:49.977712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:49.977712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:07:49.977712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:07:49.977712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:49.977712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:07:49.977712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:49.977712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:07:49.978164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:49.978164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:07:49.978164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:49.978164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:07:49.978164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:07:49.978164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:07:49.978164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:07:49.978164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:07:49.978164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:07:49.978164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:49.978164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:49.978715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:49.978715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:07:49.978715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:49.978715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:49.978715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:49.978715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:07:49.978715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:49.978715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:07:49.978715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:07:49.978715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:49.978715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:07:49.978715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:49.978715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:07:49.978715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:07:49.979463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:49.979463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:07:49.979463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:49.979463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:07:49.979463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:49.979463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:07:49.979463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:07:49.979463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:07:49.979463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:07:49.979463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:07:49.979463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:49.979463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:07:49.979463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:07:49.979463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:07:49.979796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:07:49.979796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:07:49.979796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:49.979796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:49.979796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:07:49.979796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:07:49.979796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:07:49.979796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:07:49.979796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:07:49.979796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:07:49.979796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:49.979796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:49.979796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:49.979796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:07:49.979796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:49.979796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:49.979796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:49.979796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:07:49.979796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:49.980260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:07:49.980260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:07:49.980260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:49.980260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:07:49.980260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:49.980260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:07:49.980260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:07:49.980260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:49.980260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:07:49.980260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:49.980260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:07:49.983926 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:07:49.984174 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:07:49.984289 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:07:49.984443 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:07:49.984756 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:07:49.984992 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:07:49.985122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:49.986291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:07:49.986291 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:49.986291 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:07:49.986291 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:07:49.986291 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:49.986291 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:07:49.986291 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:49.986291 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:07:49.987113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:49.987113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:07:49.987113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:49.987113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:07:49.987113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:07:49.987113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:07:49.987113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:07:49.987113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:49.987113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:49.987113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:49.987113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:07:49.987113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:49.987451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:49.987451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:49.987451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:07:49.987451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:49.987451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:07:49.987451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:07:49.987451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:49.987451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:07:49.987451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:49.987451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:07:49.987451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:07:49.987451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:49.987451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:07:49.987763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:49.987763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:07:49.988085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:49.988085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:07:49.988085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:07:49.988085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:07:49.988085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:07:49.988085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:07:49.988085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:49.988085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:07:49.988085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:07:49.988085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:07:49.988085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:07:49.988085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:07:49.988085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:49.988085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:49.988085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:07:49.988085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:07:49.989275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:07:49.989275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:07:49.989275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:49.989275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:49.989275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:49.989275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:07:49.989275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:49.989275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:49.989275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:49.989275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:07:49.989275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:49.989275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:07:49.989275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:07:49.989275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:49.989275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:07:49.989275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:49.989275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:07:49.989275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:07:49.989275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:49.989757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:07:49.989757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:49.989757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:07:54.275093 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:07:54.275753 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:07:54.275861 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:07:54.277074 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:07:54.307512 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:07:54.307512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:54.307682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:07:54.307682 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:54.307682 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:07:54.307682 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:07:54.307682 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:54.307682 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:07:54.307682 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:54.307682 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:07:54.308437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:54.308437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:07:54.308437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:54.308437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:07:54.308437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:07:54.308437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:07:54.308437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:07:54.308437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:54.308437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:54.308437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:54.308437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:07:54.308437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:54.309015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:54.309015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:54.309015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:07:54.309015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:54.309015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:07:54.309015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:07:54.309015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:54.309015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:07:54.309015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:54.309015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:07:54.309015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:07:54.309015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:54.309015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:07:54.309597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:54.309597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:07:54.309597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:54.309597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:07:54.309597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:07:54.309597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:07:54.309597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:07:54.309597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:07:54.309597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:54.309597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:07:54.309597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:07:54.309597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:07:54.309597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:07:54.309597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:07:54.309597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:54.310248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:54.310248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:07:54.310248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:07:54.310248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:07:54.310248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:07:54.310248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:54.310248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:54.310248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:54.310248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:07:54.310248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:54.310248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:54.310248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:54.310248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:07:54.310248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:54.310248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:07:54.310248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:07:54.310248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:54.310248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:07:54.310248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:54.311270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:07:54.311270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:07:54.311270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:54.311270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:07:54.311270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:54.311270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:07:54.959871 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:07:54.960527 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:07:54.960718 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:07:54.962532 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:07:54.962532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:54.963190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:07:54.963190 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:54.963190 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:07:54.963190 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:07:54.963190 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:54.963190 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:07:54.963190 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:54.963190 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:07:54.964396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:54.964396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:07:54.964396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:54.964396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:07:54.964396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:07:54.964396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:07:54.964396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:07:54.964396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:54.964396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:54.964396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:54.964396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:07:54.964396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:54.964987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:54.964987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:54.964987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:07:54.964987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:54.964987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:07:54.964987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:07:54.964987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:54.964987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:07:54.964987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:54.964987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:07:54.964987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:07:54.964987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:54.964987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:07:54.965653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:54.965653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:07:54.966025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:54.966025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:07:54.966025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:07:54.966025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:07:54.966025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:07:54.966025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:07:54.966025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:54.966025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:07:54.966025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:07:54.966025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:07:54.966025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:07:54.966025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:07:54.966025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:54.966025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:54.966025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:07:54.966025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:07:54.966755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:07:54.966755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:07:54.966755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:54.966755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:54.966755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:54.966755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:07:54.966755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:54.966755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:54.966755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:54.966755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:07:54.966755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:54.966755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:07:54.966755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:07:54.966755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:54.966755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:07:54.966755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:54.966755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:07:54.966755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:07:54.966755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:54.967632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:07:54.967632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:54.967632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:07:54.968818 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:07:54.969139 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:07:54.969327 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:07:54.969565 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:07:54.970085 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:07:54.970249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:54.971037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:07:54.971037 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:54.971037 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:07:54.971037 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:07:54.971037 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:54.971037 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:07:54.971037 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:54.971037 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:07:54.971921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:54.971921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:07:54.971921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:07:54.971921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:07:54.971921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:54.971921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:07:54.971921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:54.971921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:07:54.971921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:54.971921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:07:54.971921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:54.971921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:07:54.971921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:54.971921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:07:54.971921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:07:54.972365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:54.972365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:07:54.972365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:54.972365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:07:54.972365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:07:54.972365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:54.972365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:07:54.972365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:54.972365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:07:54.972777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:54.972777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:07:54.972777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:54.972777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:07:54.972777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:07:54.972777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:07:54.972777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:07:54.972777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:07:54.972777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:07:54.972777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:07:54.972777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:54.972777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:54.973128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:54.973128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:07:54.973128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:54.973128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:54.973128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:54.973128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:07:54.973128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:54.973128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:07:54.973128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:07:54.973128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:54.973128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:07:54.973128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:54.973128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:07:54.973128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:07:54.973540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:54.973540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:07:54.973540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:54.973540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:07:54.973805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:54.973805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:07:54.973805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:07:54.973805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:07:54.973805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:07:54.973805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:07:54.973805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:54.973805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:07:54.973805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:07:54.973805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:07:54.973805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:07:54.973805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:07:54.973805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:54.973805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:54.973805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:07:54.973805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:07:54.974260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:07:54.974260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:07:54.974260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:07:54.974260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:07:54.974260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:07:54.974260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:54.974260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:54.974260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:54.974260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:07:54.974260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:54.974260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:54.974260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:54.974260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:07:54.974260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:54.974260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:07:54.974260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:07:54.974260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:54.974260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:07:54.974769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:54.974769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:07:54.974769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:07:54.974769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:54.974769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:07:54.974769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:54.974769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:07:54.974984 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:07:54.975213 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:07:54.975313 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:07:54.975461 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:07:54.975802 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:07:54.976116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:54.976668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:07:54.976668 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:54.976668 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:07:54.976668 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:07:54.976668 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:54.976668 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:07:54.976668 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:54.976668 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:07:54.977417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:54.977417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:07:54.977417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:07:54.977417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:07:54.977417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:54.977417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:07:54.977417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:54.977417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:07:54.977417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:54.977417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:07:54.977417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:54.977417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:07:54.977417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:54.977417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:07:54.977417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:07:54.977808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:54.977808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:07:54.977808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:54.977808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:07:54.977808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:07:54.977808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:54.977808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:07:54.977808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:54.977808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:07:54.978309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:54.978309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:07:54.978309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:54.978309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:07:54.978309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:07:54.978309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:07:54.978309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:07:54.978309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:07:54.978309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:07:54.978309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:54.978309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:54.978667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:54.978667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:07:54.978667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:54.978667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:54.978667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:54.978667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:07:54.978667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:54.978667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:07:54.978667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:07:54.978667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:54.978667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:07:54.978667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:54.978667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:07:54.978667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:07:54.979067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:54.979067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:07:54.979067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:54.979067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:07:54.979428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:54.979428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:07:54.979428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:07:54.979428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:07:54.979428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:07:54.979428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:07:54.979428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:54.979428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:07:54.979428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:07:54.979428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:07:54.979428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:07:54.979428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:07:54.979428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:54.979428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:54.979428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:07:54.979428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:07:54.979428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:07:54.979893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:07:54.979893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:07:54.979893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:07:54.979893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:54.979893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:54.979893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:54.979893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:07:54.979893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:54.979893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:54.979893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:54.979893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:07:54.979893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:54.979893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:07:54.979893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:07:54.979893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:54.979893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:07:54.979893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:54.979893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:07:54.979893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:07:54.980439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:54.980439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:07:54.980439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:54.980439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:07:54.984263 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:07:54.984524 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:07:54.984629 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:07:54.984781 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:07:54.985067 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:07:54.985277 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:07:54.985383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:54.985895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:07:54.985895 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:54.985895 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:07:54.985895 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:07:54.985895 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:54.985895 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:07:54.985895 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:54.985895 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:07:54.986732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:54.986732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:07:54.986732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:54.986732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:07:54.986732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:07:54.986732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:07:54.986732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:07:54.986732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:54.986732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:54.986732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:54.986732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:07:54.986732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:54.987125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:54.987125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:54.987125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:07:54.987125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:54.987125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:07:54.987125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:07:54.987125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:54.987125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:07:54.987125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:54.987125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:07:54.987125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:07:54.987125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:54.987125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:07:54.987503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:54.987503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:07:54.987720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:54.987720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:07:54.987720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:07:54.987720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:07:54.987720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:07:54.987720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:07:54.987720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:54.987720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:07:54.987720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:07:54.987720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:07:54.987720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:07:54.987720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:07:54.987720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:54.987720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:54.987720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:07:54.987720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:07:54.988140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:07:54.988140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:07:54.988140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:54.988140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:54.988140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:54.988140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:07:54.988140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:54.988140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:54.988140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:54.988140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:07:54.988140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:54.988140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:07:54.988140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:07:54.988140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:54.988140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:07:54.988140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:54.988140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:07:54.988140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:07:54.988140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:54.988710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:07:54.988710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:54.988710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:07:59.275772 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:07:59.276445 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:07:59.276578 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:07:59.277769 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:07:59.306545 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:07:59.306819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:59.307310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:07:59.307310 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:59.307310 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:07:59.307310 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:07:59.307310 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:59.307310 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:07:59.307310 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:59.307310 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:07:59.308057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.031 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:59.308057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:07:59.308057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:59.308057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:07:59.308057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:07:59.308057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:07:59.308057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:07:59.308057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:59.308057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:59.308057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:59.308057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:07:59.308057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:59.308598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:59.308598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:59.308598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:07:59.308598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:59.308598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:07:59.308598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:07:59.308598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:59.308598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:07:59.308598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:59.308598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:07:59.308598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:07:59.308598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:59.308598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:07:59.309113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:59.309113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:07:59.309113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:59.309113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:07:59.309113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:07:59.309113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:07:59.309113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:07:59.309113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:07:59.309113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:59.309113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:07:59.309113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:07:59.309113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:07:59.309113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:07:59.309113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:07:59.309113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:59.309720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:59.309720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:07:59.309720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:07:59.309720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:07:59.309720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:07:59.309720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:59.309720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:59.309720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:59.309720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:07:59.309720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:59.309720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:59.309720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:59.309720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:07:59.309720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:59.309720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:07:59.309720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:07:59.309720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:59.309720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:07:59.309720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:59.310493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:07:59.310493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:07:59.310493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:59.310493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:07:59.310493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:59.310493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:07:59.960461 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:07:59.960990 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:07:59.961151 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:07:59.961829 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:07:59.962054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:59.964786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:07:59.964786 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:59.964786 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:07:59.964786 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:07:59.964786 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:59.964786 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:07:59.964786 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:59.964786 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:07:59.966266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:59.966266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:07:59.966266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:59.966266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:07:59.966266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:07:59.966266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:07:59.966266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:07:59.966266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:59.966266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:59.966266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:59.966266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:07:59.966266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:59.966899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:59.966899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:59.966899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:07:59.966899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:59.966899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:07:59.966899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:07:59.966899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:59.966899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:07:59.966899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:59.966899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:07:59.966899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:07:59.966899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:59.966899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:07:59.967396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:59.967396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:07:59.969373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:59.969373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:07:59.969373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:07:59.969373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:07:59.969373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:07:59.969373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:07:59.969373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:59.969373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:07:59.969373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:07:59.969373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:07:59.969373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:07:59.969373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:07:59.969373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:59.969373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:59.969373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:07:59.969373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:07:59.970134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:07:59.970134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:07:59.970134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:59.970134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:59.970134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:59.970134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:07:59.970134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:59.970134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:59.970134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:59.970134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:07:59.970134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:59.970134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:07:59.970134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:07:59.970134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:59.970134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:07:59.970134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:59.970134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:07:59.970134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:07:59.970134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:59.971031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:07:59.971031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:59.971031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:07:59.971031 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:07:59.971031 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:07:59.971031 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:07:59.971323 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:07:59.971802 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:07:59.971802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:59.972515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:07:59.972515 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:59.972515 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:07:59.972515 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:07:59.972515 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:59.972515 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:07:59.972515 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:59.972515 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:07:59.973267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:59.973267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:07:59.973267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:07:59.973267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:07:59.973267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:59.973267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:07:59.973267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:59.973267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:07:59.973267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:59.973267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:07:59.973267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:59.973267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:07:59.973267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:59.973267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:07:59.973267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:07:59.974246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:59.974246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:07:59.974246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:59.974246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:07:59.974246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:07:59.974246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:59.974246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:07:59.974246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:59.974246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:07:59.974246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:59.974246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:07:59.974246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:59.974246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:07:59.974246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:07:59.975361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:07:59.975361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:07:59.975361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:07:59.975361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:07:59.975361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:07:59.975361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:59.975361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:59.975361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:59.975361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:07:59.975361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:59.975361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:59.975361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:59.975361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:07:59.976226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:59.976226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:07:59.976226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:07:59.976226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:59.976226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:07:59.976226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:59.976226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:07:59.976226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:07:59.976226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:59.976226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:07:59.976226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:59.976226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:07:59.976756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:59.976756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:07:59.976756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:07:59.976756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:07:59.976756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:07:59.976756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:07:59.976756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:59.976756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:07:59.976756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:07:59.976756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:07:59.976756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:07:59.976756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:07:59.976756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:59.976756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:59.976756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:07:59.976756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:07:59.977160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:07:59.977160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:07:59.977160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:07:59.977160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:07:59.977160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:07:59.977160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:59.977160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:59.977160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:59.977160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:07:59.977160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:59.977160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:59.977160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:59.977160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:07:59.977160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:59.977160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:07:59.977160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:07:59.977160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:59.977160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:07:59.977678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:59.977678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:07:59.977678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:07:59.977678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:59.977678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:07:59.977678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:59.977678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:07:59.977678 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:07:59.977678 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:07:59.977678 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:07:59.977950 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:07:59.977950 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:07:59.977950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:59.979141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:07:59.979141 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:59.979141 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:07:59.979141 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:07:59.979141 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:59.979141 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:07:59.979141 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:59.979141 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:07:59.980961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:59.980961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:07:59.980961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:07:59.980961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:07:59.980961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:59.980961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:07:59.980961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:59.980961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:07:59.980961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:59.980961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:07:59.980961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:59.980961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:07:59.980961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:59.980961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:07:59.980961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:07:59.981809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:59.981809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:07:59.981809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:59.981809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:07:59.981809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:07:59.981809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:59.981809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:07:59.981809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:59.981809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:07:59.982849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:59.982849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:07:59.982849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:59.982849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:07:59.982849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:07:59.982849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:07:59.982849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:07:59.982849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:07:59.982849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:07:59.982849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:59.982849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:59.983375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:59.983375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:07:59.983375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:59.983375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:59.983375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:59.983375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:07:59.983375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:59.983375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:07:59.983375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:07:59.983375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:59.983375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:07:59.983375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:59.983375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:07:59.983375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:07:59.984015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:59.984015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:07:59.984015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:59.984015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:07:59.984015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:59.984015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:07:59.984015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:07:59.984015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:07:59.984015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:07:59.984015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:07:59.984015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:59.984015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:07:59.984015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:07:59.984015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:07:59.984356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:07:59.984356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:07:59.984356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:59.984356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:59.984356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:07:59.984356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:07:59.984356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:07:59.984356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:07:59.984356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:07:59.984356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:07:59.984356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:59.984356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:59.984356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:59.984356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:07:59.984356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:59.984356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:59.984356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:59.984356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:07:59.984356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:59.984791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:07:59.984791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:07:59.984791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:59.984791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:07:59.984791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:59.984791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:07:59.984791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:07:59.984791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:59.984791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:07:59.984791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:59.984791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:07:59.986816 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:07:59.987267 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:07:59.987493 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:07:59.987760 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:07:59.988190 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:07:59.988548 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:07:59.988718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:59.989423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:07:59.989423 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:59.989423 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:07:59.989423 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:07:59.989423 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:59.989423 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:07:59.989423 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:59.989423 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:07:59.990882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:59.990882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:07:59.990882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:59.990882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:07:59.990882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:07:59.990882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:07:59.990882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:07:59.990882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:59.990882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:59.990882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:59.990882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:07:59.990882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:59.991123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:07:59.991123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:59.991123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:07:59.991123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:59.991123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:07:59.991123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:07:59.991123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:59.991123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:07:59.991123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:59.991123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:07:59.991123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:07:59.991123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:59.991123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:07:59.991347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:59.991347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:07:59.992535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:59.992535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:07:59.992535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:07:59.992535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:07:59.992535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:07:59.992535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:07:59.992535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:59.992535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:07:59.992535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:07:59.992535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:07:59.992535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:07:59.992535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:07:59.992535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:07:59.992535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:07:59.992535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:07:59.992535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:07:59.992837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:07:59.992837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:07:59.992837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:59.992837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:59.992837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:07:59.992837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:07:59.992837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:07:59.992837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:07:59.992837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:07:59.992837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:07:59.992837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:07:59.992837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:07:59.992837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:07:59.992837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:07:59.992837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:07:59.992837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:07:59.992837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:07:59.992837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:07:59.992837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:07:59.993150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:07:59.993150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:07:59.993150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:08:04.276484 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:08:04.277265 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:08:04.277265 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:08:04.278361 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:08:04.307937 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:08:04.308345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:04.309225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:08:04.309225 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:04.309225 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:08:04.309225 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:08:04.309225 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:04.309225 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:08:04.309225 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:04.309225 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:08:04.309982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:04.309982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:08:04.309982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:04.309982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:08:04.309982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:08:04.309982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:08:04.309982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:08:04.309982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:04.309982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:04.309982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:04.309982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:08:04.309982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:04.310619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:04.310619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:04.310619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:08:04.310619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:04.310619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:08:04.310619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:08:04.310619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:04.310619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:08:04.310619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:04.310619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:08:04.310619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:08:04.310619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:04.310619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:08:04.311211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:04.311211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:08:04.311211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:04.311211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:08:04.311211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:08:04.311211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:08:04.311211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:08:04.311211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:08:04.311211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:04.311211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:08:04.311211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:08:04.311211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:08:04.311211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:08:04.311211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:08:04.311211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:04.311914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:04.311914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:08:04.311914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:08:04.311914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:08:04.311914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:08:04.311914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:04.311914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:04.311914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:04.311914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:08:04.311914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:04.311914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:04.311914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:04.311914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:08:04.311914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:04.311914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:08:04.311914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:08:04.311914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:04.311914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:08:04.311914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:04.312744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:08:04.312744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:08:04.312744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:04.312744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:08:04.312744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:04.312744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:08:04.961213 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:08:04.961812 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:08:04.962006 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:08:04.962895 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:08:04.963235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:04.964156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:08:04.964156 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:04.964156 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:08:04.964156 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:08:04.964156 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:04.964156 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:08:04.964156 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:04.964156 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:08:04.965748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:04.965748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:08:04.965748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:04.965748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:08:04.965748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:08:04.965748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:08:04.965748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:08:04.965748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:04.965748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:04.965748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:04.965748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:08:04.965748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:04.966387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:04.966387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:04.966387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:08:04.966387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:04.966387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:08:04.966387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:08:04.966387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:04.966387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:08:04.966387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:04.966387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:08:04.966387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:08:04.966387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:04.966387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:08:04.967025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:04.967025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:08:04.967025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:04.967025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:08:04.967025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:08:04.967025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:08:04.967025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:08:04.967025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:08:04.967025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:04.967025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:08:04.967025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:08:04.967025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:08:04.967025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:08:04.967025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:08:04.967025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:04.967784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:04.967784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:08:04.967784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:08:04.967784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:08:04.967784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:08:04.967784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:04.967784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:04.967784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:04.967784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:08:04.967784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:04.967784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:04.967784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:04.967784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:08:04.967784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:04.967784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:08:04.967784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:08:04.967784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:04.967784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:08:04.967784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:04.968660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:08:04.968660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:08:04.968660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:04.968660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:08:04.968660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:04.968660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:08:04.970391 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:08:04.970693 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:08:04.970817 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:08:04.970983 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:08:04.971444 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:08:04.971728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:04.972379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:08:04.972379 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:04.972379 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:08:04.972379 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:08:04.972379 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:04.972379 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:08:04.972379 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:04.972379 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:08:04.973111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:04.973111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:08:04.973111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:08:04.973111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:08:04.973111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:04.973111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:08:04.973111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:04.973111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:08:04.973111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:04.973111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:08:04.973111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:04.973111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:08:04.973111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:04.973111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:08:04.973111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:08:04.973923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:04.973923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:08:04.973923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:04.973923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:08:04.973923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:08:04.973923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:04.973923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:08:04.973923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:04.973923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:08:04.973923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:04.973923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:08:04.973923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:04.973923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:08:04.973923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:08:04.974652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:08:04.974652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:08:04.974652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:08:04.974652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:08:04.974652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:08:04.974652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:04.974652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:04.974652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:04.974652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:08:04.974652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:04.974652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:04.974652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:04.974652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:08:04.975331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:04.975331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:08:04.975331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:08:04.975331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:04.975331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:08:04.975331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:04.975331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:08:04.975331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:08:04.975331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:04.975331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:08:04.975331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:04.975331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:08:04.975962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:04.975962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:08:04.975962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:08:04.975962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:08:04.975962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:08:04.975962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:08:04.975962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:04.975962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:08:04.975962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:08:04.975962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:08:04.975962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:08:04.975962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:08:04.975962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:04.975962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:04.975962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:08:04.975962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:08:04.976899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:08:04.976899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:08:04.976899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:08:04.976899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:08:04.976899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:08:04.976899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:04.976899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:04.976899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:04.976899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:08:04.976899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:04.976899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:04.976899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:04.976899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:08:04.976899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:04.976899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:08:04.976899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:08:04.976899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:04.976899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:08:04.977755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:04.977755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:08:04.977755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:08:04.977755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:04.977755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:08:04.977755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:04.977755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:08:04.977755 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:08:04.977755 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:08:04.977755 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:08:04.978081 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:08:04.978081 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:08:04.978081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:04.978081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:08:04.978081 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:04.978081 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:08:04.978081 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:08:04.978081 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:04.978081 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:08:04.978081 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:04.978081 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:08:04.978626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:04.978626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:08:04.978626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:08:04.978626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:08:04.978626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:04.978626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:08:04.978626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:04.978626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:08:04.978626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:04.978626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:08:04.978626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:04.978626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:08:04.978626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:04.978626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:08:04.978626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:08:04.979085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:04.979085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:08:04.979085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:04.979085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:08:04.979085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:08:04.979085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:04.979085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:08:04.979085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:04.979085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:08:04.979472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:04.979472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:08:04.979472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:04.979472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:08:04.979472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:08:04.979472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:08:04.979472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:08:04.979472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:08:04.979472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:08:04.979472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:04.979472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:04.979808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:04.979808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:08:04.979808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:04.979808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:04.979808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:04.979808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:08:04.979808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:04.979808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:08:04.979808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:08:04.979808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:04.979808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:08:04.979808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:04.979808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:08:04.979808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:08:04.980185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:04.980185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:08:04.980185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:04.980185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:08:04.980491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:04.980491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:08:04.980491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:08:04.980491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:08:04.980491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:08:04.980491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:08:04.980491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:04.980491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:08:04.980491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:08:04.980491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:08:04.980491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:08:04.980491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:08:04.980491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:04.980491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:04.980491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:08:04.980491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:08:04.980491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:08:04.980983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:08:04.980983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:08:04.980983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:08:04.980983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:04.980983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:04.980983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:04.980983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:08:04.980983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:04.980983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:04.980983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:04.980983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:08:04.980983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:04.980983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:08:04.980983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:08:04.980983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:04.980983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:08:04.980983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:04.980983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:08:04.980983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:08:04.981527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:04.981527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:08:04.981527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:04.981527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:08:04.987421 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:08:04.987805 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:08:04.987914 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:08:04.988060 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:08:04.988541 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:08:04.988897 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:08:04.989081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:04.989939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:08:04.989939 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:04.989939 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:08:04.989939 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:08:04.989939 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:04.989939 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:08:04.989939 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:04.989939 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:08:04.990823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:04.990823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:08:04.990823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:04.990823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:08:04.990823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:08:04.990823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:08:04.990823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:08:04.990823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:04.990823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:04.990823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:04.990823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:08:04.990823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:04.991155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:04.991155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:04.991155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:08:04.991155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:04.991155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:08:04.991155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:08:04.991155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:04.991155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:08:04.991155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:04.991155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:08:04.991155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:08:04.991155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:04.991155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:08:04.991522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:04.991522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:08:04.991861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:04.991861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:08:04.991861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:08:04.991861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:08:04.991861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:08:04.991861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:08:04.991861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:04.991861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:08:04.991861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:08:04.991861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:08:04.991861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:08:04.991861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:08:04.991861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:04.991861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:04.991861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:08:04.991861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:08:04.993019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:08:04.993019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:08:04.993019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:04.993019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:04.993019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:04.993019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:08:04.993019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:04.993019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:04.993019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:04.993019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:08:04.993019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:04.993019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:08:04.993019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:08:04.993019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:04.993019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:08:04.993019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:04.993019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:08:04.993019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:08:04.993019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:04.993592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:08:04.993592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:04.993592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:08:09.277335 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:08:09.278251 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:08:09.278527 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:08:09.280009 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:08:09.314931 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:08:09.315200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:09.315838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:08:09.315838 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:09.315838 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:08:09.315838 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:08:09.315838 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:09.315838 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:08:09.315838 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:09.315838 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:08:09.316810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:09.316810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:08:09.316810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:09.316810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:08:09.316810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:08:09.316810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:08:09.316810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:08:09.316810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:09.316810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:09.316810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:09.316810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:08:09.316810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:09.317492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:09.317492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:09.317492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:08:09.317492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:09.317492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:08:09.317492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:08:09.317492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:09.317492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:08:09.317492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:09.317492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:08:09.317492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:08:09.317492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:09.317492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:08:09.318091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:09.318091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:08:09.318091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:09.318091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:08:09.318091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:08:09.318091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:08:09.318091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:08:09.318091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:08:09.318091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:09.318091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:08:09.318091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:08:09.318091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:08:09.318091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:08:09.318091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:08:09.318091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:09.318917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:09.318917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:08:09.318917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:08:09.318917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:08:09.318917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:08:09.318917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:09.318917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:09.318917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:09.318917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:08:09.318917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:09.318917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:09.318917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:09.318917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:08:09.318917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:09.318917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:08:09.318917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:08:09.318917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:09.318917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:08:09.318917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:09.319840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:08:09.319840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:08:09.319840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:09.319840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:08:09.319840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:09.319840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:08:09.961965 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:08:09.962560 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:08:09.962749 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:08:09.963770 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:08:09.964095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:09.964976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:08:09.964976 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:09.964976 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:08:09.964976 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:08:09.964976 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:09.964976 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:08:09.964976 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:09.964976 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:08:09.966545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:09.966545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:08:09.966545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:09.966545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:08:09.966545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:08:09.966545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:08:09.966545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:08:09.966545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:09.966545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:09.966545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:09.966545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:08:09.966545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:09.967132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:09.967132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:09.967132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:08:09.967132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:09.967132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:08:09.967132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:08:09.967132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:09.967132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:08:09.967132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:09.967132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:08:09.967132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:08:09.967132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:09.967132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:08:09.967783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:09.967783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:08:09.968158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:09.968158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:08:09.968158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:08:09.968158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:08:09.968158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:08:09.968158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:08:09.968158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:09.968158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:08:09.968158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:08:09.968158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:08:09.968158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:08:09.968158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:08:09.968158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:09.968158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:09.968158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:08:09.968158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:08:09.968938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:08:09.968938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:08:09.968938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:09.968938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:09.968938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:09.968938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:08:09.968938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:09.968938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:09.968938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:09.968938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:08:09.968938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:09.968938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:08:09.968938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:08:09.968938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:09.968938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:08:09.968938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:09.968938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:08:09.968938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:08:09.968938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:09.969824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:08:09.969824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:09.969824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:08:09.970842 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:08:09.971158 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:08:09.971267 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:08:09.971383 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:08:09.971876 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:08:09.972186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:09.974864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:08:09.974864 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:09.974864 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:08:09.974864 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:08:09.974864 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:09.974864 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:08:09.974864 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:09.974864 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:08:09.975689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:09.975689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:08:09.975689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:08:09.975689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:08:09.975689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:09.975689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:08:09.975689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:09.975689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:08:09.975689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:09.975689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:08:09.975689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:09.975689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:08:09.975689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:09.975689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:08:09.975689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:08:09.976067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:09.976067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:08:09.976067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:09.976067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:08:09.976067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:08:09.976067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:09.976067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:08:09.976067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:09.976067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:08:09.976707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:09.976707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:08:09.976707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:09.976707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:08:09.976707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:08:09.976707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:08:09.976707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:08:09.976707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:08:09.976707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:08:09.976707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:08:09.976707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:09.976707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:09.977080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:09.977080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:08:09.977080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:09.977080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:09.977080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:09.977080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:08:09.977080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:09.977080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:08:09.977080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:08:09.977080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:09.977080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:08:09.977080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:09.977080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:08:09.977080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:08:09.977534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:09.977534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:08:09.977534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:09.977534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:08:09.977534 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:08:09.978020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:09.978020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:08:09.978020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:08:09.978020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:08:09.978020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:08:09.978020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:08:09.978020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:09.978020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:08:09.978020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:08:09.978020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:08:09.978020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:08:09.978020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:08:09.978020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:09.978020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:09.978020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:08:09.978020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:08:09.978499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:08:09.978499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:08:09.978499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:08:09.978499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:08:09.978499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:08:09.978499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:09.978499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:09.978499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:09.978499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:08:09.978499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:09.978499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:09.978499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:09.978499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:08:09.978499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:09.978499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:08:09.978499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:08:09.978499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:09.978499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:08:09.978986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:09.978986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:08:09.978986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:08:09.978986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:09.978986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:08:09.978986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:09.978986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:08:09.978986 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:08:09.978986 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:08:09.978986 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:08:09.979285 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:08:09.979611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:09.980371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:08:09.980371 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:09.980371 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:08:09.980371 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:08:09.980371 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:09.980371 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:08:09.980371 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:09.980371 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:08:09.981205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:09.981205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:08:09.981205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:08:09.981205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:08:09.981205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:09.981205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:08:09.981205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:09.981205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:08:09.981205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:09.981205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:08:09.981205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:09.981205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:08:09.981205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:09.981205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:08:09.981205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:08:09.981640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:09.981640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:08:09.981640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:09.981640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:08:09.981640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:08:09.981640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:09.981640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:08:09.981640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:09.981640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:08:09.982075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:09.982075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:08:09.982075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:09.982075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:08:09.982075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:08:09.982075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:08:09.982075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:08:09.982075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:08:09.982075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:08:09.982075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:09.982075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:09.982436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:09.982436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:08:09.982436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:09.982436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:09.982436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:09.982436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:08:09.982436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:09.982436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:08:09.982436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:08:09.982436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:09.982436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:08:09.982436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:09.982436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:08:09.982436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:08:09.982827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:09.982827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:08:09.982827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:09.982827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:08:09.983123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:09.983123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:08:09.983123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:08:09.983123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:08:09.983123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:08:09.983123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:08:09.983123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:09.983123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:08:09.983123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:08:09.983123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:08:09.983123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:08:09.983123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:08:09.983123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:09.983123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:09.983123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:08:09.983123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:08:09.983123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:08:09.983623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:08:09.983623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:08:09.983623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:08:09.983623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:09.983623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:09.983623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:09.983623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:08:09.983623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:09.983623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:09.983623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:09.983623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:08:09.983623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:09.983623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:08:09.983623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:08:09.983623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:09.983623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:08:09.983623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:09.983623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:08:09.983623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:08:09.984166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:09.984166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:08:09.984166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:09.984166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:08:09.987856 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:08:09.988125 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:08:09.988232 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:08:09.988378 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:08:09.988872 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:08:09.989213 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:08:09.989386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:09.990551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:08:09.990551 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:09.990551 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:08:09.990551 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:08:09.990551 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:09.990551 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:08:09.990551 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:09.990551 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:08:09.991335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:09.991335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:08:09.991335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:09.991335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:08:09.991335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:08:09.991335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:08:09.991335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:08:09.991335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:09.991335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:09.991335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:09.991335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:08:09.991335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:09.991620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:09.991620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:09.991620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:08:09.991620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:09.991620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:08:09.991620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:08:09.991620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:09.991620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:08:09.991620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:09.991620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:08:09.991620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:08:09.991620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:09.991620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:08:09.991857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:09.991857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:08:09.992270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:09.992270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:08:09.992270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:08:09.992270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:08:09.992270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:08:09.992270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:08:09.992270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:09.992270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:08:09.992270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:08:09.992270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:08:09.992270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:08:09.992270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:08:09.992270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:09.992270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:09.992270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:08:09.992270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:08:09.992579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:08:09.992579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:08:09.992579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:09.992579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:09.992579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:09.992579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:08:09.992579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:09.992579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:09.992579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:09.992579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:08:09.992579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:09.992579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:08:09.992579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:08:09.992579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:09.992579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:08:09.992579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:09.992579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:08:09.992579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:08:09.992579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:09.992915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:08:09.992915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:09.992915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:08:14.278382 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:08:14.279061 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:08:14.279061 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:08:14.280243 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:08:14.308211 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:08:14.308652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:14.309438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:08:14.309438 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:14.309438 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:08:14.309438 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:08:14.309438 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:14.309438 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:08:14.309438 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:14.309438 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:08:14.310189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.030 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:14.310189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:08:14.310189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:14.310189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:08:14.310189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:08:14.310189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:08:14.310189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:08:14.310189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:14.310189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:14.310189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:14.310189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:08:14.310189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:14.310837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:14.310837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:14.310837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:08:14.310837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:14.310837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:08:14.310837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:08:14.310837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:14.310837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:08:14.310837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:14.310837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:08:14.310837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:08:14.310837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:14.310837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:08:14.311463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:14.311463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:08:14.311463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:14.311463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:08:14.311463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:08:14.311463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:08:14.311463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:08:14.311463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:08:14.311463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:14.311463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:08:14.311463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:08:14.311463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:08:14.311463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:08:14.311463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:08:14.311463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:14.312152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:14.312152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:08:14.312152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:08:14.312152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:08:14.312152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:08:14.312152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:14.312152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:14.312152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:14.312152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:08:14.312152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:14.312152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:14.312152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:14.312152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:08:14.312152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:14.312152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:08:14.312152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:08:14.312152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:14.312152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:08:14.312152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:14.313008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:08:14.313008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:08:14.313008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:14.313008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:08:14.313008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:14.313008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:08:14.962512 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:08:14.963443 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:08:14.963763 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:08:14.965057 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:08:14.965277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:14.965842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:08:14.965842 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:14.965842 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:08:14.965842 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:08:14.965842 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:14.965842 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:08:14.965842 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:14.965842 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:08:14.966741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:14.966741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:08:14.966741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:14.966741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:08:14.966741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:08:14.966741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:08:14.966741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:08:14.966741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:14.966741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:14.966741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:14.966741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:08:14.966741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:14.967115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:14.967115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:14.967115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:08:14.967115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:14.967115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:08:14.967115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:08:14.967115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:14.967115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:08:14.967115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:14.967115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:08:14.967115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:08:14.967115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:14.967115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:08:14.967479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:14.967479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:08:14.967549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:14.967549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:08:14.967549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:08:14.967549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:08:14.967549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:08:14.967549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:08:14.967549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:14.967549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:08:14.967549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:08:14.967549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:08:14.967549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:08:14.967549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:08:14.967549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:14.967549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:14.967549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:08:14.967549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:08:14.967995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:08:14.967995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:08:14.967995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:14.967995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:14.967995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:14.967995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:08:14.967995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:14.967995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:14.967995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:14.967995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:08:14.967995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:14.967995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:08:14.967995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:08:14.967995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:14.967995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:08:14.967995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:14.967995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:08:14.967995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:08:14.967995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:14.968557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:08:14.968557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:14.968557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:08:14.971332 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:08:14.971560 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:08:14.971639 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:08:14.971721 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:08:14.972134 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:08:14.972434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:14.972891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:08:14.972891 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:14.972891 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:08:14.972891 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:08:14.972891 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:14.972891 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:08:14.972891 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:14.972891 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:08:14.975492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:14.975492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:08:14.975492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:08:14.975492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:08:14.975492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:14.975492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:08:14.975492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:14.975492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:08:14.975492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:14.975492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:08:14.975492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:14.975492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:08:14.975492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:14.975492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:08:14.975492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:08:14.976484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:14.976484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:08:14.976484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:14.976484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:08:14.976484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:08:14.976484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:14.976484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:08:14.976484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:14.976484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:08:14.976484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:14.976484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:08:14.976484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:14.976484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:08:14.976484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:08:14.977171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:08:14.977171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:08:14.977171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:08:14.977171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:08:14.977171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:08:14.977171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:14.977171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:14.977171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:14.977171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:08:14.977171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:14.977171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:14.977171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:14.977171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:08:14.977848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:14.977848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:08:14.977848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:08:14.977848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:14.977848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:08:14.977848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:14.977848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:08:14.977848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:08:14.977848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:14.977848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:08:14.977848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:14.977848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:08:14.978440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:14.978440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:08:14.978440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:08:14.978440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:08:14.978440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:08:14.978440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:08:14.978440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:14.978440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:08:14.978440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:08:14.978440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:08:14.978440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:08:14.978440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:08:14.978440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:14.978440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:14.978440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:08:14.978440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:08:14.979218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:08:14.979218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:08:14.979218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:08:14.979218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:08:14.979218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:08:14.979218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:14.979218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:14.979218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:14.979218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:08:14.979218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:14.979218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:14.979218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:14.979218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:08:14.979218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:14.979218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:08:14.979218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:08:14.979218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:14.979218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:08:14.980110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:14.980110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:08:14.980110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:08:14.980110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:14.980110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:08:14.980110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:14.980110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:08:14.980110 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:08:14.980110 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:08:14.980110 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:08:14.980703 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:08:14.980703 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:08:14.980703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:14.980703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:08:14.980703 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:14.980703 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:08:14.980703 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:08:14.980703 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:14.980703 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:08:14.980703 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:14.980703 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:08:14.980703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:14.981122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:08:14.981122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:08:14.981122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:08:14.981122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:14.981122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:08:14.981122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:14.981122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:08:14.981122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:14.981122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:08:14.981122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:14.981122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:08:14.981122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:14.981122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:08:14.981122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:08:14.981122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:14.981122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:08:14.981122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:14.981596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:08:14.981596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:08:14.981596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:14.981596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:08:14.981596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:14.981596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:08:14.981596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:14.981596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:08:14.981596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:14.981596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:08:14.981596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:08:14.981596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:08:14.981596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:08:14.981992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:08:14.981992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:08:14.981992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:14.981992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:14.981992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:14.981992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:08:14.981992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:14.981992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:14.981992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:14.981992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:08:14.981992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:14.981992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:08:14.981992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:08:14.982554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:14.982554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:08:14.982554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:14.982554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:08:14.982554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:08:14.982554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:14.982554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:08:14.982554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:14.982554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:08:14.982554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:14.982554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:08:14.982554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:08:14.982893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:08:14.982893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:08:14.982893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:08:14.982893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:14.982893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:08:14.982893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:08:14.982893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:08:14.982893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:08:14.982893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:08:14.982893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:14.982893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:14.982893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:08:14.982893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:08:14.982893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:08:14.982893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:08:14.982893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:08:14.982893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:08:14.982893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:14.982893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:14.983634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:14.983634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:08:14.983634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:14.983634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:14.983634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:14.983634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:08:14.983634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:14.983634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:08:14.983634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:08:14.983634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:14.983634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:08:14.983634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:14.983634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:08:14.983634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:08:14.983634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:14.983634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:08:14.983634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:14.983634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:08:14.988276 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:08:14.988544 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:08:14.988611 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:08:14.988726 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:08:14.988897 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:08:14.989056 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:08:14.989128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:14.989464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:08:14.989464 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:14.989464 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:08:14.989464 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:08:14.989464 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:14.989464 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:08:14.989464 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:14.989464 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:08:14.989996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:14.989996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:08:14.989996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:14.989996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:08:14.989996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:08:14.989996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:08:14.989996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:08:14.989996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:14.989996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:14.989996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:14.989996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:08:14.989996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:14.990352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:14.990352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:14.990352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:08:14.990352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:14.990352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:08:14.990352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:08:14.990352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:14.990352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:08:14.990352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:14.990352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:08:14.990352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:08:14.990352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:14.990352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:08:14.990709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:14.990709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:08:14.990709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:14.990709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:08:14.990709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:08:14.990709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:08:14.990709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:08:14.990709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:08:14.990709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:14.990709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:08:14.990709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:08:14.990709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:08:14.990709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:08:14.990709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:08:14.990709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:14.991191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:14.991191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:08:14.991191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:08:14.991191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:08:14.991191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:08:14.991191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:14.991191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:14.991191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:14.991191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:08:14.991191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:14.991191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:14.991191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:14.991191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:08:14.991191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:14.991191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:08:14.991191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:08:14.991191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:14.991191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:08:14.991191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:14.991816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:08:14.991816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:08:14.991816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:14.991816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:08:14.991816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:14.991816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:08:18.530371 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(uuid=07469b1c-e1c5-4466-a578-89865b33cc50, enabled=[], external_ids={'neutron:is_ext_gw': 'True', 'neutron:network_name': 'neutron-a7d094cd-23f3-4e0c-9e88-24be6ee17b2c', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-09e042cf-39a9-42c8-b683-b29ffc42b049', 'neutron:subnet_ids': '2f0821e3-c6a2-49d6-80ee-3480a17ba8d2 644269b7-6b84-4d8a-ad28-d59a7c6b7c5e'}, gateway_chassis=[], ha_chassis_group=9ded45e1-2903-453f-99bd-84e8906cebb6, ipv6_prefix=[], ipv6_ra_configs={}, mac='fa:16:3e:c7:86:37', name='lrp-71ded037-e812-40c0-8d8f-70c141c74323', networks=['172.24.5.175/24', '2001:db8::d8/64'], options={'gateway_mtu': '1430', 'reside-on-redirect-chassis': 'true'}, peer=[], status={}) old= {{(pid=105581) matches /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}}
May 09 14:08:19.279015 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:08:19.279585 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:08:19.279738 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:08:19.280584 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:08:19.309381 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:08:19.309892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:19.310940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:08:19.310940 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:19.310940 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:08:19.310940 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:08:19.310940 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:19.310940 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:08:19.310940 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:19.310940 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:08:19.312589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.031 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:19.312589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:08:19.312589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:19.312589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:08:19.312589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:08:19.312589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:08:19.312589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:08:19.312589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:19.312589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:19.312589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:19.312589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:08:19.312589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:19.313182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:19.313182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:19.313182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:08:19.313182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:19.313182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:08:19.313182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:08:19.313182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:19.313182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:08:19.313182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:19.313182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:08:19.313182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:08:19.313182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:19.313182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:08:19.313786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:19.313786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:08:19.314382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:19.314382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:08:19.314382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:08:19.314382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:08:19.314382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:08:19.314382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:08:19.314382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:19.314382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:08:19.314382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:08:19.314382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:08:19.314382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:08:19.314382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:08:19.314382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:19.314382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:19.314382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:08:19.314382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:08:19.315156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:08:19.315156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:08:19.315156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:19.315156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:19.315156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:19.315156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:08:19.315156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:19.315156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:19.315156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:19.315156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:08:19.315156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:19.315156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:08:19.315156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:08:19.315156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:19.315156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:08:19.315156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:19.315156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:08:19.315156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:08:19.315156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:19.316040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:08:19.316040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:19.316040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:08:19.963576 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:08:19.964461 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:08:19.964461 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:08:19.965879 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:08:19.966214 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:08:19.966635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:19.967521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:08:19.967521 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:19.967521 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:08:19.967521 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:08:19.967521 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:19.967521 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:08:19.967521 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:19.967521 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:08:19.968750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:19.968750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:08:19.968750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:19.968750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:08:19.968750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:08:19.968750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:08:19.968750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:08:19.968750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:19.968750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:19.968750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:19.968750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:08:19.968750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:19.969294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:19.969294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:19.969294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:08:19.969294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:19.969294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:08:19.969294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:08:19.969294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:19.969294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:08:19.969294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:19.969294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:08:19.969294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:08:19.969294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:19.969294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:08:19.969912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:19.969912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:08:19.970629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:19.970629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:08:19.970629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:08:19.970629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:08:19.970629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:08:19.970629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:08:19.970629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:19.970629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:08:19.970629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:08:19.970629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:08:19.970629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:08:19.970629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:08:19.970629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:19.970629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:19.970629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:08:19.970629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:08:19.971598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:08:19.971598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:08:19.971598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:19.971598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:19.971598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:19.971598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:08:19.971598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:19.971598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:19.971598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:19.971598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:08:19.971598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:19.971598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:08:19.971598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:08:19.971598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:19.971598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:08:19.971598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:19.971598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:08:19.971598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:08:19.971598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:19.972673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:08:19.972673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:19.972673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:08:19.972673 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:08:19.972673 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:08:19.972673 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:08:19.972673 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:08:19.973540 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:08:19.974097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:19.975223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:08:19.975223 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:19.975223 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:08:19.975223 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:08:19.975223 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:19.975223 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:08:19.975223 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:19.975223 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:08:19.976163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:19.976163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:08:19.976163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:08:19.976163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:08:19.976163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:19.976163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:08:19.976163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:19.976163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:08:19.976163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:19.976163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:08:19.976163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:19.976163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:08:19.976163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:19.976163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:08:19.976163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:08:19.976832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:19.976832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:08:19.976832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:19.976832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:08:19.976832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:08:19.976832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:19.976832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:08:19.976832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:19.976832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:08:19.977337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:19.977337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:08:19.977337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:19.977337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:08:19.977337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:08:19.977337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:08:19.977337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:08:19.977337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:08:19.977337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:08:19.977337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:08:19.977337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:19.977337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:19.977919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:19.977919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:08:19.977919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:19.977919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:19.977919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:19.977919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:08:19.977919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:19.977919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:08:19.977919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:08:19.977919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:19.977919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:08:19.977919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:19.977919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:08:19.977919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:08:19.978532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:19.978532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:08:19.978532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:19.978532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:08:19.979229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:19.979229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:08:19.979229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:08:19.979229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:08:19.979229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:08:19.979229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:08:19.979229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:19.979229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:08:19.979229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:08:19.979229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:08:19.979229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:08:19.979229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:08:19.979229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:19.979229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:19.979229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:08:19.979229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:08:19.980149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:08:19.980149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:08:19.980149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:08:19.980149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:08:19.980149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:08:19.980149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:19.980149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:19.980149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:19.980149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:08:19.980149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:19.980149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:19.980149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:19.980149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:08:19.980149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:19.980149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:08:19.980149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:08:19.980149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:19.980149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:08:19.981183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:19.981183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:08:19.981183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:08:19.981183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:19.981183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:08:19.981183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:19.981183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:08:19.981183 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:08:19.981183 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:08:19.981183 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:08:19.981536 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:08:19.981536 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:08:19.981536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:19.981912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:08:19.981912 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:19.981912 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:08:19.981912 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:08:19.981912 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:19.981912 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:08:19.981912 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:19.981912 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:08:19.982900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:19.982900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:08:19.982900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:08:19.982900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:08:19.982900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:19.982900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:08:19.982900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:19.982900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:08:19.982900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:19.982900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:08:19.982900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:19.982900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:08:19.982900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:19.982900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:08:19.982900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:08:19.983547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:19.983547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:08:19.983547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:19.983547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:08:19.983547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:08:19.983547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:19.983547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:08:19.983547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:19.983547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:08:19.984124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:19.984124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:08:19.984124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:19.984124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:08:19.984124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:08:19.984124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:08:19.984124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:08:19.984124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:08:19.984124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:08:19.984124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:19.984124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:19.984691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:19.984691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:08:19.984691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:19.984691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:19.984691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:19.984691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:08:19.984691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:19.984691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:08:19.984691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:08:19.984691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:19.984691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:08:19.984691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:19.984691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:08:19.984691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:08:19.985246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:19.985246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:08:19.985246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:19.985246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:08:19.985728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:19.985728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:08:19.985728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:08:19.985728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:08:19.985728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:08:19.985728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:08:19.985728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:19.985728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:08:19.985728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:08:19.985728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:08:19.985728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:08:19.985728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:08:19.985728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:19.985728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:19.985728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:08:19.985728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:08:19.985728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:08:19.986268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:08:19.986268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:08:19.986268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:08:19.986268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:19.986268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:19.986268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:19.986268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:08:19.986268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:19.986268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:19.986268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:19.986268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:08:19.986268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:19.986268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:08:19.986268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:08:19.986268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:19.986268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:08:19.986268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:19.986268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:08:19.986268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:08:19.986721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:19.986721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:08:19.986721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:19.986721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:08:19.988578 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:08:19.988806 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:08:19.988912 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:08:19.989056 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:08:19.989421 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:08:19.989657 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:08:19.989883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:19.990464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:08:19.990464 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:19.990464 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:08:19.990464 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:08:19.990464 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:19.990464 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:08:19.990464 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:19.990464 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:08:19.991272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:19.991272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:08:19.991272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:19.991272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:08:19.991272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:08:19.991272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:08:19.991272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:08:19.991272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:19.991272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:19.991272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:19.991272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:08:19.991272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:19.991709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:19.991709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:19.991709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:08:19.991709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:19.991709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:08:19.991709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:08:19.991709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:19.991709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:08:19.991709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:19.991709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:08:19.991709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:08:19.991709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:19.991709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:08:19.992117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:19.992117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:08:19.992260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:19.992260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:08:19.992260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:08:19.992260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:08:19.992260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:08:19.992260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:08:19.992260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:19.992260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:08:19.992260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:08:19.992260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:08:19.992260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:08:19.992260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:08:19.992260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:19.992260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:19.992260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:08:19.992260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:08:19.992661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:08:19.992661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:08:19.992661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:19.992661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:19.992661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:19.992661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:08:19.992661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:19.992661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:19.992661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:19.992661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:08:19.992661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:19.992661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:08:19.992661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:08:19.992661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:19.992661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:08:19.992661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:19.992661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:08:19.992661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:08:19.992661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:19.993017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:08:19.993017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:19.993017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:08:20.828200 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(uuid=bbb1127f-bc46-4204-8f67-25a88d511cb6, enabled=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-632a2ddb-5634-4bb3-aad0-f05d97abd45e', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-09e042cf-39a9-42c8-b683-b29ffc42b049', 'neutron:subnet_ids': '32b523fe-135f-4f9e-a98b-fc0537fcdb35'}, gateway_chassis=[], ha_chassis_group=[], ipv6_prefix=[], ipv6_ra_configs={}, mac='fa:16:3e:48:7c:a3', name='lrp-ea1db480-42b0-4ab5-9c1c-8de2b79b7d3a', networks=['192.168.24.1/24'], options={}, peer=[], status={}) old= {{(pid=105581) matches /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}}
May 09 14:08:20.858329 np0000173183 neutron-ovn-maintenance-worker[105581]: INFO neutron.common.ovn.utils [-] HA Chassis Group neutron-632a2ddb-5634-4bb3-aad0-f05d97abd45e synchronized; highest priority chassis 4524ca43-0613-40ef-9118-5c14197adb05
May 09 14:08:20.858955 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): HAChassisGroupAddCommand(_result=None, name=neutron-632a2ddb-5634-4bb3-aad0-f05d97abd45e, may_exist=True, columns={'external_ids': {'neutron:availability_zone_hints': '', 'neutron:network_id': '632a2ddb-5634-4bb3-aad0-f05d97abd45e', 'neutron:router_id': '09e042cf-39a9-42c8-b683-b29ffc42b049'}}) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:08:20.859507 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): HAChassisGroupAddChassisCommand(_result=None, hcg=neutron-632a2ddb-5634-4bb3-aad0-f05d97abd45e, chassis=4524ca43-0613-40ef-9118-5c14197adb05, priority=1, columns={}) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:08:20.859883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:20.860634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
May 09 14:08:20.860634 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:20.860634 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:08:20.860634 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:08:20.860634 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:20.860634 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:08:20.860634 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:20.860634 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:08:20.861488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event [-] Unexpected exception in notify_loop: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:20.861488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event Traceback (most recent call last):
May 09 14:08:20.861488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/event.py", line 185, in notify_loop
May 09 14:08:20.861488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event     match.run(event, row, updates)
May 09 14:08:20.861488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/services/ovn_l3/ovsdb_monitor.py", line 86, in run
May 09 14:08:20.861488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event     self.l3_plugin._ovn_client.link_network_ha_chassis_group(
May 09 14:08:20.861488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2416, in link_network_ha_chassis_group
May 09 14:08:20.861488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:08:20.861488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:20.861488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event     next(self.gen)
May 09 14:08:20.861488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:20.861488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event     with super().transaction(*args, **kwargs) as t:
May 09 14:08:20.861488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:20.861488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event     next(self.gen)
May 09 14:08:20.861488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:20.861488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event     with self.create_transaction(
May 09 14:08:20.861488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:20.861488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event     self.result = self.commit()
May 09 14:08:20.861488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event                   ^^^^^^^^^^^^^
May 09 14:08:20.861943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:20.861943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event     raise result.ex
May 09 14:08:20.861943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:20.861943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event     txn.results.put(txn.do_commit())
May 09 14:08:20.861943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event                     ^^^^^^^^^^^^^^^
May 09 14:08:20.861943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:20.861943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event     raise RuntimeError(msg)
May 09 14:08:20.861943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:20.861943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.event 
May 09 14:08:24.279700 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:08:24.280299 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:08:24.280529 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:08:24.281773 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:08:24.284497 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG dbcounter [-] [105581] Writing DB stats neutron:SELECT=257 {{(pid=105581) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 14:08:24.311174 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:08:24.311608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:24.312564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:08:24.312564 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:24.312564 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:08:24.312564 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:08:24.312564 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:24.312564 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:08:24.312564 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:24.312564 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:08:24.313353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:24.313353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:08:24.313353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:24.313353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:08:24.313353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:08:24.313353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:08:24.313353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:08:24.313353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:24.313353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:24.313353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:24.313353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:08:24.313353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:24.313644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:24.313644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:24.313644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:08:24.313644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:24.313644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:08:24.313644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:08:24.313644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:24.313644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:08:24.313644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:24.313644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:08:24.313644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:08:24.313644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:24.313644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:08:24.313896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:24.313896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:08:24.314121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:24.314121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:08:24.314121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:08:24.314121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:08:24.314121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:08:24.314121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:08:24.314121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:24.314121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:08:24.314121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:08:24.314121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:08:24.314121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:08:24.314121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:08:24.314121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:24.314121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:24.314121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:08:24.314121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:08:24.314471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:08:24.314471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:08:24.314471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:24.314471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:24.314471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:24.314471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:08:24.314471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:24.314471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:24.314471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:24.314471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:08:24.314471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:24.314471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:08:24.314471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:08:24.314471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:24.314471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:08:24.314471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:24.314471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:08:24.314471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:08:24.314471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:24.314818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:08:24.314818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:24.314818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:08:24.964309 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:08:24.964862 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:08:24.965070 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:08:24.966259 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:08:24.966640 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:08:24.966860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:24.967737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:08:24.967737 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:24.967737 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:08:24.967737 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:08:24.967737 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:24.967737 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:08:24.967737 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:24.967737 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:08:24.969296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:24.969296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:08:24.969296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:24.969296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:08:24.969296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:08:24.969296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:08:24.969296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:08:24.969296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:24.969296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:24.969296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:24.969296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:08:24.969296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:24.969844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:24.969844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:24.969844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:08:24.969844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:24.969844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:08:24.969844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:08:24.969844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:24.969844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:08:24.969844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:24.969844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:08:24.969844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:08:24.969844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:24.969844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:08:24.970461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:24.970461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:08:24.971048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:24.971048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:08:24.971048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:08:24.971048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:08:24.971048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:08:24.971048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:08:24.971048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:24.971048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:08:24.971048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:08:24.971048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:08:24.971048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:08:24.971048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:08:24.971048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:24.971048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:24.971048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:08:24.971048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:08:24.971799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:08:24.971799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:08:24.971799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:24.971799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:24.971799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:24.971799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:08:24.971799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:24.971799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:24.971799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:24.971799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:08:24.971799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:24.971799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:08:24.971799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:08:24.971799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:24.971799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:08:24.971799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:24.971799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:08:24.971799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:08:24.971799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:24.972725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:08:24.972725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:24.972725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:08:24.972725 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:08:24.972725 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:08:24.972962 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:08:24.972962 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:08:24.973443 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:08:24.973733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:24.974416 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:08:24.974416 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:24.974416 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:08:24.974416 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:08:24.974416 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:24.974416 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:08:24.974416 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:24.974416 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:08:24.975608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:24.975608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:08:24.975608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:08:24.975608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:08:24.975608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:24.975608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:08:24.975608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:24.975608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:08:24.975608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:24.975608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:08:24.975608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:24.975608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:08:24.975608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:24.975608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:08:24.975608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:08:24.976019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:24.976019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:08:24.976019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:24.976019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:08:24.976019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:08:24.976019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:24.976019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:08:24.976019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:24.976019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:08:24.976551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:24.976551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:08:24.976551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:24.976551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:08:24.976551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:08:24.976551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:08:24.976551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:08:24.976551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:08:24.976551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:08:24.976551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:08:24.976551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:24.976551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:24.976905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:24.976905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:08:24.976905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:24.976905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:24.976905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:24.976905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:08:24.976905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:24.976905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:08:24.976905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:08:24.976905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:24.976905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:08:24.976905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:24.976905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:08:24.976905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:08:24.977289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:24.977289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:08:24.977289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:24.977289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:08:24.977537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:24.977537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:08:24.977537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:08:24.977537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:08:24.977537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:08:24.977537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:08:24.977537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:24.977537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:08:24.977537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:08:24.977537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:08:24.977537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:08:24.977537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:08:24.977537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:24.977537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:24.977537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:08:24.977537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:08:24.978028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:08:24.978028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:08:24.978028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:08:24.978028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:08:24.978028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:08:24.978028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:24.978028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:24.978028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:24.978028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:08:24.978028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:24.978028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:24.978028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:24.978028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:08:24.978028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:24.978028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:08:24.978028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:08:24.978028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:24.978028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:08:24.978541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:24.978541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:08:24.978541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:08:24.978541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:24.978541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:08:24.978541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:24.978541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:08:24.979576 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:08:24.979843 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:08:24.979952 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:08:24.980073 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:08:24.980554 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:08:24.980887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:24.981582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:08:24.981582 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:24.981582 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:08:24.981582 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:08:24.981582 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:24.981582 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:08:24.981582 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:24.981582 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:08:24.982272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:24.982272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:08:24.982272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:08:24.982272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:08:24.982272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:24.982272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:08:24.982272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:24.982272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:08:24.982272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:24.982272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:08:24.982272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:24.982272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:08:24.982272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:24.982272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:08:24.982272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:08:24.982759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:24.982759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:08:24.982759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:24.982759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:08:24.982759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:08:24.982759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:24.982759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:08:24.982759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:24.982759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:08:24.983137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:24.983137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:08:24.983137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:24.983137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:08:24.983137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:08:24.983137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:08:24.983137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:08:24.983137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:08:24.983137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:08:24.983137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:24.983137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:24.983496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:24.983496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:08:24.983496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:24.983496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:24.983496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:24.983496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:08:24.983496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:24.983496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:08:24.983496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:08:24.983496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:24.983496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:08:24.983496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:24.983496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:08:24.983496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:08:24.983915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:24.983915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:08:24.983915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:24.983915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:08:24.984255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:24.984255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:08:24.984255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:08:24.984255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:08:24.984255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:08:24.984255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:08:24.984255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:24.984255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:08:24.984255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:08:24.984255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:08:24.984255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:08:24.984255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:08:24.984255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:24.984255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:24.984255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:08:24.984255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:08:24.984255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:08:24.984801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:08:24.984801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:08:24.984801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:08:24.984801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:24.984801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:24.984801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:24.984801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:08:24.984801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:24.984801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:24.984801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:24.984801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:08:24.984801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:24.984801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:08:24.984801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:08:24.984801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:24.984801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:08:24.984801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:24.984801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:08:24.984801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:08:24.985336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:24.985336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:08:24.985336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:24.985336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:08:24.989027 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:08:24.989292 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:08:24.989391 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:08:24.989590 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:08:24.989964 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:08:24.990293 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:08:24.990537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:24.991179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:08:24.991179 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:24.991179 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:08:24.991179 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:08:24.991179 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:24.991179 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:08:24.991179 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:24.991179 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:08:24.992003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:24.992003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:08:24.992003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:24.992003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:08:24.992003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:08:24.992003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:08:24.992003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:08:24.992003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:24.992003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:24.992003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:24.992003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:08:24.992003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:24.992434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:24.992434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:24.992434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:08:24.992434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:24.992434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:08:24.992434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:08:24.992434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:24.992434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:08:24.992434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:24.992434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:08:24.992434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:08:24.992434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:24.992434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:08:24.992791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:24.992791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:08:24.993013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:24.993013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:08:24.993013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:08:24.993013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:08:24.993013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:08:24.993013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:08:24.993013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:24.993013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:08:24.993013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:08:24.993013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:08:24.993013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:08:24.993013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:08:24.993013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:24.993013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:24.993013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:08:24.993013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:08:24.993509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:08:24.993509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:08:24.993509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:24.993509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:24.993509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:24.993509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:08:24.993509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:24.993509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:24.993509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:24.993509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:08:24.993509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:24.993509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:08:24.993509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:08:24.993509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:24.993509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:08:24.993509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:24.993509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:08:24.993509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:08:24.993509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:24.994034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:08:24.994034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:24.994034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:08:29.280448 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:08:29.281256 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:08:29.281329 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:08:29.282630 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:08:29.317050 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:08:29.317318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:29.318179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:08:29.318179 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:29.318179 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:08:29.318179 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:08:29.318179 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:29.318179 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:08:29.318179 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:29.318179 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:08:29.318918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:29.318918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:08:29.318918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:29.318918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:08:29.318918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:08:29.318918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:08:29.318918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:08:29.318918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:29.318918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:29.318918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:29.318918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:08:29.318918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:29.319601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:29.319601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:29.319601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:08:29.319601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:29.319601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:08:29.319601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:08:29.319601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:29.319601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:08:29.319601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:29.319601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:08:29.319601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:08:29.319601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:29.319601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:08:29.320231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:29.320231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:08:29.320231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:29.320231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:08:29.320231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:08:29.320231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:08:29.320231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:08:29.320231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:08:29.320231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:29.320231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:08:29.320231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:08:29.320231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:08:29.320231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:08:29.320231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:08:29.320231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:29.321249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:29.321249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:08:29.321249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:08:29.321249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:08:29.321249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:08:29.321249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:29.321249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:29.321249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:29.321249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:08:29.321249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:29.321249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:29.321249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:29.321249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:08:29.321249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:29.321249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:08:29.321249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:08:29.321249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:29.321249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:08:29.321249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:29.322295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:08:29.322295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:08:29.322295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:29.322295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:08:29.322295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:29.322295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:08:29.964870 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:08:29.965549 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:08:29.965775 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:08:29.966962 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:08:29.967294 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:08:29.967591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:29.968544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:08:29.968544 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:29.968544 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:08:29.968544 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:08:29.968544 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:29.968544 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:08:29.968544 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:29.968544 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:08:29.970103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:29.970103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:08:29.970103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:29.970103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:08:29.970103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:08:29.970103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:08:29.970103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:08:29.970103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:29.970103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:29.970103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:29.970103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:08:29.970103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:29.970552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:29.970552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:29.970552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:08:29.970552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:29.970552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:08:29.970552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:08:29.970552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:29.970552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:08:29.970552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:29.970552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:08:29.970552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:08:29.970552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:29.970552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:08:29.970920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:29.970920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:08:29.971980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:29.971980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:08:29.971980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:08:29.971980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:08:29.971980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:08:29.971980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:08:29.971980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:29.971980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:08:29.971980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:08:29.971980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:08:29.971980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:08:29.971980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:08:29.971980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:29.971980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:29.971980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:08:29.971980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:08:29.972848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:08:29.972848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:08:29.972848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:29.972848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:29.972848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:29.972848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:08:29.972848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:29.972848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:29.972848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:29.972848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:08:29.972848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:29.972848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:08:29.972848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:08:29.972848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:29.972848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:08:29.972848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:29.972848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:08:29.972848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:08:29.972848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:29.973781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:08:29.973781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:29.973781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:08:29.973781 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:08:29.973781 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:08:29.973781 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:08:29.973781 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:08:29.973781 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:08:29.974209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:29.974605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:08:29.974605 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:29.974605 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:08:29.974605 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:08:29.974605 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:29.974605 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:08:29.974605 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:29.974605 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:08:29.975439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:29.975439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:08:29.975439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:08:29.975439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:08:29.975439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:29.975439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:08:29.975439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:29.975439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:08:29.975439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:29.975439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:08:29.975439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:29.975439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:08:29.975439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:29.975439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:08:29.975439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:08:29.976189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:29.976189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:08:29.976189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:29.976189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:08:29.976189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:08:29.976189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:29.976189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:08:29.976189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:29.976189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:08:29.976657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:29.976657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:08:29.976657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:29.976657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:08:29.976657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:08:29.976657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:08:29.976657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:08:29.976657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:08:29.976657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:08:29.976657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:08:29.976657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:29.976657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:29.977230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:29.977230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:08:29.977230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:29.977230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:29.977230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:29.977230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:08:29.977230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:29.977230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:08:29.977230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:08:29.977230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:29.977230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:08:29.977230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:29.977230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:08:29.977230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:08:29.977890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:29.977890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:08:29.977890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:29.977890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:08:29.977890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:29.977890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:08:29.977890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:08:29.977890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:08:29.977890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:08:29.977890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:08:29.977890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:29.977890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:08:29.977890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:08:29.977890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:08:29.978555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:08:29.978555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:08:29.978555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:29.978555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:29.978555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:08:29.978555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:08:29.978555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:08:29.978555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:08:29.978555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:08:29.978555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:08:29.978555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:08:29.978555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:29.978555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:29.978555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:29.978555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:08:29.978555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:29.978555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:29.978555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:29.978555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:08:29.979473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:29.979473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:08:29.979473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:08:29.979473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:29.979473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:08:29.979473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:29.979473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:08:29.979473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:08:29.979473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:29.979473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:08:29.979473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:29.979473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:08:29.979976 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:08:29.980288 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:08:29.980458 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:08:29.980649 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:08:29.981070 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:08:29.981419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:29.982090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:08:29.982090 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:29.982090 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:08:29.982090 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:08:29.982090 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:29.982090 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:08:29.982090 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:29.982090 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:08:29.982931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:29.982931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:08:29.982931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:08:29.982931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:08:29.982931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:29.982931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:08:29.982931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:29.982931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:08:29.982931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:29.982931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:08:29.982931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:29.982931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:08:29.982931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:29.982931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:08:29.982931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:08:29.983375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:29.983375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:08:29.983375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:29.983375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:08:29.983375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:08:29.983375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:29.983375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:08:29.983375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:29.983375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:08:29.983865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:29.983865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:08:29.983865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:29.983865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:08:29.983865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:08:29.983865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:08:29.983865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:08:29.983865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:08:29.983865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:08:29.983865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:29.983865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:29.984210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:29.984210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:08:29.984210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:29.984210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:29.984210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:29.984210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:08:29.984210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:29.984210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:08:29.984210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:08:29.984210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:29.984210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:08:29.984210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:29.984210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:08:29.984210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:08:29.984636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:29.984636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:08:29.984636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:29.984636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:08:29.984983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:29.984983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:08:29.984983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:08:29.984983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:08:29.984983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:08:29.984983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:08:29.984983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:29.984983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:08:29.984983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:08:29.984983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:08:29.984983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:08:29.984983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:08:29.984983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:29.984983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:29.984983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:08:29.984983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:08:29.984983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:08:29.985531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:08:29.985531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:08:29.985531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:08:29.985531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:29.985531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:29.985531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:29.985531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:08:29.985531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:29.985531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:29.985531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:29.985531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:08:29.985531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:29.985531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:08:29.985531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:08:29.985531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:29.985531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:08:29.985531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:29.985531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:08:29.985531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:08:29.986079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:29.986079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:08:29.986079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:29.986079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:08:29.989445 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:08:29.989714 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:08:29.989824 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:08:29.989971 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:08:29.990382 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:08:29.990744 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:08:29.991009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:29.991693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:08:29.991693 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:29.991693 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:08:29.991693 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:08:29.991693 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:29.991693 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:08:29.991693 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:29.991693 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:08:29.992498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:29.992498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:08:29.992498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:29.992498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:08:29.992498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:08:29.992498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:08:29.992498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:08:29.992498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:29.992498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:29.992498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:29.992498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:08:29.992498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:29.992863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:29.992863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:29.992863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:08:29.992863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:29.992863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:08:29.992863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:08:29.992863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:29.992863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:08:29.992863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:29.992863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:08:29.992863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:08:29.992863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:29.992863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:08:29.993213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:29.993213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:08:29.993435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:29.993435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:08:29.993435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:08:29.993435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:08:29.993435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:08:29.993435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:08:29.993435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:29.993435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:08:29.993435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:08:29.993435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:08:29.993435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:08:29.993435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:08:29.993435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:29.993435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:29.993435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:08:29.993435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:08:29.994069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:08:29.994069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:08:29.994069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:29.994069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:29.994069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:29.994069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:08:29.994069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:29.994069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:29.994069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:29.994069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:08:29.994069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:29.994069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:08:29.994069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:08:29.994069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:29.994069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:08:29.994069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:29.994069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:08:29.994069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:08:29.994069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:29.994741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:08:29.994741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:29.994741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:08:34.281086 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:08:34.281883 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:08:34.281883 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:08:34.282306 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:08:34.314898 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:08:34.315214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:34.316004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:08:34.316004 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:34.316004 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:08:34.316004 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:08:34.316004 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:34.316004 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:08:34.316004 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:34.316004 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:08:34.317964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:34.317964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:08:34.317964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:34.317964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:08:34.317964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:08:34.317964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:08:34.317964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:08:34.317964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:34.317964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:34.317964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:34.317964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:08:34.317964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:34.318420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:34.318420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:34.318420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:08:34.318420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:34.318420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:08:34.318420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:08:34.318420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:34.318420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:08:34.318420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:34.318420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:08:34.318420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:08:34.318420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:34.318420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:08:34.318916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:34.318916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:08:34.318916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:34.318916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:08:34.318916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:08:34.318916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:08:34.318916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:08:34.318916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:08:34.318916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:34.318916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:08:34.318916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:08:34.318916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:08:34.318916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:08:34.318916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:08:34.318916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:34.320118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:34.320118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:08:34.320118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:08:34.320118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:08:34.320118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:08:34.320118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:34.320118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:34.320118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:34.320118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:08:34.320118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:34.320118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:34.320118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:34.320118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:08:34.320118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:34.320118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:08:34.320118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:08:34.320118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:34.320118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:08:34.320118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:34.320663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:08:34.320663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:08:34.320663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:34.320663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:08:34.320663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:34.320663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:08:34.965831 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:08:34.966277 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:08:34.966673 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:08:34.968389 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:08:34.968629 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:08:34.968762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:34.969563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:08:34.969563 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:34.969563 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:08:34.969563 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:08:34.969563 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:34.969563 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:08:34.969563 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:34.969563 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:08:34.970979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:34.970979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:08:34.970979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:34.970979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:08:34.970979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:08:34.970979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:08:34.970979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:08:34.970979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:34.970979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:34.970979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:34.970979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:08:34.970979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:34.971401 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:34.971401 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:34.971401 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:08:34.971401 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:34.971401 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:08:34.971401 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:08:34.971401 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:34.971401 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:08:34.971401 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:34.971401 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:08:34.971401 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:08:34.971401 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:34.971401 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:08:34.971814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:34.971814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:08:34.972017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:34.972017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:08:34.972017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:08:34.972017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:08:34.972017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:08:34.972017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:08:34.972017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:34.972017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:08:34.972017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:08:34.972017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:08:34.972017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:08:34.972017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:08:34.972017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:34.972017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:34.972017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:08:34.972017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:08:34.972511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:08:34.972511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:08:34.972511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:34.972511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:34.972511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:34.972511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:08:34.972511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:34.972511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:34.972511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:34.972511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:08:34.972511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:34.972511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:08:34.972511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:08:34.972511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:34.972511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:08:34.972511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:34.972511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:08:34.972511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:08:34.972511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:34.973036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:08:34.973036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:34.973036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:08:34.973118 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:08:34.973427 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:08:34.973502 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:08:34.973692 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:08:34.974010 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:08:34.974207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:34.974927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:08:34.974927 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:34.974927 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:08:34.974927 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:08:34.974927 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:34.974927 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:08:34.974927 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:34.974927 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:08:34.975729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:34.975729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:08:34.975729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:08:34.975729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:08:34.975729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:34.975729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:08:34.975729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:34.975729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:08:34.975729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:34.975729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:08:34.975729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:34.975729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:08:34.975729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:34.975729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:08:34.975729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:08:34.976268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:34.976268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:08:34.976268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:34.976268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:08:34.976268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:08:34.976268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:34.976268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:08:34.976268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:34.976268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:08:34.976627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:34.976627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:08:34.976627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:34.976627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:08:34.976627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:08:34.976627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:08:34.976627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:08:34.976627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:08:34.976627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:08:34.976627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:08:34.976627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:34.976627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:34.977040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:34.977040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:08:34.977040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:34.977040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:34.977040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:34.977040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:08:34.977040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:34.977040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:08:34.977040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:08:34.977040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:34.977040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:08:34.977040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:34.977040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:08:34.977040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:08:34.977478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:34.977478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:08:34.977478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:34.977478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:08:34.977674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:34.977674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:08:34.977674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:08:34.977674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:08:34.977674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:08:34.977674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:08:34.977674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:34.977674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:08:34.977674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:08:34.977674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:08:34.977674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:08:34.977674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:08:34.977674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:34.977674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:34.977674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:08:34.977674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:08:34.978137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:08:34.978137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:08:34.978137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:08:34.978137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:08:34.978137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:08:34.978137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:34.978137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:34.978137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:34.978137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:08:34.978137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:34.978137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:34.978137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:34.978137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:08:34.978137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:34.978137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:08:34.978137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:08:34.978137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:34.978137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:08:34.978733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:34.978733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:08:34.978733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:08:34.978733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:34.978733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:08:34.978733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:34.978733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:08:34.980388 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:08:34.980610 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:08:34.980790 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:08:34.980918 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:08:34.982033 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:08:34.982192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:34.982679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:08:34.982679 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:34.982679 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:08:34.982679 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:08:34.982679 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:34.982679 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:08:34.982679 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:34.982679 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:08:34.983440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:34.983440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:08:34.983440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:08:34.983440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:08:34.983440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:34.983440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:08:34.983440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:34.983440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:08:34.983440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:34.983440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:08:34.983440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:34.983440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:08:34.983440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:34.983440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:08:34.983440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:08:34.983861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:34.983861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:08:34.983861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:34.983861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:08:34.983861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:08:34.983861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:34.983861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:08:34.983861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:34.983861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:08:34.984330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:34.984330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:08:34.984330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:34.984330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:08:34.984330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:08:34.984330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:08:34.984330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:08:34.984330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:08:34.984330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:08:34.984330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:34.984330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:34.984727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:34.984727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:08:34.984727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:34.984727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:34.984727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:34.984727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:08:34.984727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:34.984727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:08:34.984727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:08:34.984727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:34.984727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:08:34.984727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:34.984727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:08:34.984727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:08:34.985124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:34.985124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:08:34.985124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:34.985124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:08:34.985702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:34.985702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:08:34.985702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:08:34.985702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:08:34.985702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:08:34.985702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:08:34.985702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:34.985702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:08:34.985702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:08:34.985702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:08:34.985702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:08:34.985702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:08:34.985702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:34.985702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:34.985702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:08:34.985702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:08:34.985702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:08:34.986512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:08:34.986512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:08:34.986512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:08:34.986512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:34.986512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:34.986512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:34.986512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:08:34.986512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:34.986512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:34.986512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:34.986512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:08:34.986512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:34.986512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:08:34.986512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:08:34.986512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:34.986512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:08:34.986512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:34.986512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:08:34.986512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:08:34.988050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:34.988050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:08:34.988050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:34.988050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:08:34.989800 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:08:34.990056 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:08:34.990161 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:08:34.990304 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:08:34.990588 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:08:34.990888 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:08:34.991130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:34.993072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:08:34.993072 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:34.993072 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:08:34.993072 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:08:34.993072 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:34.993072 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:08:34.993072 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:34.993072 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:08:34.993957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:34.993957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:08:34.993957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:34.993957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:08:34.993957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:08:34.993957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:08:34.993957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:08:34.993957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:34.993957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:34.993957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:34.993957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:08:34.993957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:34.994418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:34.994418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:34.994418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:08:34.994418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:34.994418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:08:34.994418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:08:34.994418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:34.994418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:08:34.994418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:34.994418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:08:34.994418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:08:34.994418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:34.994418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:08:34.994783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:34.994783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:08:34.994962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:34.994962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:08:34.994962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:08:34.994962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:08:34.994962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:08:34.994962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:08:34.994962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:34.994962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:08:34.994962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:08:34.994962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:08:34.994962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:08:34.994962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:08:34.994962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:34.994962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:34.994962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:08:34.994962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:08:34.995474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:08:34.995474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:08:34.995474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:34.995474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:34.995474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:34.995474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:08:34.995474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:34.995474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:34.995474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:34.995474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:08:34.995474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:34.995474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:08:34.995474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:08:34.995474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:34.995474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:08:34.995474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:34.995474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:08:34.995474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:08:34.995474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:34.995991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:08:34.995991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:34.995991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:08:39.281757 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:08:39.282289 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:08:39.282495 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:08:39.283641 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:08:39.320153 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:08:39.320424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:39.321039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:08:39.321039 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:39.321039 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:08:39.321039 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:08:39.321039 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:39.321039 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:08:39.321039 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:39.321039 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:08:39.322242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:39.322242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:08:39.322242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:39.322242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:08:39.322242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:08:39.322242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:08:39.322242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:08:39.322242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:39.322242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:39.322242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:39.322242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:08:39.322242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:39.323207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:39.323207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:39.323207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:08:39.323207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:39.323207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:08:39.323207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:08:39.323207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:39.323207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:08:39.323207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:39.323207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:08:39.323207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:08:39.323207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:39.323207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:08:39.323807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:39.323807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:08:39.323807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:39.323807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:08:39.323807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:08:39.323807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:08:39.323807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:08:39.323807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:08:39.323807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:39.323807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:08:39.323807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:08:39.323807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:08:39.323807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:08:39.323807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:08:39.323807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:39.324701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:39.324701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:08:39.324701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:08:39.324701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:08:39.324701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:08:39.324701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:39.324701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:39.324701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:39.324701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:08:39.324701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:39.324701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:39.324701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:39.324701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:08:39.324701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:39.324701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:08:39.324701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:08:39.324701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:39.324701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:08:39.324701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:39.325786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:08:39.325786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:08:39.325786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:39.325786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:08:39.325786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:39.325786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:08:39.966477 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:08:39.966921 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:08:39.967025 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:08:39.967862 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:08:39.968060 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:08:39.968219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:39.968884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:08:39.968884 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:39.968884 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:08:39.968884 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:08:39.968884 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:39.968884 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:08:39.968884 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:39.968884 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:08:39.970780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:39.970780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:08:39.970780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:39.970780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:08:39.970780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:08:39.970780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:08:39.970780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:08:39.970780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:39.970780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:39.970780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:39.970780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:08:39.970780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:39.971330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:39.971330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:39.971330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:08:39.971330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:39.971330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:08:39.971330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:08:39.971330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:39.971330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:08:39.971330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:39.971330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:08:39.971330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:08:39.971330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:39.971330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:08:39.971948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:39.971948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:08:39.972703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:39.972703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:08:39.972703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:08:39.972703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:08:39.972703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:08:39.972703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:08:39.972703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:39.972703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:08:39.972703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:08:39.972703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:08:39.972703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:08:39.972703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:08:39.972703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:39.972703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:39.972703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:08:39.972703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:08:39.973384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:08:39.973384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:08:39.973384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:39.973384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:39.973384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:39.973384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:08:39.973384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:39.973384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:39.973384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:39.973384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:08:39.973384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:39.973384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:08:39.973384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:08:39.973384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:39.973384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:08:39.973384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:39.973384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:08:39.973384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:08:39.973384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:39.974157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:08:39.974157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:39.974157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:08:39.974157 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:08:39.974157 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:08:39.974157 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:08:39.974477 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:08:39.974947 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:08:39.975274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:39.976112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:08:39.976112 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:39.976112 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:08:39.976112 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:08:39.976112 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:39.976112 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:08:39.976112 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:39.976112 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:08:39.977125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:39.977125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:08:39.977125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:08:39.977125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:08:39.977125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:39.977125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:08:39.977125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:39.977125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:08:39.977125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:39.977125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:08:39.977125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:39.977125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:08:39.977125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:39.977125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:08:39.977125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:08:39.977537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:39.977537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:08:39.977537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:39.977537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:08:39.977537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:08:39.977537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:39.977537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:08:39.977537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:39.977537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:08:39.978073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:39.978073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:08:39.978073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:39.978073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:08:39.978073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:08:39.978073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:08:39.978073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:08:39.978073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:08:39.978073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:08:39.978073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:08:39.978073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:39.978073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:39.978422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:39.978422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:08:39.978422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:39.978422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:39.978422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:39.978422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:08:39.978422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:39.978422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:08:39.978422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:08:39.978422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:39.978422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:08:39.978422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:39.978422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:08:39.978422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:08:39.978821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:39.978821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:08:39.978821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:39.978821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:08:39.979174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:39.979174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:08:39.979174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:08:39.979174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:08:39.979174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:08:39.979174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:08:39.979174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:39.979174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:08:39.979174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:08:39.979174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:08:39.979174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:08:39.979174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:08:39.979174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:39.979174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:39.979174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:08:39.979174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:08:39.979609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:08:39.979609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:08:39.979609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:08:39.979609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:08:39.979609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:08:39.979609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:39.979609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:39.979609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:39.979609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:08:39.979609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:39.979609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:39.979609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:39.979609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:08:39.979609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:39.979609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:08:39.979609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:08:39.979609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:39.979609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:08:39.980093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:39.980093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:08:39.980093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:08:39.980093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:39.980093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:08:39.980093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:39.980093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:08:39.980660 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:08:39.980891 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:08:39.980954 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:08:39.981078 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:08:39.981262 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:08:39.981423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:39.981771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:08:39.981771 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:39.981771 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:08:39.981771 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:08:39.981771 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:39.981771 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:08:39.981771 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:39.981771 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:08:39.982332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:39.982332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:08:39.982332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:08:39.982332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:08:39.982332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:39.982332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:08:39.982332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:39.982332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:08:39.982332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:39.982332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:08:39.982332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:39.982332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:08:39.982332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:39.982332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:08:39.982332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:08:39.982744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:39.982744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:08:39.982744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:39.982744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:08:39.982744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:08:39.982744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:39.982744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:08:39.982744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:39.982744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:08:39.982973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:39.982973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:08:39.982973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:39.982973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:08:39.982973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:08:39.982973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:08:39.982973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:08:39.982973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:08:39.982973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:08:39.982973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:39.982973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:39.983340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:39.983340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:08:39.983340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:39.983340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:39.983340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:39.983340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:08:39.983340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:39.983340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:08:39.983340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:08:39.983340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:39.983340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:08:39.983340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:39.983340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:08:39.983340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:08:39.983715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:39.983715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:08:39.983715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:39.983715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:08:39.983843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:39.983843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:08:39.983843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:08:39.983843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:08:39.983843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:08:39.983843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:08:39.983843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:39.983843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:08:39.983843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:08:39.983843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:08:39.983843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:08:39.983843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:08:39.983843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:39.983843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:39.983843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:08:39.983843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:08:39.983843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:08:39.984279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:08:39.984279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:08:39.984279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:08:39.984279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:39.984279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:39.984279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:39.984279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:08:39.984279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:39.984279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:39.984279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:39.984279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:08:39.984279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:39.984279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:08:39.984279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:08:39.984279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:39.984279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:08:39.984279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:39.984279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:08:39.984279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:08:39.984789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:39.984789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:08:39.984789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:39.984789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:08:39.990120 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:08:39.990383 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:08:39.990543 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:08:39.990718 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:08:39.991498 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:08:39.991921 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:08:39.992158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:39.993033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:08:39.993033 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:39.993033 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:08:39.993033 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:08:39.993033 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:39.993033 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:08:39.993033 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:39.993033 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:08:39.993939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:39.993939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:08:39.993939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:39.993939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:08:39.993939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:08:39.993939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:08:39.993939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:08:39.993939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:39.993939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:39.993939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:39.993939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:08:39.993939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:39.994295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:39.994295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:39.994295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:08:39.994295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:39.994295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:08:39.994295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:08:39.994295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:39.994295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:08:39.994295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:39.994295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:08:39.994295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:08:39.994295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:39.994295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:08:39.994747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:39.994747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:08:39.995010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:39.995010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:08:39.995010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:08:39.995010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:08:39.995010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:08:39.995010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:08:39.995010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:39.995010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:08:39.995010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:08:39.995010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:08:39.995010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:08:39.995010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:08:39.995010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:39.995010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:39.995010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:08:39.995010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:08:39.995726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:08:39.995726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:08:39.995726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:39.995726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:39.995726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:39.995726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:08:39.995726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:39.995726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:39.995726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:39.995726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:08:39.995726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:39.995726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:08:39.995726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:08:39.995726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:39.995726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:08:39.995726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:39.995726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:08:39.995726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:08:39.995726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:39.996606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:08:39.996606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:39.996606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:08:44.282289 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:08:44.282996 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:08:44.283076 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:08:44.284327 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:08:44.328659 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:08:44.329189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:44.330514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:08:44.330514 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:44.330514 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:08:44.330514 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:08:44.330514 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:44.330514 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:08:44.330514 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:44.330514 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:08:44.332996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.047 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:44.332996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:08:44.332996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:44.332996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:08:44.332996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:08:44.332996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:08:44.332996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:08:44.332996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:44.332996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:44.332996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:44.332996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:08:44.332996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:44.333824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:44.333824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:44.333824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:08:44.333824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:44.333824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:08:44.333824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:08:44.333824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:44.333824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:08:44.333824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:44.333824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:08:44.333824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:08:44.333824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:44.333824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:08:44.334425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:44.334425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:08:44.334892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:44.334892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:08:44.334892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:08:44.334892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:08:44.334892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:08:44.334892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:08:44.334892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:44.334892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:08:44.334892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:08:44.334892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:08:44.334892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:08:44.334892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:08:44.334892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:44.334892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:44.334892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:08:44.334892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:08:44.335710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:08:44.335710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:08:44.335710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:44.335710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:44.335710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:44.335710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:08:44.335710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:44.335710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:44.335710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:44.335710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:08:44.335710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:44.335710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:08:44.335710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:08:44.335710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:44.335710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:08:44.335710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:44.335710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:08:44.335710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:08:44.335710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:44.336613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:08:44.336613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:44.336613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:08:44.967085 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:08:44.968282 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:08:44.968528 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:08:44.969835 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:08:44.970222 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:08:44.970712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:44.971809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:08:44.971809 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:44.971809 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:08:44.971809 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:08:44.971809 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:44.971809 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:08:44.971809 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:44.971809 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:08:44.974956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:44.974956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:08:44.974956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:44.974956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:08:44.974956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:08:44.974956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:08:44.974956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:08:44.974956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:44.974956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:44.974956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:44.974956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:08:44.974956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:44.975202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:44.975202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:44.975202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:08:44.975202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:44.975202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:08:44.975202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:08:44.975202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:44.975202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:08:44.975202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:44.975202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:08:44.975202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:08:44.975202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:44.975202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:08:44.975444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:44.975444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:08:44.975444 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:08:44.976865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:44.976865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:08:44.976865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:08:44.976865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:08:44.976865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:08:44.976865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:08:44.976865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:44.976865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:08:44.976865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:08:44.976865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:08:44.976865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:08:44.976865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:08:44.976865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:44.976865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:44.976865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:08:44.976865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:08:44.977167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:08:44.977167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:08:44.977167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:44.977167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:44.977167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:44.977167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:08:44.977167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:44.977167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:44.977167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:44.977167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:08:44.977167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:44.977167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:08:44.977167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:08:44.977167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:44.977167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:08:44.977167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:44.977167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:08:44.977167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:08:44.977167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:44.977522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:08:44.977522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:44.977522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:08:44.977522 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:08:44.977522 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:08:44.977522 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:08:44.977522 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:08:44.977722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:44.978100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:08:44.978100 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:44.978100 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:08:44.978100 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:08:44.978100 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:44.978100 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:08:44.978100 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:44.978100 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:08:44.978748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:44.978748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:08:44.978748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:08:44.978748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:08:44.978748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:44.978748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:08:44.978748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:44.978748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:08:44.978748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:44.978748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:08:44.978748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:44.978748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:08:44.978748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:44.978748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:08:44.978748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:08:44.979629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:44.979629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:08:44.979629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:44.979629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:08:44.979629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:08:44.979629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:44.979629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:08:44.979629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:44.979629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:08:44.979629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:44.979629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:08:44.979629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:44.979629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:08:44.979629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:08:44.980607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:08:44.980607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:08:44.980607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:08:44.980607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:08:44.980607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:08:44.980607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:44.980607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:44.980607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:44.980607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:08:44.980607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:44.980607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:44.980607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:44.980607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:08:44.981371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:44.981371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:08:44.981371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:08:44.981371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:44.981371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:08:44.981371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:44.981371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:08:44.981371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:08:44.981371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:44.981371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:08:44.981371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:44.981371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:08:44.982162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:44.982162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:08:44.982162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:08:44.982162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:08:44.982162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:08:44.982162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:08:44.982162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:44.982162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:08:44.982162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:08:44.982162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:08:44.982162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:08:44.982162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:08:44.982162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:44.982162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:44.982162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:08:44.982162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:08:44.983151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:08:44.983151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:08:44.983151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:08:44.983151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:08:44.983151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:08:44.983151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:44.983151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:44.983151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:44.983151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:08:44.983151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:44.983151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:44.983151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:44.983151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:08:44.983151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:44.983151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:08:44.983151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:08:44.983151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:44.983151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:08:44.984320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:44.984320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:08:44.984320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:08:44.984320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:44.984320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:08:44.984320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:44.984320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:08:44.984320 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:08:44.984320 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:08:44.984320 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:08:44.985027 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:08:44.985027 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:08:44.985027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:44.985027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:08:44.985027 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:44.985027 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:08:44.985027 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:08:44.985027 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:44.985027 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:08:44.985027 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:44.985027 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:08:44.985027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:44.985826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:08:44.985826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:08:44.985826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:08:44.985826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:44.985826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:08:44.985826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:44.985826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:08:44.985826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:44.985826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:08:44.985826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:44.985826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:08:44.985826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:44.985826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:08:44.985826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:08:44.985826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:44.985826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:08:44.985826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:44.986833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:08:44.986833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:08:44.986833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:44.986833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:08:44.986833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:44.986833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:08:44.986833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:44.986833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:08:44.986833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:44.986833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:08:44.986833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:08:44.986833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:08:44.986833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:08:44.987742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:08:44.987742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:08:44.987742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:44.987742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:44.987742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:44.987742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:08:44.987742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:44.987742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:44.987742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:44.987742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:08:44.987742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:44.987742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:08:44.987742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:08:44.988707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:44.988707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:08:44.988707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:44.988707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:08:44.988707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:08:44.988707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:44.988707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:08:44.988707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:44.988707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:08:44.988707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:44.988707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:08:44.988707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:08:44.989141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:08:44.989141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:08:44.989141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:08:44.989141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:44.989141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:08:44.989141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:08:44.989141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:08:44.989141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:08:44.989141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:08:44.989141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:44.989141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:44.989141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:08:44.989141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:08:44.989141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:08:44.989141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:08:44.989141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:08:44.989141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:08:44.989141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:44.989141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:44.989659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:44.989659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:08:44.989659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:44.989659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:44.989659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:44.989659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:08:44.989659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:44.989659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:08:44.989659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:08:44.989659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:44.989659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:08:44.989659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:44.989659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:08:44.989659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:08:44.989659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:44.989659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:08:44.989659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:44.989659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:08:44.990547 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:08:44.990846 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:08:44.990920 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:08:44.991049 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:08:44.991232 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:08:44.991386 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:08:44.991517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:44.991835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:08:44.991835 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:44.991835 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:08:44.991835 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:08:44.991835 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:44.991835 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:08:44.991835 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:44.991835 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:08:44.992628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:44.992628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:08:44.992628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:44.992628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:08:44.992628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:08:44.992628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:08:44.992628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:08:44.992628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:44.992628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:44.992628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:44.992628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:08:44.992628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:44.992980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:44.992980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:44.992980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:08:44.992980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:44.992980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:08:44.992980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:08:44.992980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:44.992980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:08:44.992980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:44.992980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:08:44.992980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:08:44.992980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:44.992980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:08:44.993250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:44.993250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:08:44.993382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:44.993382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:08:44.993382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:08:44.993382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:08:44.993382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:08:44.993382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:08:44.993382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:44.993382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:08:44.993382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:08:44.993382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:08:44.993382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:08:44.993382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:08:44.993382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:44.993382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:44.993382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:08:44.993382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:08:44.993698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:08:44.993698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:08:44.993698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:44.993698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:44.993698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:44.993698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:08:44.993698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:44.993698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:44.993698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:44.993698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:08:44.993698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:44.993698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:08:44.993698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:08:44.993698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:44.993698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:08:44.993698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:44.993698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:08:44.993698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:08:44.993698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:44.994039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:08:44.994039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:44.994039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:08:49.283718 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:08:49.284517 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:08:49.284735 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:08:49.286092 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:08:49.323969 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:08:49.324163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:49.324653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:08:49.324653 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:49.324653 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:08:49.324653 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:08:49.324653 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:49.324653 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:08:49.324653 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:49.324653 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:08:49.325315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:49.325315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:08:49.325315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:49.325315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:08:49.325315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:08:49.325315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:08:49.325315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:08:49.325315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:49.325315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:49.325315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:49.325315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:08:49.325315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:49.325917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:49.325917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:49.325917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:08:49.325917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:49.325917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:08:49.325917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:08:49.325917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:49.325917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:08:49.325917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:49.325917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:08:49.325917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:08:49.325917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:49.325917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:08:49.326487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:49.326487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:08:49.326487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:49.326487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:08:49.326487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:08:49.326487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:08:49.326487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:08:49.326487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:08:49.326487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:49.326487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:08:49.326487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:08:49.326487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:08:49.326487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:08:49.326487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:08:49.326487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:49.327239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:49.327239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:08:49.327239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:08:49.327239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:08:49.327239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:08:49.327239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:49.327239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:49.327239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:49.327239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:08:49.327239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:49.327239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:49.327239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:49.327239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:08:49.327239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:49.327239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:08:49.327239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:08:49.327239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:49.327239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:08:49.327239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:49.328062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:08:49.328062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:08:49.328062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:49.328062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:08:49.328062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:49.328062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:08:49.967783 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:08:49.968828 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:08:49.969031 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:08:49.970119 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:08:49.970565 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:08:49.970805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:49.971765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:08:49.971765 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:49.971765 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:08:49.971765 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:08:49.971765 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:49.971765 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:08:49.971765 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:49.971765 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:08:49.973210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:49.973210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:08:49.973210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:49.973210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:08:49.973210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:08:49.973210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:08:49.973210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:08:49.973210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:49.973210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:49.973210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:49.973210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:08:49.973210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:49.973864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:49.973864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:49.973864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:08:49.973864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:49.973864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:08:49.973864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:08:49.973864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:49.973864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:08:49.973864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:49.973864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:08:49.973864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:08:49.973864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:49.973864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:08:49.974473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:49.974473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:08:49.974962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:49.974962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:08:49.974962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:08:49.974962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:08:49.974962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:08:49.974962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:08:49.974962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:49.974962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:08:49.974962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:08:49.974962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:08:49.974962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:08:49.974962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:08:49.974962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:49.974962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:49.974962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:08:49.974962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:08:49.975952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:08:49.975952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:08:49.975952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:49.975952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:49.975952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:49.975952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:08:49.975952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:49.975952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:49.975952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:49.975952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:08:49.975952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:49.975952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:08:49.975952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:08:49.975952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:49.975952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:08:49.975952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:49.975952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:08:49.975952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:08:49.975952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:49.976808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:08:49.976808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:49.976808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:08:49.977401 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:08:49.977802 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:08:49.977921 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:08:49.978053 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:08:49.978299 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:08:49.978513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:49.978971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:08:49.978971 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:49.978971 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:08:49.978971 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:08:49.978971 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:49.978971 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:08:49.978971 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:49.978971 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:08:49.979708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:49.979708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:08:49.979708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:08:49.979708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:08:49.979708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:49.979708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:08:49.979708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:49.979708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:08:49.979708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:49.979708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:08:49.979708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:49.979708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:08:49.979708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:49.979708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:08:49.979708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:08:49.980273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:49.980273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:08:49.980273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:49.980273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:08:49.980273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:08:49.980273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:49.980273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:08:49.980273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:49.980273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:08:49.980607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:49.980607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:08:49.980607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:49.980607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:08:49.980607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:08:49.980607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:08:49.980607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:08:49.980607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:08:49.980607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:08:49.980607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:08:49.980607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:49.980607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:49.980968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:49.980968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:08:49.980968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:49.980968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:49.980968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:49.980968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:08:49.980968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:49.980968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:08:49.980968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:08:49.980968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:49.980968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:08:49.980968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:49.980968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:08:49.980968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:08:49.981466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:49.981466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:08:49.981466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:49.981466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:08:49.981582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:49.981582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:08:49.981582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:08:49.981582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:08:49.981582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:08:49.981582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:08:49.981582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:49.981582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:08:49.981582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:08:49.981582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:08:49.981582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:08:49.981582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:08:49.981582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:49.981582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:49.981582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:08:49.981582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:08:49.982071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:08:49.982071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:08:49.982071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:08:49.982071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:08:49.982071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:08:49.982071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:49.982071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:49.982071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:49.982071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:08:49.982071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:49.982071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:49.982071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:49.982071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:08:49.982071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:49.982071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:08:49.982071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:08:49.982071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:49.982071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:08:49.982575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:49.982575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:08:49.982575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:08:49.982575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:49.982575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:08:49.982575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:49.982575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:08:49.982575 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:08:49.982575 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:08:49.982575 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:08:49.982920 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:08:49.982920 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:08:49.982993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:49.983390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:08:49.983390 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:49.983390 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:08:49.983390 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:08:49.983390 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:49.983390 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:08:49.983390 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:49.983390 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:08:49.984152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:49.984152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:08:49.984152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:08:49.984152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:08:49.984152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:49.984152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:08:49.984152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:49.984152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:08:49.984152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:49.984152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:08:49.984152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:49.984152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:08:49.984152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:49.984152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:08:49.984152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:08:49.984621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:49.984621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:08:49.984621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:49.984621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:08:49.984621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:08:49.984621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:49.984621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:08:49.984621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:49.984621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:08:49.985001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:49.985001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:08:49.985001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:49.985001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:08:49.985001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:08:49.985001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:08:49.985001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:08:49.985001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:08:49.985001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:08:49.985001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:49.985001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:49.985422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:49.985422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:08:49.985422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:49.985422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:49.985422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:49.985422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:08:49.985422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:49.985422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:08:49.985422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:08:49.985422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:49.985422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:08:49.985422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:49.985422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:08:49.985422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:08:49.985832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:49.985832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:08:49.985832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:49.985832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:08:49.986054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:49.986054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:08:49.986054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:08:49.986054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:08:49.986054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:08:49.986054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:08:49.986054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:49.986054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:08:49.986054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:08:49.986054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:08:49.986054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:08:49.986054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:08:49.986054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:49.986054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:49.986054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:08:49.986054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:08:49.986054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:08:49.986572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:08:49.986572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:08:49.986572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:08:49.986572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:49.986572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:49.986572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:49.986572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:08:49.986572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:49.986572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:49.986572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:49.986572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:08:49.986572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:49.986572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:08:49.986572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:08:49.986572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:49.986572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:08:49.986572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:49.986572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:08:49.986572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:08:49.987146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:49.987146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:08:49.987146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:49.987146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:08:49.990895 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:08:49.991200 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:08:49.991280 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:08:49.991391 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:08:49.991585 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:08:49.991741 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:08:49.991855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:49.992173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:08:49.992173 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:49.992173 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:08:49.992173 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:08:49.992173 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:49.992173 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:08:49.992173 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:49.992173 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:08:49.992749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:49.992749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:08:49.992749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:49.992749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:08:49.992749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:08:49.992749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:08:49.992749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:08:49.992749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:49.992749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:49.992749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:49.992749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:08:49.992749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:49.993141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:49.993141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:49.993141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:08:49.993141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:49.993141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:08:49.993141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:08:49.993141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:49.993141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:08:49.993141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:49.993141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:08:49.993141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:08:49.993141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:49.993141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:08:49.993534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:49.993534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:08:49.993534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:49.993534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:08:49.993534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:08:49.993534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:08:49.993534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:08:49.993534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:08:49.993534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:49.993534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:08:49.993534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:08:49.993534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:08:49.993534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:08:49.993534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:08:49.993534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:49.993989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:49.993989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:08:49.993989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:08:49.993989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:08:49.993989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:08:49.993989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:49.993989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:49.993989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:49.993989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:08:49.993989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:49.993989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:49.993989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:49.993989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:08:49.993989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:49.993989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:08:49.993989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:08:49.993989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:49.993989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:08:49.993989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:49.994566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:08:49.994566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:08:49.994566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:49.994566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:08:49.994566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:49.994566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:08:54.284894 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:08:54.285591 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:08:54.285591 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:08:54.286890 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:08:54.321566 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:08:54.321868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:54.322548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:08:54.322548 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:54.322548 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:08:54.322548 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:08:54.322548 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:54.322548 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:08:54.322548 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:54.322548 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:08:54.323453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:54.323453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:08:54.323453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:54.323453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:08:54.323453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:08:54.323453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:08:54.323453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:08:54.323453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:54.323453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:54.323453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:54.323453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:08:54.323453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:54.324051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:54.324051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:54.324051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:08:54.324051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:54.324051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:08:54.324051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:08:54.324051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:54.324051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:08:54.324051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:54.324051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:08:54.324051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:08:54.324051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:54.324051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:08:54.324662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:54.324662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:08:54.324662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:54.324662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:08:54.324662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:08:54.324662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:08:54.324662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:08:54.324662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:08:54.324662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:54.324662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:08:54.324662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:08:54.324662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:08:54.324662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:08:54.324662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:08:54.324662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:54.325350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:54.325350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:08:54.325350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:08:54.325350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:08:54.325350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:08:54.325350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:54.325350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:54.325350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:54.325350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:08:54.325350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:54.325350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:54.325350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:54.325350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:08:54.325350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:54.325350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:08:54.325350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:08:54.325350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:54.325350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:08:54.325350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:54.326281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:08:54.326281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:08:54.326281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:54.326281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:08:54.326281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:54.326281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:08:54.968756 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:08:54.969599 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:08:54.969815 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:08:54.970998 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:08:54.971294 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:08:54.971579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:54.972572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:08:54.972572 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:54.972572 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:08:54.972572 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:08:54.972572 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:54.972572 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:08:54.972572 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:54.972572 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:08:54.974115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:54.974115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:08:54.974115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:54.974115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:08:54.974115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:08:54.974115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:08:54.974115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:08:54.974115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:54.974115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:54.974115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:54.974115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:08:54.974115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:54.974974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:54.974974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:54.974974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:08:54.974974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:54.974974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:08:54.974974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:08:54.974974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:54.974974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:08:54.974974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:54.974974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:08:54.974974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:08:54.974974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:54.974974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:08:54.975814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:54.975814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:08:54.975814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:54.975814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:08:54.975814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:08:54.975814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:08:54.975814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:08:54.975814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:08:54.975814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:54.975814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:08:54.975814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:08:54.975814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:08:54.975814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:08:54.975814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:08:54.975814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:54.976786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:54.976786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:08:54.976786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:08:54.976786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:08:54.976786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:08:54.976786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:54.976786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:54.976786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:54.976786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:08:54.976786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:54.976786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:54.976786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:54.976786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:08:54.976786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:54.976786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:08:54.976786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:08:54.976786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:54.976786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:08:54.976786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:54.978040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:08:54.978040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:08:54.978040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:54.978040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:08:54.978040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:54.978040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:08:54.978040 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:08:54.978395 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:08:54.978395 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:08:54.978578 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:08:54.978822 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:08:54.979018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:54.979467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:08:54.979467 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:54.979467 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:08:54.979467 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:08:54.979467 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:54.979467 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:08:54.979467 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:54.979467 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:08:54.980164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:54.980164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:08:54.980164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:08:54.980164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:08:54.980164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:54.980164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:08:54.980164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:54.980164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:08:54.980164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:54.980164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:08:54.980164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:54.980164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:08:54.980164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:54.980164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:08:54.980164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:08:54.980814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:54.980814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:08:54.980814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:54.980814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:08:54.980814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:08:54.980814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:54.980814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:08:54.980814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:54.980814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:08:54.981194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:54.981194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:08:54.981194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:54.981194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:08:54.981194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:08:54.981194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:08:54.981194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:08:54.981194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:08:54.981194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:08:54.981194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:08:54.981194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:54.981194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:54.981885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:54.981885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:08:54.981885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:54.981885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:54.981885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:54.981885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:08:54.981885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:54.981885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:08:54.981885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:08:54.981885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:54.981885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:08:54.981885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:54.981885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:08:54.981885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:08:54.982569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:54.982569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:08:54.982569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:54.982569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:08:54.982569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:54.982569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:08:54.982569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:08:54.982569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:08:54.982569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:08:54.982569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:08:54.982569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:54.982569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:08:54.982569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:08:54.982569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:08:54.983118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:08:54.983118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:08:54.983118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:54.983118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:54.983118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:08:54.983118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:08:54.983118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:08:54.983118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:08:54.983118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:08:54.983118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:08:54.983118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:08:54.983118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:54.983118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:54.983118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:54.983118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:08:54.983118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:54.983118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:54.983118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:54.983118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:08:54.983917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:54.983917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:08:54.983917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:08:54.983917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:54.983917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:08:54.983917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:54.983917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:08:54.983917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:08:54.983917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:54.983917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:08:54.983917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:54.983917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:08:54.983917 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:08:54.983917 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:08:54.984514 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:08:54.984514 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:08:54.984514 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:08:54.984514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:54.984514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:08:54.984514 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:54.984514 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:08:54.984514 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:08:54.984514 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:54.984514 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:08:54.984514 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:54.984514 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:08:54.985027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:54.985027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:08:54.985027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:08:54.985027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:08:54.985027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:54.985027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:08:54.985027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:54.985027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:08:54.985027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:54.985027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:08:54.985027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:54.985027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:08:54.985027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:54.985027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:08:54.985027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:08:54.985657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:54.985657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:08:54.985657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:54.985657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:08:54.985657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:08:54.985657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:54.985657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:08:54.985657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:54.985657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:08:54.986024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:54.986024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:08:54.986024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:54.986024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:08:54.986024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:08:54.986024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:08:54.986024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:08:54.986024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:08:54.986024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:08:54.986024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:54.986024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:54.986487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:54.986487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:08:54.986487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:54.986487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:54.986487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:54.986487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:08:54.986487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:54.986487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:08:54.986487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:08:54.986487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:54.986487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:08:54.986487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:54.986487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:08:54.986487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:08:54.986884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:54.986884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:08:54.986884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:54.986884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:08:54.986884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:54.986884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:08:54.986884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:08:54.986884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:08:54.986884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:08:54.986884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:08:54.986884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:54.986884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:08:54.986884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:08:54.986884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:08:54.987309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:08:54.987309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:08:54.987309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:54.987309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:54.987309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:08:54.987309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:08:54.987309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:08:54.987309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:08:54.987309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:08:54.987309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:08:54.987309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:54.987309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:54.987309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:54.987309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:08:54.987309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:54.987309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:54.987309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:54.987309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:08:54.987309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:54.987864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:08:54.987864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:08:54.987864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:54.987864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:08:54.987864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:54.987864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:08:54.987864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:08:54.987864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:54.987864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:08:54.987864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:54.987864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:08:54.991376 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:08:54.991760 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:08:54.991874 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:08:54.992046 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:08:54.992268 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:08:54.992516 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:08:54.992656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:54.993080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:08:54.993080 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:54.993080 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:08:54.993080 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:08:54.993080 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:54.993080 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:08:54.993080 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:54.993080 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:08:54.993907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:54.993907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:08:54.993907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:54.993907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:08:54.993907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:08:54.993907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:08:54.993907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:08:54.993907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:54.993907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:54.993907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:54.993907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:08:54.993907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:54.994297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:54.994297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:54.994297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:08:54.994297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:54.994297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:08:54.994297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:08:54.994297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:54.994297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:08:54.994297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:54.994297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:08:54.994297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:08:54.994297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:54.994297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:08:54.994870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:54.994870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:08:54.994870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:54.994870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:08:54.994870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:08:54.994870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:08:54.994870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:08:54.994870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:08:54.994870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:54.994870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:08:54.994870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:08:54.994870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:08:54.994870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:08:54.994870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:08:54.994870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:54.995486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:54.995486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:08:54.995486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:08:54.995486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:08:54.995486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:08:54.995486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:54.995486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:54.995486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:54.995486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:08:54.995486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:54.995486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:54.995486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:54.995486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:08:54.995486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:54.995486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:08:54.995486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:08:54.995486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:54.995486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:08:54.995486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:54.996182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:08:54.996182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:08:54.996182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:54.996182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:08:54.996182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:54.996182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:08:59.285177 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:08:59.285758 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:08:59.285959 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:08:59.287338 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:08:59.318167 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:08:59.318167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:59.318589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:08:59.318589 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:59.318589 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:08:59.318589 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:08:59.318589 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:59.318589 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:08:59.318589 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:59.318589 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:08:59.319304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:59.319304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:08:59.319304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:59.319304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:08:59.319304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:08:59.319304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:08:59.319304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:08:59.319304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:59.319304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:59.319304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:59.319304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:08:59.319304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:59.319932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:59.319932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:59.319932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:08:59.319932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:59.319932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:08:59.319932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:08:59.319932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:59.319932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:08:59.319932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:59.319932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:08:59.319932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:08:59.319932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:59.319932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:08:59.320486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:59.320486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:08:59.320486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:59.320486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:08:59.320486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:08:59.320486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:08:59.320486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:08:59.320486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:08:59.320486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:59.320486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:08:59.320486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:08:59.320486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:08:59.320486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:08:59.320486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:08:59.320486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:59.320876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:59.320876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:08:59.320876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:08:59.320876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:08:59.320876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:08:59.320876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:59.320876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:59.320876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:59.320876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:08:59.320876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:59.320876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:59.320876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:59.320876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:08:59.320876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:59.320876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:08:59.320876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:08:59.320876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:59.320876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:08:59.320876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:59.321360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:08:59.321360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:08:59.321360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:59.321360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:08:59.321360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:59.321360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:08:59.969517 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:08:59.970097 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:08:59.970283 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:08:59.971290 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:08:59.971510 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:08:59.971620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:59.972323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:08:59.972323 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:59.972323 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:08:59.972323 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:08:59.972323 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:59.972323 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:08:59.972323 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:59.972323 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:08:59.973882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:59.973882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:08:59.973882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:59.973882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:08:59.973882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:08:59.973882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:08:59.973882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:08:59.973882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:59.973882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:59.973882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:59.973882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:08:59.973882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:59.974582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:59.974582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:59.974582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:08:59.974582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:59.974582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:08:59.974582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:08:59.974582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:59.974582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:08:59.974582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:59.974582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:08:59.974582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:08:59.974582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:59.974582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:08:59.975121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:59.975121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:08:59.975599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:59.975599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:08:59.975599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:08:59.975599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:08:59.975599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:08:59.975599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:08:59.975599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:59.975599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:08:59.975599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:08:59.975599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:08:59.975599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:08:59.975599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:08:59.975599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:59.975599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:59.975599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:08:59.975599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:08:59.976478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:08:59.976478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:08:59.976478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:59.976478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:59.976478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:59.976478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:08:59.976478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:59.976478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:59.976478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:59.976478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:08:59.976478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:59.976478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:08:59.976478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:08:59.976478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:59.976478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:08:59.976478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:59.976478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:08:59.976478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:08:59.976478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:59.977009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:08:59.977009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:59.977009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:08:59.978393 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:08:59.978814 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:08:59.979009 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:08:59.979208 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:08:59.981199 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:08:59.981598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:59.982602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:08:59.982602 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:59.982602 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:08:59.982602 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:08:59.982602 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:59.982602 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:08:59.982602 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:59.982602 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:08:59.984709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:59.984709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:08:59.984709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:08:59.984709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:08:59.984709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:59.984709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:08:59.984709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:59.984709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:08:59.984709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:59.984709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:08:59.984709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:59.984709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:08:59.984709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:59.984709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:08:59.984709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:08:59.985173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:59.985173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:08:59.985173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:59.985173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:08:59.985173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:08:59.985173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:59.985173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:08:59.985173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:59.985173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:08:59.986560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:59.986560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:08:59.986560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:59.986560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:08:59.986560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:08:59.986560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:08:59.986560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:08:59.986560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:08:59.986560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:08:59.986560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:08:59.986560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:59.986560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:59.986940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:59.986940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:08:59.986940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:59.986940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:59.986940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:59.986940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:08:59.986940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:59.986940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:08:59.986940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:08:59.986940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:59.986940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:08:59.986940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:59.986940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:08:59.986940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:08:59.987384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:59.987384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:08:59.987384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:59.987384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:08:59.988580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:59.988580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:08:59.988580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:08:59.988580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:08:59.988580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:08:59.988580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:08:59.988580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:59.988580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:08:59.988580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:08:59.988580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:08:59.988580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:08:59.988580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:08:59.988580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:59.988580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:59.988580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:08:59.988580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:08:59.989272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:08:59.989272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:08:59.989272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:08:59.989272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:08:59.989272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:08:59.989272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:59.989272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:59.989272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:59.989272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:08:59.989272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:59.989272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:59.989272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:59.989272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:08:59.989272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:59.989272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:08:59.989272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:08:59.989272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:59.989272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:08:59.989858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:59.989858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:08:59.989858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:08:59.989858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:59.989858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:08:59.989858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:59.989858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:08:59.989858 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:08:59.989858 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:08:59.989858 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:08:59.990118 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:08:59.990118 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:08:59.990385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:59.991158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:08:59.991158 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:59.991158 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:08:59.991158 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:08:59.991158 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:59.991158 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:08:59.991158 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:59.991158 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:08:59.991968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:59.991968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:08:59.991968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:08:59.991968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:08:59.991968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:59.991968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:08:59.991968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:59.991968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:08:59.991968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:59.991968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:08:59.991968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:59.991968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:08:59.991968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:59.991968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:08:59.991968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:08:59.992599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:59.992599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:08:59.992599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:59.992599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:08:59.992599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:08:59.992599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:59.992599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:08:59.992599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:59.992599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:08:59.993000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:59.993000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:08:59.993000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:59.993000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:08:59.993000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:08:59.993000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:08:59.993000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:08:59.993000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:08:59.993000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:08:59.993000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:59.993000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:59.993380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:59.993380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:08:59.993380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:59.993380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:59.993380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:59.993380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:08:59.993380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:59.993380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:08:59.993380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:08:59.993380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:59.993380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:08:59.993380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:59.993380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:08:59.993380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:08:59.993920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:59.993920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:08:59.993920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:59.993920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:08:59.994307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:59.994307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:08:59.994307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:08:59.994307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:08:59.994307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:08:59.994307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:08:59.994307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:59.994307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:08:59.994307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:08:59.994307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:08:59.994307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:08:59.994307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:08:59.994307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:08:59.994307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:59.994307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:08:59.994307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:08:59.994307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:08:59.994821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:08:59.994821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:08:59.994821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:08:59.994821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:59.994821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:59.994821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:59.994821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:08:59.994821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:59.994821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:08:59.994821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:59.994821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:08:59.994821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:59.994821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:08:59.994821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:08:59.994821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:59.994821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:08:59.994821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:59.994821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:08:59.994821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:08:59.995346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:59.995346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:08:59.995346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:59.995346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:08:59.995953 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:08:59.996210 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:08:59.996330 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:08:59.996501 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:08:59.997022 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:08:59.997319 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:08:59.997586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:59.998275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:08:59.998275 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:59.998275 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:08:59.998275 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:08:59.998275 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:59.998275 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:08:59.998275 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:59.998275 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:08:59.999131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:59.999131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:08:59.999131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:08:59.999131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:08:59.999131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:08:59.999131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:08:59.999131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:08:59.999131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:59.999131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:59.999131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:08:59.999131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:08:59.999131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:08:59.999521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:08:59.999521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:08:59.999521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:08:59.999521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:08:59.999521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:08:59.999521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:08:59.999521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:08:59.999521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:08:59.999521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:08:59.999521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:08:59.999521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:08:59.999521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:08:59.999521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:08:59.999899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:08:59.999899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:09:00.000299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:00.000299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:09:00.000299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:09:00.000299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:09:00.000299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:09:00.000299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:09:00.000299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:00.000299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:09:00.000299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:09:00.000299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:09:00.000299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:09:00.000299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:09:00.000299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:00.000299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:00.000299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:09:00.000299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:09:00.000798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:09:00.000798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:09:00.000798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:00.000798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:00.000798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:00.000798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:09:00.000798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:00.000798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:00.000798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:00.000798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:09:00.000798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:00.000798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:09:00.000798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:09:00.000798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:00.000798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:09:00.000798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:00.000798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:09:00.000798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:09:00.000798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:00.001396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:09:00.001396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:00.001396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:09:04.286077 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:09:04.287003 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:09:04.287003 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:09:04.288012 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:09:04.329575 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:04.330304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:04.331119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:09:04.331119 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:04.331119 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:09:04.331119 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:09:04.331119 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:04.331119 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:09:04.331119 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:04.331119 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:09:04.332087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:04.332087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:09:04.332087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:04.332087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:09:04.332087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:09:04.332087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:09:04.332087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:09:04.332087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:04.332087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:04.332087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:04.332087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:09:04.332087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:04.332479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:04.332479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:04.332479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:09:04.332479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:04.332479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:09:04.332479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:09:04.332479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:04.332479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:09:04.332479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:04.332479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:09:04.332479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:09:04.332479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:04.332479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:09:04.332844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:04.332844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:09:04.332915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:04.332915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:09:04.332915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:09:04.332915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:09:04.332915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:09:04.332915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:09:04.332915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:04.332915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:09:04.332915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:09:04.332915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:09:04.332915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:09:04.332915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:09:04.332915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:04.332915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:04.332915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:09:04.332915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:09:04.333344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:09:04.333344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:09:04.333344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:04.333344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:04.333344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:04.333344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:09:04.333344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:04.333344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:04.333344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:04.333344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:09:04.333344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:04.333344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:09:04.333344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:09:04.333344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:04.333344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:09:04.333344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:04.333344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:09:04.333344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:09:04.333344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:04.333880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:09:04.333880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:04.333880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:09:04.970070 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:09:04.971640 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:09:04.971823 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:09:04.973087 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:04.974629 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:04.974629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:04.974886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:09:04.974886 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:04.974886 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:09:04.974886 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:09:04.974886 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:04.974886 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:09:04.974886 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:04.974886 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:09:04.976560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:04.976560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:09:04.976560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:04.976560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:09:04.976560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:09:04.976560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:09:04.976560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:09:04.976560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:04.976560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:04.976560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:04.976560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:09:04.976560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:04.977144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:04.977144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:04.977144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:09:04.977144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:04.977144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:09:04.977144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:09:04.977144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:04.977144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:09:04.977144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:04.977144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:09:04.977144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:09:04.977144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:04.977144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:09:04.977732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:04.977732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:09:04.978301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:04.978301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:09:04.978301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:09:04.978301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:09:04.978301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:09:04.978301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:09:04.978301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:04.978301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:09:04.978301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:09:04.978301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:09:04.978301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:09:04.978301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:09:04.978301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:04.978301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:04.978301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:09:04.978301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:09:04.979592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:09:04.979592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:09:04.979592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:04.979592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:04.979592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:04.979592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:09:04.979592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:04.979592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:04.979592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:04.979592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:09:04.979592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:04.979592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:09:04.979592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:09:04.979592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:04.979592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:09:04.979592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:04.979592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:09:04.979592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:09:04.979592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:04.980078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:09:04.980078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:04.980078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:09:04.980078 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:09:04.980078 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:09:04.980078 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:09:04.980078 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:09:04.980078 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:04.980340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:04.980731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:09:04.980731 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:04.980731 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:09:04.980731 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:09:04.980731 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:04.980731 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:09:04.980731 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:04.980731 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:09:04.981472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:04.981472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:09:04.981472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:09:04.981472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:09:04.981472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:04.981472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:09:04.981472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:04.981472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:09:04.981472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:04.981472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:09:04.981472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:04.981472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:09:04.981472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:04.981472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:09:04.981472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:09:04.981896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:04.981896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:09:04.981896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:04.981896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:09:04.981896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:09:04.981896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:04.981896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:09:04.981896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:04.981896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:09:04.982252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:04.982252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:09:04.982252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:04.982252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:09:04.982252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:09:04.982252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:09:04.982252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:09:04.982252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:09:04.982252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:09:04.982252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:09:04.982252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:04.982252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:04.982606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:04.982606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:09:04.982606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:04.982606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:04.982606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:04.982606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:09:04.982606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:04.982606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:09:04.982606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:09:04.982606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:04.982606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:09:04.982606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:04.982606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:09:04.982606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:09:04.982978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:04.982978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:09:04.982978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:04.982978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:09:04.983253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:04.983253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:09:04.983253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:09:04.983253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:09:04.983253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:09:04.983253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:09:04.983253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:04.983253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:09:04.983253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:09:04.983253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:09:04.983253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:09:04.983253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:09:04.983253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:04.983253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:04.983253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:09:04.983253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:09:04.983686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:09:04.983686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:09:04.983686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:09:04.983686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:09:04.983686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:09:04.983686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:04.983686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:04.983686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:04.983686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:09:04.983686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:04.983686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:04.983686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:04.983686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:09:04.983686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:04.983686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:09:04.983686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:09:04.983686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:04.983686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:09:04.984180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:04.984180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:09:04.984180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:09:04.984180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:04.984180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:09:04.984180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:04.984180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:09:04.989477 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:09:04.990123 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:09:04.990123 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:09:04.990123 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:09:04.990521 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:04.990917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:04.993533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:09:04.993533 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:04.993533 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:09:04.993533 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:09:04.993533 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:04.993533 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:09:04.993533 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:04.993533 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:09:04.993533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:04.993533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:09:04.993533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:09:04.993533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:09:04.993533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:04.993533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:09:04.993533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:04.993533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:09:04.993533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:04.993533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:09:04.994113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:04.994113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:09:04.994113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:04.994113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:09:04.994113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:09:04.994113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:04.994113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:09:04.994113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:04.994113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:09:04.994113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:09:04.994113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:04.994113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:09:04.994113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:04.994113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:09:04.994113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:04.994545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:09:04.994545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:04.994545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:09:04.994545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:09:04.994545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:09:04.994545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:09:04.994545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:09:04.994545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:09:04.994545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:04.994545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:04.994545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:04.994545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:09:04.994545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:04.994904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:04.994904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:04.994904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:09:04.994904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:04.994904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:09:04.994904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:09:04.994904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:04.994904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:09:04.994904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:04.994904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:09:04.994904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:09:04.994904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:04.994904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:09:04.995233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:04.995233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:09:04.995233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:04.995233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:09:04.995233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:09:04.995233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:09:04.995233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:09:04.995233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:09:04.995233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:04.995233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:09:04.995233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:09:04.995233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:09:04.995233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:09:04.995233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:09:04.995233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:04.995697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:04.995697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:09:04.995697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:09:04.995697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:09:04.995697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:09:04.995697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:09:04.995697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:09:04.995697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:04.995697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:04.995697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:04.995697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:09:04.995697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:04.995697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:04.995697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:04.995697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:09:04.995697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:04.995697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:09:04.995697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:09:04.995697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:04.996168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:09:04.996168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:04.996168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:09:04.996168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:09:04.996168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:04.996168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:09:04.996168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:04.996168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:09:04.996364 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:09:04.997919 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:09:04.997919 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:09:04.997919 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:09:04.997919 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:04.997919 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:04.997919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:04.998423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:09:04.998423 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:04.998423 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:09:04.998423 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:09:04.998423 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:04.998423 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:09:04.998423 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:04.998423 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:09:04.999144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:04.999144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:09:04.999144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:04.999144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:09:04.999144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:09:04.999144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:09:04.999144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:09:04.999144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:04.999144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:04.999144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:04.999144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:09:04.999144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:04.999771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:04.999771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:04.999771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:09:04.999771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:04.999771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:09:04.999771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:09:04.999771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:04.999771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:09:04.999771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:04.999771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:09:04.999771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:09:04.999771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:04.999771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:09:05.000194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:05.000194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:09:05.000194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:05.000194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:09:05.000194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:09:05.000194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:09:05.000194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:09:05.000194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:09:05.000194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:05.000194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:09:05.000194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:09:05.000194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:09:05.000194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:09:05.000194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:09:05.000194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:05.000756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:05.000756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:09:05.000756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:09:05.000756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:09:05.000756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:09:05.000756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:05.000756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:05.000756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:05.000756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:09:05.000756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:05.000756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:05.000756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:05.000756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:09:05.000756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:05.000756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:09:05.000756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:09:05.000756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:05.000756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:09:05.000756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:05.001494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:09:05.001494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:09:05.001494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:05.001494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:09:05.001494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:05.001494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:09:09.286796 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:09:09.287867 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:09:09.288077 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:09:09.289754 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:09:09.331493 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:09.331933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:09.332975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:09:09.332975 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:09.332975 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:09:09.332975 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:09:09.332975 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:09.332975 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:09:09.332975 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:09.332975 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:09:09.334624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.045 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:09.334624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:09:09.334624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:09.334624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:09:09.334624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:09:09.334624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:09:09.334624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:09:09.334624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:09.334624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:09.334624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:09.334624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:09:09.334624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:09.335250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:09.335250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:09.335250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:09:09.335250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:09.335250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:09:09.335250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:09:09.335250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:09.335250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:09:09.335250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:09.335250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:09:09.335250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:09:09.335250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:09.335250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:09:09.335892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:09.335892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:09:09.336615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:09.336615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:09:09.336615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:09:09.336615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:09:09.336615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:09:09.336615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:09:09.336615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:09.336615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:09:09.336615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:09:09.336615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:09:09.336615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:09:09.336615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:09:09.336615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:09.336615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:09.336615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:09:09.336615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:09:09.337354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:09:09.337354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:09:09.337354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:09.337354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:09.337354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:09.337354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:09:09.337354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:09.337354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:09.337354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:09.337354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:09:09.337354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:09.337354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:09:09.337354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:09:09.337354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:09.337354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:09:09.337354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:09.337354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:09:09.337354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:09:09.337354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:09.338247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:09:09.338247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:09.338247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:09:09.971811 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:09:09.972697 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:09:09.972855 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:09:09.974434 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:09.974852 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:09.975092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:09.976067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:09:09.976067 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:09.976067 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:09:09.976067 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:09:09.976067 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:09.976067 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:09:09.976067 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:09.976067 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:09:09.977904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:09.977904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:09:09.977904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:09.977904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:09:09.977904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:09:09.977904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:09:09.977904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:09:09.977904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:09.977904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:09.977904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:09.977904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:09:09.977904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:09.978496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:09.978496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:09.978496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:09:09.978496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:09.978496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:09:09.978496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:09:09.978496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:09.978496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:09:09.978496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:09.978496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:09:09.978496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:09:09.978496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:09.978496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:09:09.979071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:09.979071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:09:09.979665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:09.979665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:09:09.979665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:09:09.979665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:09:09.979665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:09:09.979665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:09:09.979665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:09.979665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:09:09.979665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:09:09.979665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:09:09.979665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:09:09.979665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:09:09.979665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:09.979665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:09.979665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:09:09.979665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:09:09.980350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:09:09.980350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:09:09.980350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:09.980350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:09.980350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:09.980350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:09:09.980350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:09.980350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:09.980350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:09.980350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:09:09.980350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:09.980350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:09:09.980350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:09:09.980350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:09.980350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:09:09.980350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:09.980350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:09:09.980350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:09:09.980350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:09.981130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:09:09.981130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:09.981130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:09:09.981130 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:09:09.981130 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:09:09.981130 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:09:09.981130 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:09:09.981661 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:09.981982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:09.982949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:09:09.982949 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:09.982949 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:09:09.982949 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:09:09.982949 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:09.982949 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:09:09.982949 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:09.982949 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:09:09.984296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:09.984296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:09:09.984296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:09:09.984296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:09:09.984296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:09.984296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:09:09.984296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:09.984296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:09:09.984296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:09.984296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:09:09.984296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:09.984296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:09:09.984296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:09.984296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:09:09.984296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:09:09.984750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:09.984750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:09:09.984750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:09.984750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:09:09.984750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:09:09.984750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:09.984750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:09:09.984750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:09.984750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:09:09.985932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:09.985932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:09:09.985932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:09.985932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:09:09.985932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:09:09.985932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:09:09.985932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:09:09.985932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:09:09.985932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:09:09.985932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:09:09.985932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:09.985932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:09.986307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:09.986307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:09:09.986307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:09.986307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:09.986307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:09.986307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:09:09.986307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:09.986307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:09:09.986307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:09:09.986307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:09.986307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:09:09.986307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:09.986307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:09:09.986307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:09:09.986693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:09.986693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:09:09.986693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:09.986693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:09:09.987110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:09.987110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:09:09.987110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:09:09.987110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:09:09.987110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:09:09.987110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:09:09.987110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:09.987110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:09:09.987110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:09:09.987110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:09:09.987110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:09:09.987110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:09:09.987110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:09.987110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:09.987110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:09:09.987110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:09:09.987550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:09:09.987550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:09:09.987550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:09:09.987550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:09:09.987550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:09:09.987550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:09.987550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:09.987550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:09.987550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:09:09.987550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:09.987550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:09.987550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:09.987550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:09:09.987550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:09.987550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:09:09.987550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:09:09.987550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:09.987550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:09:09.988064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:09.988064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:09:09.988064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:09:09.988064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:09.988064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:09:09.988064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:09.988064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:09:09.990052 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:09:09.990389 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:09:09.990538 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:09:09.990715 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:09:09.991382 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:09.991835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:09.992646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:09:09.992646 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:09.992646 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:09:09.992646 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:09:09.992646 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:09.992646 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:09:09.992646 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:09.992646 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:09:09.993595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:09.993595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:09:09.993595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:09:09.993595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:09:09.993595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:09.993595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:09:09.993595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:09.993595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:09:09.993595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:09.993595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:09:09.993595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:09.993595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:09:09.993595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:09.993595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:09:09.993595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:09:09.993987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:09.993987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:09:09.993987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:09.993987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:09:09.993987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:09:09.993987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:09.993987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:09:09.993987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:09.993987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:09:09.994710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:09.994710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:09:09.994710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:09.994710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:09:09.994710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:09:09.994710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:09:09.994710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:09:09.994710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:09:09.994710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:09:09.994710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:09.994710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:09.995046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:09.995046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:09:09.995046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:09.995046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:09.995046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:09.995046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:09:09.995046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:09.995046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:09:09.995046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:09:09.995046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:09.995046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:09:09.995046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:09.995046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:09:09.995046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:09:09.995439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:09.995439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:09:09.995439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:09.995439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:09:09.996134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:09.996134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:09:09.996134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:09:09.996134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:09:09.996134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:09:09.996134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:09:09.996134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:09.996134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:09:09.996134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:09:09.996134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:09:09.996134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:09:09.996134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:09:09.996134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:09.996134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:09.996134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:09:09.996134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:09:09.996134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:09:09.996635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:09:09.996635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:09:09.996635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:09:09.996635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:09.996635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:09.996635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:09.996635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:09:09.996635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:09.996635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:09.996635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:09.996635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:09:09.996635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:09.996635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:09:09.996635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:09:09.996635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:09.996635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:09:09.996635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:09.996635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:09:09.996635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:09:09.997176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:09.997176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:09:09.997176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:09.997176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:09:09.997176 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:09:09.997176 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:09:09.997176 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:09:09.997429 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:09:09.997844 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:09.998225 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:09.998495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:09.999276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:09:09.999276 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:09.999276 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:09:09.999276 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:09:09.999276 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:09.999276 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:09:09.999276 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:09.999276 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:09:10.000141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:10.000141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:09:10.000141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:10.000141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:09:10.000141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:09:10.000141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:09:10.000141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:09:10.000141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:10.000141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:10.000141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:10.000141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:09:10.000141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:10.000541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:10.000541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:10.000541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:09:10.000541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:10.000541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:09:10.000541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:09:10.000541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:10.000541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:09:10.000541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:10.000541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:09:10.000541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:09:10.000541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:10.000541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:09:10.000911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:10.000911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:09:10.001161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:10.001161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:09:10.001161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:09:10.001161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:09:10.001161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:09:10.001161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:09:10.001161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:10.001161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:09:10.001161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:09:10.001161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:09:10.001161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:09:10.001161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:09:10.001161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:10.001161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:10.001161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:09:10.001161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:09:10.001645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:09:10.001645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:09:10.001645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:10.001645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:10.001645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:10.001645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:09:10.001645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:10.001645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:10.001645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:10.001645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:09:10.001645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:10.001645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:09:10.001645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:09:10.001645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:10.001645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:09:10.001645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:10.001645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:09:10.001645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:09:10.001645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:10.002159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:09:10.002159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:10.002159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:09:14.287955 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:09:14.288886 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:09:14.289138 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:09:14.290741 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:09:14.328523 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:14.328523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:14.330014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:09:14.330014 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:14.330014 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:09:14.330014 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:09:14.330014 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:14.330014 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:09:14.330014 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:14.330014 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:09:14.331886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:14.331886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:09:14.331886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:14.331886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:09:14.331886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:09:14.331886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:09:14.331886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:09:14.331886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:14.331886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:14.331886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:14.331886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:09:14.331886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:14.332450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:14.332450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:14.332450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:09:14.332450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:14.332450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:09:14.332450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:09:14.332450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:14.332450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:09:14.332450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:14.332450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:09:14.332450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:09:14.332450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:14.332450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:09:14.332952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:14.332952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:09:14.333076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:14.333076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:09:14.333076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:09:14.333076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:09:14.333076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:09:14.333076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:09:14.333076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:14.333076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:09:14.333076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:09:14.333076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:09:14.333076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:09:14.333076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:09:14.333076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:14.333076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:14.333076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:09:14.333076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:09:14.333748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:09:14.333748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:09:14.333748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:14.333748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:14.333748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:14.333748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:09:14.333748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:14.333748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:14.333748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:14.333748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:09:14.333748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:14.333748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:09:14.333748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:09:14.333748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:14.333748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:09:14.333748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:14.333748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:09:14.333748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:09:14.333748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:14.334586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:09:14.334586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:14.334586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:09:14.972559 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:09:14.973589 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:09:14.973589 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:09:14.974772 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:14.975116 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:14.975359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:14.976371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:09:14.976371 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:14.976371 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:09:14.976371 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:09:14.976371 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:14.976371 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:09:14.976371 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:14.976371 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:09:14.978074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:14.978074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:09:14.978074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:14.978074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:09:14.978074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:09:14.978074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:09:14.978074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:09:14.978074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:14.978074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:14.978074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:14.978074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:09:14.978074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:14.978726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:14.978726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:14.978726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:09:14.978726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:14.978726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:09:14.978726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:09:14.978726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:14.978726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:09:14.978726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:14.978726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:09:14.978726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:09:14.978726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:14.978726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:09:14.979354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:14.979354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:09:14.979852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:14.979852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:09:14.979852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:09:14.979852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:09:14.979852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:09:14.979852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:09:14.979852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:14.979852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:09:14.979852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:09:14.979852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:09:14.979852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:09:14.979852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:09:14.979852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:14.979852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:14.979852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:09:14.979852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:09:14.980704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:09:14.980704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:09:14.980704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:14.980704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:14.980704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:14.980704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:09:14.980704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:14.980704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:14.980704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:14.980704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:09:14.980704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:14.980704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:09:14.980704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:09:14.980704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:14.980704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:09:14.980704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:14.980704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:09:14.980704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:09:14.980704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:14.981462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:09:14.981462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:14.981462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:09:14.981462 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:09:14.981462 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:09:14.981462 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:09:14.981462 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:09:14.981864 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:14.982153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:14.982901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:09:14.982901 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:14.982901 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:09:14.982901 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:09:14.982901 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:14.982901 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:09:14.982901 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:14.982901 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:09:14.983674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:14.983674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:09:14.983674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:09:14.983674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:09:14.983674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:14.983674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:09:14.983674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:14.983674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:09:14.983674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:14.983674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:09:14.983674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:14.983674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:09:14.983674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:14.983674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:09:14.983674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:09:14.984103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:14.984103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:09:14.984103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:14.984103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:09:14.984103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:09:14.984103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:14.984103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:09:14.984103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:14.984103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:09:14.984471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:14.984471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:09:14.984471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:14.984471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:09:14.984471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:09:14.984471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:09:14.984471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:09:14.984471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:09:14.984471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:09:14.984471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:09:14.984471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:14.984471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:14.984873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:14.984873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:09:14.984873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:14.984873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:14.984873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:14.984873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:09:14.984873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:14.984873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:09:14.984873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:09:14.984873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:14.984873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:09:14.984873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:14.984873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:09:14.984873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:09:14.985267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:14.985267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:09:14.985267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:14.985267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:09:14.985549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:14.985549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:09:14.985549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:09:14.985549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:09:14.985549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:09:14.985549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:09:14.985549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:14.985549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:09:14.985549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:09:14.985549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:09:14.985549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:09:14.985549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:09:14.985549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:14.985549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:14.985549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:09:14.985549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:09:14.985994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:09:14.985994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:09:14.985994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:09:14.985994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:09:14.985994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:09:14.985994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:14.985994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:14.985994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:14.985994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:09:14.985994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:14.985994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:14.985994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:14.985994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:09:14.985994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:14.985994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:09:14.985994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:09:14.985994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:14.985994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:09:14.986529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:14.986529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:09:14.986529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:09:14.986529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:14.986529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:09:14.986529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:14.986529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:09:14.990524 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:09:14.990815 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:09:14.990927 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:09:14.991059 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:09:14.991524 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:14.991865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:14.992539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:09:14.992539 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:14.992539 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:09:14.992539 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:09:14.992539 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:14.992539 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:09:14.992539 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:14.992539 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:09:14.993299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:14.993299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:09:14.993299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:09:14.993299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:09:14.993299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:14.993299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:09:14.993299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:14.993299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:09:14.993299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:14.993299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:09:14.993299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:14.993299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:09:14.993299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:14.993299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:09:14.993299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:09:14.993743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:14.993743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:09:14.993743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:14.993743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:09:14.993743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:09:14.993743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:14.993743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:09:14.993743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:14.993743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:09:14.994159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:14.994159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:09:14.994159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:14.994159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:09:14.994159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:09:14.994159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:09:14.994159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:09:14.994159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:09:14.994159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:09:14.994159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:14.994159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:14.994504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:14.994504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:09:14.994504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:14.994504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:14.994504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:14.994504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:09:14.994504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:14.994504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:09:14.994504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:09:14.994504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:14.994504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:09:14.994504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:14.994504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:09:14.994504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:09:14.994920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:14.994920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:09:14.994920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:14.994920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:09:14.995196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:14.995196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:09:14.995196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:09:14.995196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:09:14.995196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:09:14.995196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:09:14.995196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:14.995196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:09:14.995196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:09:14.995196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:09:14.995196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:09:14.995196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:09:14.995196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:14.995196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:14.995196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:09:14.995196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:09:14.995196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:09:14.995687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:09:14.995687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:09:14.995687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:09:14.995687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:14.995687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:14.995687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:14.995687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:09:14.995687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:14.995687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:14.995687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:14.995687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:09:14.995687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:14.995687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:09:14.995687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:09:14.995687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:14.995687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:09:14.995687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:14.995687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:09:14.995687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:09:14.996215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:14.996215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:09:14.996215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:14.996215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:09:14.997180 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:09:14.997491 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:09:14.997618 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:09:14.997764 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:09:14.998154 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:14.998516 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:14.998729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:14.999346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:09:14.999346 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:14.999346 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:09:14.999346 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:09:14.999346 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:14.999346 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:09:14.999346 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:14.999346 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:09:15.000153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:15.000153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:09:15.000153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:15.000153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:09:15.000153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:09:15.000153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:09:15.000153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:09:15.000153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:15.000153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:15.000153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:15.000153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:09:15.000153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:15.000540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:15.000540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:15.000540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:09:15.000540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:15.000540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:09:15.000540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:09:15.000540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:15.000540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:09:15.000540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:15.000540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:09:15.000540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:09:15.000540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:15.000540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:09:15.000936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:15.000936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:09:15.001144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:15.001144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:09:15.001144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:09:15.001144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:09:15.001144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:09:15.001144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:09:15.001144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:15.001144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:09:15.001144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:09:15.001144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:09:15.001144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:09:15.001144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:09:15.001144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:15.001144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:15.001144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:09:15.001144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:09:15.001604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:09:15.001604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:09:15.001604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:15.001604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:15.001604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:15.001604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:09:15.001604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:15.001604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:15.001604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:15.001604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:09:15.001604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:15.001604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:09:15.001604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:09:15.001604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:15.001604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:09:15.001604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:15.001604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:09:15.001604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:09:15.001604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:15.002161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:09:15.002161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:15.002161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:09:19.288558 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:09:19.289076 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:09:19.289142 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:09:19.290399 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:09:19.315934 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:19.316208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:19.316752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:09:19.316752 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:19.316752 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:09:19.316752 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:09:19.316752 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:19.316752 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:09:19.316752 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:19.316752 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:09:19.317493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.028 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:19.317493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:09:19.317493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:19.317493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:09:19.317493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:09:19.317493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:09:19.317493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:09:19.317493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:19.317493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:19.317493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:19.317493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:09:19.317493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:19.318187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:19.318187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:19.318187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:09:19.318187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:19.318187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:09:19.318187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:09:19.318187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:19.318187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:09:19.318187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:19.318187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:09:19.318187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:09:19.318187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:19.318187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:09:19.318782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:19.318782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:09:19.318782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:19.318782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:09:19.318782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:09:19.318782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:09:19.318782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:09:19.318782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:09:19.318782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:19.318782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:09:19.318782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:09:19.318782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:09:19.318782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:09:19.318782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:09:19.318782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:19.319447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:19.319447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:09:19.319447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:09:19.319447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:09:19.319447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:09:19.319447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:19.319447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:19.319447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:19.319447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:09:19.319447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:19.319447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:19.319447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:19.319447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:09:19.319447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:19.319447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:09:19.319447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:09:19.319447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:19.319447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:09:19.319447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:19.320439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:09:19.320439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:09:19.320439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:19.320439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:09:19.320439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:19.320439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:09:19.973334 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:09:19.975173 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:09:19.975322 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:09:19.976178 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:19.976178 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:19.976321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:19.976810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:09:19.976810 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:19.976810 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:09:19.976810 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:09:19.976810 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:19.976810 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:09:19.976810 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:19.976810 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:09:19.978076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:19.978076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:09:19.978076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:19.978076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:09:19.978076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:09:19.978076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:09:19.978076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:09:19.978076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:19.978076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:19.978076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:19.978076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:09:19.978076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:19.978809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:19.978809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:19.978809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:09:19.978809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:19.978809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:09:19.978809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:09:19.978809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:19.978809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:09:19.978809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:19.978809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:09:19.978809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:09:19.978809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:19.978809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:09:19.979517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:19.979517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:09:19.979517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:19.979517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:09:19.979517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:09:19.979517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:09:19.979517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:09:19.979517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:09:19.979517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:19.979517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:09:19.979517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:09:19.979517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:09:19.979517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:09:19.979517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:09:19.979517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:19.980144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:19.980144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:09:19.980144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:09:19.980144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:09:19.980144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:09:19.980144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:19.980144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:19.980144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:19.980144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:09:19.980144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:19.980144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:19.980144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:19.980144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:09:19.980144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:19.980144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:09:19.980144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:09:19.980144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:19.980144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:09:19.980144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:19.980710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:09:19.980710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:09:19.980710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:19.980710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:09:19.980710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:19.980710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:09:19.981062 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:09:19.983975 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:09:19.984083 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:09:19.984219 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:09:19.985164 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:19.985567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:19.986326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:09:19.986326 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:19.986326 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:09:19.986326 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:09:19.986326 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:19.986326 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:09:19.986326 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:19.986326 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:09:19.987160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:19.987160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:09:19.987160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:09:19.987160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:09:19.987160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:19.987160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:09:19.987160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:19.987160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:09:19.987160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:19.987160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:09:19.987160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:19.987160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:09:19.987160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:19.987160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:09:19.987160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:09:19.988048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:19.988048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:09:19.988048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:19.988048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:09:19.988048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:09:19.988048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:19.988048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:09:19.988048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:19.988048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:09:19.988048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:19.988048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:09:19.988048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:19.988048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:09:19.988048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:09:19.988781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:09:19.988781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:09:19.988781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:09:19.988781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:09:19.988781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:09:19.988781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:19.988781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:19.988781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:19.988781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:09:19.988781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:19.988781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:19.988781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:19.988781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:09:19.989452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:19.989452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:09:19.989452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:09:19.989452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:19.989452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:09:19.989452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:19.989452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:09:19.989452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:09:19.989452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:19.989452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:09:19.989452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:19.989452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:09:19.990075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:19.990075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:09:19.990075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:09:19.990075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:09:19.990075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:09:19.990075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:09:19.990075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:19.990075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:09:19.990075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:09:19.990075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:09:19.990075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:09:19.990075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:09:19.990075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:19.990075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:19.990075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:09:19.990075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:09:19.991070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:09:19.991070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:09:19.991070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:09:19.991070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:09:19.991070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:09:19.991070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:19.991070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:19.991070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:19.991070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:09:19.991070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:19.991070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:19.991070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:19.991070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:09:19.991070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:19.991070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:09:19.991070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:09:19.991070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:19.991070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:09:19.991945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:19.991945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:09:19.991945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:09:19.991945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:19.991945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:09:19.991945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:19.991945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:09:19.991945 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:09:19.991945 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:09:19.991945 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:09:19.992397 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:09:19.992397 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:19.992397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:19.992583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:09:19.992583 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:19.992583 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:09:19.992583 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:09:19.992583 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:19.992583 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:09:19.992583 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:19.992583 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:09:19.993303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:19.993303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:09:19.993303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:09:19.993303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:09:19.993303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:19.993303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:09:19.993303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:19.993303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:09:19.993303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:19.993303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:09:19.993303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:19.993303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:09:19.993303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:19.993303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:09:19.993303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:09:19.994574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:19.994574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:09:19.994574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:19.994574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:09:19.994574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:09:19.994574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:19.994574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:09:19.994574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:19.994574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:09:19.994574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:19.994574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:09:19.994574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:19.994574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:09:19.994574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:09:19.995326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:09:19.995326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:09:19.995326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:09:19.995326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:09:19.995326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:19.995326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:19.995326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:19.995326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:09:19.995326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:19.995326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:19.995326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:19.995326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:09:19.995326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:19.995768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:09:19.995768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:09:19.995768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:19.995768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:09:19.995768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:19.995768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:09:19.995768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:09:19.995768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:19.995768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:09:19.995768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:19.995768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:09:19.995768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:19.996033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:09:19.996033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:09:19.996033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:09:19.996033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:09:19.996033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:09:19.996033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:19.996033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:09:19.996033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:09:19.996033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:09:19.996033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:09:19.996033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:09:19.996033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:19.996033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:19.996033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:09:19.996033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:09:19.996033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:09:19.996033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:09:19.996033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:09:19.996033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:09:19.996440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:19.996440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:19.996440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:19.996440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:09:19.996440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:19.996440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:19.996440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:19.996440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:09:19.996440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:19.996440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:09:19.996440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:09:19.996440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:19.996440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:09:19.996440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:19.996440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:09:19.996440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:09:19.996440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:19.996440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:09:19.996440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:19.996440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:09:19.997669 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:09:19.997988 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:09:19.998147 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:09:19.998304 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:09:19.998572 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:19.998772 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:19.998935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:19.999389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:09:19.999389 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:19.999389 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:09:19.999389 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:09:19.999389 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:19.999389 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:09:19.999389 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:19.999389 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:09:20.000180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:20.000180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:09:20.000180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:20.000180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:09:20.000180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:09:20.000180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:09:20.000180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:09:20.000180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:20.000180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:20.000180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:20.000180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:09:20.000180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:20.000509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:20.000509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:20.000509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:09:20.000509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:20.000509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:09:20.000509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:09:20.000509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:20.000509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:09:20.000509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:20.000509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:09:20.000509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:09:20.000509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:20.000509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:09:20.000770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:20.000770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:09:20.001139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:20.001139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:09:20.001139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:09:20.001139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:09:20.001139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:09:20.001139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:09:20.001139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:20.001139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:09:20.001139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:09:20.001139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:09:20.001139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:09:20.001139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:09:20.001139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:20.001139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:20.001139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:09:20.001139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:09:20.001487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:09:20.001487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:09:20.001487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:20.001487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:20.001487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:20.001487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:09:20.001487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:20.001487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:20.001487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:20.001487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:09:20.001487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:20.001487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:09:20.001487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:09:20.001487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:20.001487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:09:20.001487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:20.001487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:09:20.001487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:09:20.001487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:20.001880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:09:20.001880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:20.001880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:09:24.289142 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:09:24.289811 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:09:24.289974 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:09:24.291249 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:09:24.293081 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG dbcounter [-] [105581] Writing DB stats neutron:SELECT=240 {{(pid=105581) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 14:09:24.330864 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:24.331263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:24.332122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:09:24.332122 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:24.332122 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:09:24.332122 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:09:24.332122 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:24.332122 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:09:24.332122 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:24.332122 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:09:24.333107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:24.333107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:09:24.333107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:24.333107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:09:24.333107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:09:24.333107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:09:24.333107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:09:24.333107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:24.333107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:24.333107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:24.333107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:09:24.333107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:24.333807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:24.333807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:24.333807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:09:24.333807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:24.333807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:09:24.333807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:09:24.333807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:24.333807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:09:24.333807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:24.333807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:09:24.333807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:09:24.333807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:24.333807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:09:24.334500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:24.334500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:09:24.334500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:24.334500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:09:24.334500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:09:24.334500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:09:24.334500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:09:24.334500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:09:24.334500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:24.334500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:09:24.334500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:09:24.334500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:09:24.334500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:09:24.334500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:09:24.334500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:24.335174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:24.335174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:09:24.335174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:09:24.335174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:09:24.335174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:09:24.335174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:24.335174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:24.335174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:24.335174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:09:24.335174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:24.335174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:24.335174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:24.335174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:09:24.335174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:24.335174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:09:24.335174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:09:24.335174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:24.335174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:09:24.335174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:24.336210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:09:24.336210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:09:24.336210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:24.336210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:09:24.336210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:24.336210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:09:24.976211 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:09:24.976211 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:09:24.976211 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:09:24.977651 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:24.977651 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:24.977651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:24.979105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:09:24.979105 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:24.979105 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:09:24.979105 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:09:24.979105 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:24.979105 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:09:24.979105 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:24.979105 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:09:24.979870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:24.979870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:09:24.979870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:24.979870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:09:24.979870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:09:24.979870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:09:24.979870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:09:24.979870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:24.979870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:24.979870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:24.979870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:09:24.979870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:24.980222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:24.980222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:24.980222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:09:24.980222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:24.980222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:09:24.980222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:09:24.980222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:24.980222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:09:24.980222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:24.980222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:09:24.980222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:09:24.980222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:24.980222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:09:24.980602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:24.980602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:09:24.980602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:24.980602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:09:24.980602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:09:24.980602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:09:24.980602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:09:24.980602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:09:24.980602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:24.980602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:09:24.980602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:09:24.980602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:09:24.980602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:09:24.980602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:09:24.980602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:24.981006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:24.981006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:09:24.981006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:09:24.981006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:09:24.981006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:09:24.981006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:24.981006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:24.981006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:24.981006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:09:24.981006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:24.981006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:24.981006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:24.981006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:09:24.981006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:24.981006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:09:24.981006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:09:24.981006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:24.981006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:09:24.981006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:24.981500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:09:24.981500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:09:24.981500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:24.981500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:09:24.981500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:24.981500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:09:24.984152 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:09:24.984814 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:09:24.984814 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:09:24.984814 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:09:24.984933 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:24.985168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:24.985678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:09:24.985678 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:24.985678 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:09:24.985678 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:09:24.985678 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:24.985678 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:09:24.985678 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:24.985678 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:09:24.986289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:24.986289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:09:24.986289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:09:24.986289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:09:24.986289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:24.986289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:09:24.986289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:24.986289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:09:24.986289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:24.986289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:09:24.986289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:24.986289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:09:24.986289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:24.986289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:09:24.986289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:09:24.987446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:24.987446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:09:24.987446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:24.987446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:09:24.987446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:09:24.987446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:24.987446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:09:24.987446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:24.987446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:09:24.987446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:24.987446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:09:24.987446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:24.987446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:09:24.987446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:09:24.988005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:09:24.988005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:09:24.988005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:09:24.988005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:09:24.988005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:09:24.988005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:24.988005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:24.988005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:24.988005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:09:24.988005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:24.988005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:24.988005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:24.988005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:09:24.988356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:24.988356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:09:24.988356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:09:24.988356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:24.988356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:09:24.988356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:24.988356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:09:24.988356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:09:24.988356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:24.988356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:09:24.988356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:24.988356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:09:24.988683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:24.988683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:09:24.988683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:09:24.988683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:09:24.988683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:09:24.988683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:09:24.988683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:24.988683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:09:24.988683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:09:24.988683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:09:24.988683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:09:24.988683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:09:24.988683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:24.988683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:24.988683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:09:24.988683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:09:24.989068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:09:24.989068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:09:24.989068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:09:24.989068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:09:24.989068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:09:24.989068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:24.989068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:24.989068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:24.989068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:09:24.989068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:24.989068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:24.989068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:24.989068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:09:24.989068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:24.989068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:09:24.989068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:09:24.989068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:24.989068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:09:24.990136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:24.990136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:09:24.990136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:09:24.990136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:24.990136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:09:24.990136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:24.990136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:09:24.991370 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:09:24.991603 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:09:24.991674 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:09:24.991788 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:09:24.992131 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:24.992293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:24.992666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:09:24.992666 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:24.992666 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:09:24.992666 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:09:24.992666 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:24.992666 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:09:24.992666 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:24.992666 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:09:24.993171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:24.993171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:09:24.993171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:09:24.993171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:09:24.993171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:24.993171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:09:24.993171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:24.993171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:09:24.993171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:24.993171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:09:24.993171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:24.993171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:09:24.993171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:24.993171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:09:24.993171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:09:24.993570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:24.993570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:09:24.993570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:24.993570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:09:24.993570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:09:24.993570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:24.993570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:09:24.993570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:24.993570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:09:24.993795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:24.993795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:09:24.993795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:24.993795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:09:24.993795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:09:24.993795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:09:24.993795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:09:24.993795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:09:24.993795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:09:24.993795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:24.993795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:24.994221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:24.994221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:09:24.994221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:24.994221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:24.994221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:24.994221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:09:24.994221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:24.994221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:09:24.994221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:09:24.994221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:24.994221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:09:24.994221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:24.994221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:09:24.994221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:09:24.994590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:24.994590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:09:24.994590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:24.994590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:09:24.994590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:24.994590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:09:24.994590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:09:24.994590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:09:24.994590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:09:24.994590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:09:24.994590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:24.994590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:09:24.994590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:09:24.994590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:09:24.994946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:09:24.994946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:09:24.994946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:24.994946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:24.994946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:09:24.994946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:09:24.994946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:09:24.994946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:09:24.994946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:09:24.994946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:09:24.994946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:24.994946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:24.994946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:24.994946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:09:24.994946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:24.994946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:24.994946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:24.994946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:09:24.994946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:24.995396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:09:24.995396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:09:24.995396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:24.995396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:09:24.995396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:24.995396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:09:24.995396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:09:24.995396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:24.995396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:09:24.995396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:24.995396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:09:24.998049 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:09:24.998268 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:09:24.998325 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:09:24.998495 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:09:24.999071 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:24.999224 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:24.999321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:24.999700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:09:24.999700 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:24.999700 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:09:24.999700 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:09:24.999700 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:24.999700 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:09:24.999700 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:24.999700 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:09:25.000294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:25.000294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:09:25.000294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:25.000294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:09:25.000294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:09:25.000294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:09:25.000294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:09:25.000294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:25.000294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:25.000294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:25.000294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:09:25.000294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:25.000665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:25.000665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:25.000665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:09:25.000665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:25.000665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:09:25.000665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:09:25.000665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:25.000665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:09:25.000665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:25.000665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:09:25.000665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:09:25.000665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:25.000665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:09:25.000989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:25.000989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:09:25.000989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:25.000989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:09:25.000989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:09:25.000989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:09:25.000989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:09:25.000989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:09:25.000989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:25.000989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:09:25.000989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:09:25.000989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:09:25.000989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:09:25.000989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:09:25.000989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:25.001356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:25.001356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:09:25.001356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:09:25.001356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:09:25.001356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:09:25.001356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:25.001356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:25.001356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:25.001356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:09:25.001356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:25.001356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:25.001356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:25.001356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:09:25.001356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:25.001356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:09:25.001356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:09:25.001356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:25.001356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:09:25.001356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:25.001829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:09:25.001829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:09:25.001829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:25.001829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:09:25.001829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:25.001829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:09:29.290223 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:09:29.291488 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:09:29.291488 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:09:29.292525 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:09:29.326628 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:29.326960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:29.327849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:09:29.327849 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:29.327849 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:09:29.327849 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:09:29.327849 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:29.327849 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:09:29.327849 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:29.327849 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:09:29.328723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:29.328723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:09:29.328723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:29.328723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:09:29.328723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:09:29.328723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:09:29.328723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:09:29.328723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:29.328723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:29.328723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:29.328723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:09:29.328723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:29.329131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:29.329131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:29.329131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:09:29.329131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:29.329131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:09:29.329131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:09:29.329131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:29.329131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:09:29.329131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:29.329131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:09:29.329131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:09:29.329131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:29.329131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:09:29.329587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:29.329587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:09:29.329587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:29.329587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:09:29.329587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:09:29.329587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:09:29.329587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:09:29.329587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:09:29.329587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:29.329587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:09:29.329587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:09:29.329587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:09:29.329587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:09:29.329587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:09:29.329587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:29.330028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:29.330028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:09:29.330028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:09:29.330028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:09:29.330028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:09:29.330028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:29.330028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:29.330028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:29.330028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:09:29.330028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:29.330028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:29.330028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:29.330028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:09:29.330028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:29.330028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:09:29.330028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:09:29.330028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:29.330028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:09:29.330028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:29.330568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:09:29.330568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:09:29.330568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:29.330568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:09:29.330568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:29.330568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:09:29.976707 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:09:29.977341 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:09:29.977544 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:09:29.978936 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:29.979127 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:29.979270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:29.979860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:09:29.979860 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:29.979860 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:09:29.979860 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:09:29.979860 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:29.979860 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:09:29.979860 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:29.979860 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:09:29.981402 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:29.981402 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:09:29.981402 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:29.981402 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:09:29.981402 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:09:29.981402 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:09:29.981402 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:09:29.981402 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:29.981402 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:29.981402 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:29.981402 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:09:29.981402 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:29.982442 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:29.982442 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:29.982442 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:09:29.982442 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:29.982442 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:09:29.982442 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:09:29.982442 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:29.982442 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:09:29.982442 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:29.982442 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:09:29.982442 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:09:29.982442 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:29.982442 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:09:29.983199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:29.983199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:09:29.983199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:29.983199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:09:29.983199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:09:29.983199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:09:29.983199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:09:29.983199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:09:29.983199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:29.983199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:09:29.983199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:09:29.983199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:09:29.983199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:09:29.983199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:09:29.983199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:29.983674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:29.983674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:09:29.983674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:09:29.983674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:09:29.983674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:09:29.983674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:29.983674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:29.983674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:29.983674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:09:29.983674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:29.983674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:29.983674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:29.983674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:09:29.983674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:29.983674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:09:29.983674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:09:29.983674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:29.983674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:09:29.983674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:29.984240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:09:29.984240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:09:29.984240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:29.984240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:09:29.984240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:29.984240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:09:29.984627 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:09:29.985324 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:09:29.985324 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:09:29.985324 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:09:29.985824 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:29.986037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:29.986512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:09:29.986512 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:29.986512 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:09:29.986512 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:09:29.986512 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:29.986512 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:09:29.986512 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:29.986512 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:09:29.987210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:29.987210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:09:29.987210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:09:29.987210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:09:29.987210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:29.987210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:09:29.987210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:29.987210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:09:29.987210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:29.987210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:09:29.987210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:29.987210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:09:29.987210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:29.987210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:09:29.987210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:09:29.988037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:29.988037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:09:29.988037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:29.988037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:09:29.988037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:09:29.988037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:29.988037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:09:29.988037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:29.988037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:09:29.988037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:29.988037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:09:29.988037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:29.988037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:09:29.988037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:09:29.988534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:09:29.988534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:09:29.988534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:09:29.988534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:09:29.988534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:09:29.988534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:29.988534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:29.988534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:29.988534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:09:29.988534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:29.988534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:29.988534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:29.988534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:09:29.988924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:29.988924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:09:29.988924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:09:29.988924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:29.988924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:09:29.988924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:29.988924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:09:29.988924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:09:29.988924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:29.988924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:09:29.988924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:29.988924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:09:29.989271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:29.989271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:09:29.989271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:09:29.989271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:09:29.989271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:09:29.989271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:09:29.989271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:29.989271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:09:29.989271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:09:29.989271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:09:29.989271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:09:29.989271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:09:29.989271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:29.989271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:29.989271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:09:29.989271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:09:29.989736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:09:29.989736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:09:29.989736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:09:29.989736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:09:29.989736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:09:29.989736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:29.989736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:29.989736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:29.989736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:09:29.989736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:29.989736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:29.989736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:29.989736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:09:29.989736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:29.989736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:09:29.989736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:09:29.989736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:29.989736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:09:29.990243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:29.990243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:09:29.990243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:09:29.990243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:29.990243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:09:29.990243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:29.990243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:09:29.991832 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:09:29.992308 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:09:29.992308 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:09:29.992449 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:09:29.992707 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:29.992978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:29.993381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:09:29.993381 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:29.993381 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:09:29.993381 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:09:29.993381 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:29.993381 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:09:29.993381 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:29.993381 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:09:29.994110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:29.994110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:09:29.994110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:09:29.994110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:09:29.994110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:29.994110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:09:29.994110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:29.994110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:09:29.994110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:29.994110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:09:29.994110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:29.994110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:09:29.994110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:29.994110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:09:29.994110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:09:29.994440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:29.994440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:09:29.994440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:29.994440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:09:29.994440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:09:29.994440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:29.994440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:09:29.994440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:29.994440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:09:29.994986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:29.994986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:09:29.994986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:29.994986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:09:29.994986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:09:29.994986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:09:29.994986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:09:29.994986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:09:29.994986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:09:29.994986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:29.994986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:29.995532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:29.995532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:09:29.995532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:29.995532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:29.995532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:29.995532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:09:29.995532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:29.995532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:09:29.995532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:09:29.995532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:29.995532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:09:29.995532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:29.995532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:09:29.995532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:09:29.996104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:29.996104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:09:29.996104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:29.996104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:09:29.996104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:29.996104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:09:29.996104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:09:29.996104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:09:29.996104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:09:29.996104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:09:29.996104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:29.996104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:09:29.996104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:09:29.996104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:09:29.996466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:09:29.996466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:09:29.996466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:29.996466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:29.996466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:09:29.996466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:09:29.996466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:09:29.996466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:09:29.996466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:09:29.996466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:09:29.996466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:29.996466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:29.996466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:29.996466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:09:29.996466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:29.996466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:29.996466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:29.996466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:09:29.996466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:29.996802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:09:29.996802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:09:29.996802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:29.996802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:09:29.996802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:29.996802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:09:29.996802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:09:29.996802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:29.996802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:09:29.996802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:29.996802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:09:29.998488 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:09:29.998796 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:09:29.998901 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:09:29.999042 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:09:29.999463 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:29.999826 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:30.000081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:30.000763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:09:30.000763 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:30.000763 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:09:30.000763 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:09:30.000763 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:30.000763 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:09:30.000763 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:30.000763 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:09:30.001571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:30.001571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:09:30.001571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:30.001571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:09:30.001571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:09:30.001571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:09:30.001571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:09:30.001571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:30.001571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:30.001571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:30.001571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:09:30.001571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:30.001888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:30.001888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:30.001888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:09:30.001888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:30.001888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:09:30.001888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:09:30.001888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:30.001888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:09:30.001888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:30.001888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:09:30.001888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:09:30.001888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:30.001888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:09:30.002301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:30.002301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:09:30.002678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:30.002678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:09:30.002678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:09:30.002678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:09:30.002678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:09:30.002678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:09:30.002678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:30.002678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:09:30.002678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:09:30.002678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:09:30.002678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:09:30.002678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:09:30.002678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:30.002678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:30.002678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:09:30.002678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:09:30.004025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:09:30.004025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:09:30.004025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:30.004025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:30.004025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:30.004025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:09:30.004025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:30.004025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:30.004025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:30.004025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:09:30.004025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:30.004025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:09:30.004025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:09:30.004025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:30.004025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:09:30.004025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:30.004025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:09:30.004025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:09:30.004025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:30.004596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:09:30.004596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:30.004596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:09:34.268620 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:09:34.269338 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_for_inconsistencies {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:09:34.269574 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Checking Neutron and OVN revision consistency. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:09:34.279241 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Synchronizing Neutron and OVN databases started {{(pid=105581) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:440}}
May 09 14:09:34.279482 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Number of inconsistencies found at create/update: networks=4, security_groups=3, routers=1, router_ports=1 {{(pid=105581) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:419}}
May 09 14:09:34.279652 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Fixing resource 0b4a5724-2518-4e33-a6d3-0d92c749f303 (type: networks) at create/update {{(pid=105581) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 14:09:34.290765 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:09:34.314518 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-0b4a5724-2518-4e33-a6d3-0d92c749f303, resource={'id': '0b4a5724-2518-4e33-a6d3-0d92c749f303', 'name': 'private', 'tenant_id': '5796380176014a32af8ff613b146faed', 'project_id': '5796380176014a32af8ff613b146faed', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['23eb2e2e-ec2f-4f98-bc47-7125a2f92952', 'db2636e9-0880-488d-9aa3-e100ce5eeb9f'], 'standard_attr_id': 12, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-09T14:04:20Z', 'updated_at': '2026-05-09T14:04:23Z', 'revision_number': 3, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 52278}, resource_type=networks, if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:34.314845 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-0b4a5724-2518-4e33-a6d3-0d92c749f303, col_values=(('external_ids', {'neutron:network_name': 'private', 'neutron:mtu': '1372', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:34.315236 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-f71b8cb1-fe70-438c-84f9-bbb7008a596d, external_ids_update=None, columns={'tag': 52278}, if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:34.315350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:34.316054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:09:34.316054 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:34.316054 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:09:34.316054 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:09:34.316054 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:34.316054 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:09:34.316054 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:34.316054 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:09:34.317459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Failed to fix resource 0b4a5724-2518-4e33-a6d3-0d92c749f303 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:34.317459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:09:34.317459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 14:09:34.317459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 14:09:34.317459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 14:09:34.317459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 14:09:34.317459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 09 14:09:34.317459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:09:34.317459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:34.317459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:34.317459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:34.317796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:09:34.317796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:34.317796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:34.317796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:34.317796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:09:34.317796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:34.317796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:09:34.317796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:09:34.317796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:34.317796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:09:34.317796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:34.317796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:09:34.317796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:09:34.317796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:34.318157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:09:34.318157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:34.318157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:09:34.318157 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Fixing resource a7d094cd-23f3-4e0c-9e88-24be6ee17b2c (type: networks) at create/update {{(pid=105581) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 14:09:34.346176 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-a7d094cd-23f3-4e0c-9e88-24be6ee17b2c, resource={'id': 'a7d094cd-23f3-4e0c-9e88-24be6ee17b2c', 'name': 'public', 'tenant_id': '5796380176014a32af8ff613b146faed', 'project_id': '5796380176014a32af8ff613b146faed', 'admin_state_up': True, 'mtu': 1430, 'status': 'ACTIVE', 'subnets': ['2f0821e3-c6a2-49d6-80ee-3480a17ba8d2', '644269b7-6b84-4d8a-ad28-d59a7c6b7c5e'], 'standard_attr_id': 18, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': True, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'is_default': True, 'tags': [], 'created_at': '2026-05-09T14:04:26Z', 'updated_at': '2026-05-09T14:04:34Z', 'revision_number': 3, 'provider:network_type': 'flat', 'provider:physical_network': 'public', 'provider:segmentation_id': None}, resource_type=networks, if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:34.346469 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-a7d094cd-23f3-4e0c-9e88-24be6ee17b2c, col_values=(('external_ids', {'neutron:network_name': 'public', 'neutron:mtu': '1430', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'flat', 'neutron:provnet-physical-network': 'public'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false', 'fdb_age_threshold': '300', 'broadcast-arps-to-all-routers': 'true'})), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:34.346846 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-08fea115-9b05-4667-a0d0-487c303084a5, external_ids_update=None, columns={'tag': []}, if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:34.347086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:34.347630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:09:34.347630 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:34.347630 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:09:34.347630 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:09:34.347630 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:34.347630 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:09:34.347630 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:34.347630 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:09:34.348779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Failed to fix resource a7d094cd-23f3-4e0c-9e88-24be6ee17b2c (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:34.348779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:09:34.348779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 14:09:34.348779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 14:09:34.348779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 14:09:34.348779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 14:09:34.348779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 09 14:09:34.348779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:09:34.348779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:34.348779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:34.348779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:34.349292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:09:34.349292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:34.349292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:34.349292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:34.349292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:09:34.349292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:34.349292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:09:34.349292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:09:34.349292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:34.349292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:09:34.349292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:34.349292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:09:34.349292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:09:34.349292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:34.349858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:09:34.349858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:34.349858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:09:34.349858 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Fixing resource 3e80d551-6de5-4370-806b-36d1db5f6fda (type: networks) at create/update {{(pid=105581) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 14:09:34.377288 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-3e80d551-6de5-4370-806b-36d1db5f6fda, resource={'id': '3e80d551-6de5-4370-806b-36d1db5f6fda', 'name': 'admin_net', 'tenant_id': '590c644b03ae4bed862ecf492c6b2eb2', 'project_id': '590c644b03ae4bed862ecf492c6b2eb2', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['453849e5-22a7-43ca-911e-9401ed2b0d74'], 'standard_attr_id': 42, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-09T14:07:41Z', 'updated_at': '2026-05-09T14:07:42Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 35952}, resource_type=networks, if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:34.377692 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-3e80d551-6de5-4370-806b-36d1db5f6fda, col_values=(('external_ids', {'neutron:network_name': 'admin_net', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:34.378120 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-f84050fb-ff65-46ce-b744-016e1ff15440, external_ids_update=None, columns={'tag': 35952}, if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:34.378256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:34.378936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:09:34.378936 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:34.378936 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:09:34.378936 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:09:34.378936 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:34.378936 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:09:34.378936 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:34.378936 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:09:34.379819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Failed to fix resource 3e80d551-6de5-4370-806b-36d1db5f6fda (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:34.379819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:09:34.379819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 14:09:34.379819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 14:09:34.379819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 14:09:34.379819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 14:09:34.379819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 09 14:09:34.379819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:09:34.379819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:34.379819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:34.379819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:34.380297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:09:34.380297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:34.380297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:34.380297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:34.380297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:09:34.380297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:34.380297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:09:34.380297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:09:34.380297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:34.380297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:09:34.380297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:34.380297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:09:34.380297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:09:34.380297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:34.380871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:09:34.380871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:34.380871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:09:34.380871 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Fixing resource 632a2ddb-5634-4bb3-aad0-f05d97abd45e (type: networks) at create/update {{(pid=105581) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 14:09:34.407146 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-632a2ddb-5634-4bb3-aad0-f05d97abd45e, resource={'id': '632a2ddb-5634-4bb3-aad0-f05d97abd45e', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-5urlz', 'tenant_id': '1f17f4a2a7ac4543a1253522e61993d2', 'project_id': '1f17f4a2a7ac4543a1253522e61993d2', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['32b523fe-135f-4f9e-a98b-fc0537fcdb35'], 'standard_attr_id': 52, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-09T14:08:15Z', 'updated_at': '2026-05-09T14:08:16Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 39245}, resource_type=networks, if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:34.407395 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-632a2ddb-5634-4bb3-aad0-f05d97abd45e, col_values=(('external_ids', {'neutron:network_name': 'k8s-clusterapi-cluster-magnum-system-kube-5urlz', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:34.407879 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-bca44f95-fe96-432e-84b8-1edc05e10ce0, external_ids_update=None, columns={'tag': 39245}, if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:34.408010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:34.408784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:09:34.408784 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:34.408784 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:09:34.408784 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:09:34.408784 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:34.408784 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:09:34.408784 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:34.408784 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:09:34.409679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Failed to fix resource 632a2ddb-5634-4bb3-aad0-f05d97abd45e (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:34.409679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:09:34.409679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 14:09:34.409679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 14:09:34.409679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 14:09:34.409679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 14:09:34.409679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 09 14:09:34.409679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:09:34.409679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:34.409679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:34.409679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:34.410165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:09:34.410165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:34.410165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:34.410165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:34.410165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:09:34.410165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:34.410165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:09:34.410165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:09:34.410165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:34.410165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:09:34.410165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:34.410165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:09:34.410165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:09:34.410165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:34.410712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:09:34.410712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:34.410712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:09:34.410712 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Fixing resource c7e878e1-076f-45f6-8a69-36d3ccc7662d (type: security_groups) at create/update {{(pid=105581) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 14:09:34.463592 np0000173183 neutron-ovn-maintenance-worker[105581]: INFO neutron.db.ovn_revision_numbers_db [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Successfully bumped revision number for resource c7e878e1-076f-45f6-8a69-36d3ccc7662d (type: security_groups) to 9
May 09 14:09:34.463731 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Fixing resource 25fc319c-33b9-4943-846b-abe7e37b483f (type: security_groups) at create/update {{(pid=105581) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 14:09:34.498283 np0000173183 neutron-ovn-maintenance-worker[105581]: INFO neutron.db.ovn_revision_numbers_db [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Successfully bumped revision number for resource 25fc319c-33b9-4943-846b-abe7e37b483f (type: security_groups) to 10
May 09 14:09:34.498454 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Fixing resource 8c1430f7-e302-42a6-b93c-6a32b8df2ee0 (type: security_groups) at create/update {{(pid=105581) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 14:09:34.542693 np0000173183 neutron-ovn-maintenance-worker[105581]: INFO neutron.db.ovn_revision_numbers_db [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Successfully bumped revision number for resource 8c1430f7-e302-42a6-b93c-6a32b8df2ee0 (type: security_groups) to 15
May 09 14:09:34.542693 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Fixing resource 09e042cf-39a9-42c8-b683-b29ffc42b049 (type: routers) at create/update {{(pid=105581) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 14:09:34.642378 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, resource={'id': '09e042cf-39a9-42c8-b683-b29ffc42b049', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-5urlz', 'project_id': '1f17f4a2a7ac4543a1253522e61993d2', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': 'a7d094cd-23f3-4e0c-9e88-24be6ee17b2c', 'external_fixed_ips': [{'subnet_id': '2f0821e3-c6a2-49d6-80ee-3480a17ba8d2', 'ip_address': '172.24.5.175'}, {'subnet_id': '644269b7-6b84-4d8a-ad28-d59a7c6b7c5e', 'ip_address': '2001:db8::d8'}], 'enable_snat': True}, 'gw_port_id': '71ded037-e812-40c0-8d8f-70c141c74323', 'description': 'Created by cluster-api-provider-openstack cluster magnum-system-kube-5urlz', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': 'a7d094cd-23f3-4e0c-9e88-24be6ee17b2c', 'external_fixed_ips': [{'ip_address': '172.24.5.175', 'subnet_id': '2f0821e3-c6a2-49d6-80ee-3480a17ba8d2'}, {'ip_address': '2001:db8::d8', 'subnet_id': '644269b7-6b84-4d8a-ad28-d59a7c6b7c5e'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2026-05-09T14:08:17Z', 'updated_at': '2026-05-09T14:08:19Z', 'revision_number': 3, 'tenant_id': '1f17f4a2a7ac4543a1253522e61993d2'}, resource_type=routers, if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:34.642831 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, columns={'external_ids': {'neutron:router_name': 'k8s-clusterapi-cluster-magnum-system-kube-5urlz', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:34.643180 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-a7d094cd-23f3-4e0c-9e88-24be6ee17b2c, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "71ded037-e812-40c0-8d8f-70c141c74323"')], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:34.643446 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-a7d094cd-23f3-4e0c-9e88-24be6ee17b2c, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "71ded037-e812-40c0-8d8f-70c141c74323"')], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:34.643657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:34.644561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:09:34.644561 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:34.644561 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:09:34.644561 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:09:34.644561 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:34.644561 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:09:34.644561 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:34.644561 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:09:34.644957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Unable to update router 09e042cf-39a9-42c8-b683-b29ffc42b049. Error: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:34.646057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Failed to fix resource 09e042cf-39a9-42c8-b683-b29ffc42b049 (type: routers): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:34.646057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:09:34.646057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 14:09:34.646057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 14:09:34.646057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 14:09:34.646057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 14:09:34.646057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1635, in update_router
May 09 14:09:34.646057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with excutils.save_and_reraise_exception():
May 09 14:09:34.646057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 271, in __exit__
May 09 14:09:34.646057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.force_reraise()
May 09 14:09:34.646057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 233, in force_reraise
May 09 14:09:34.646610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise self.value
May 09 14:09:34.646610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1569, in update_router
May 09 14:09:34.646610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:09:34.646610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:34.646610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:34.646610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:34.646610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:09:34.646610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:34.646610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:34.646610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:34.646610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:09:34.646610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:34.646610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:09:34.646610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:09:34.647205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:34.647205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:09:34.647205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:34.647205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:09:34.647205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:09:34.647205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:34.647205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:09:34.647205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:34.647205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:09:34.647205 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Fixing resource 71ded037-e812-40c0-8d8f-70c141c74323 (type: router_ports) at create/update {{(pid=105581) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 14:09:34.762682 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-71ded037-e812-40c0-8d8f-70c141c74323, resource={'id': '71ded037-e812-40c0-8d8f-70c141c74323', 'name': '', 'network_id': 'a7d094cd-23f3-4e0c-9e88-24be6ee17b2c', 'project_id': '', 'mac_address': 'fa:16:3e:c7:86:37', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '09e042cf-39a9-42c8-b683-b29ffc42b049', 'device_owner': 'network:router_gateway', 'standard_attr_id': 57, 'fixed_ips': [{'subnet_id': '2f0821e3-c6a2-49d6-80ee-3480a17ba8d2', 'ip_address': '172.24.5.175'}, {'subnet_id': '644269b7-6b84-4d8a-ad28-d59a7c6b7c5e', 'ip_address': '2001:db8::d8'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0000173183', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'ovs_create_tap': False, 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2026-05-09T14:08:17Z', 'updated_at': '2026-05-09T14:08:19Z', 'revision_number': 5, 'tenant_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:34.763015 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-71ded037-e812-40c0-8d8f-70c141c74323, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': '2f0821e3-c6a2-49d6-80ee-3480a17ba8d2 644269b7-6b84-4d8a-ad28-d59a7c6b7c5e', 'neutron:network_name': 'neutron-a7d094cd-23f3-4e0c-9e88-24be6ee17b2c', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-09e042cf-39a9-42c8-b683-b29ffc42b049'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['2001:db8::d8/64', '172.24.5.175/24'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:34.763366 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=71ded037-e812-40c0-8d8f-70c141c74323, lrouter_port=lrp-71ded037-e812-40c0-8d8f-70c141c74323, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:34.763721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:34.764439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:09:34.764439 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:34.764439 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:09:34.764439 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:09:34.764439 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:34.764439 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:09:34.764439 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:34.764439 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:09:34.765280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Failed to fix resource 71ded037-e812-40c0-8d8f-70c141c74323 (type: router_ports): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:34.765280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:09:34.765280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 14:09:34.765280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 14:09:34.765280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 14:09:34.765280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 14:09:34.765280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1976, in update_router_port
May 09 14:09:34.765280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:09:34.765280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:34.765280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:34.765280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:34.766213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:09:34.766213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:34.766213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:34.766213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:34.766213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:09:34.766213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:34.766213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:09:34.766213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:09:34.766213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:34.766213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:09:34.766213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:34.766213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:09:34.766213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:09:34.766213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:34.766811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:09:34.766811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:34.766811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:09:34.766811 np0000173183 neutron-ovn-maintenance-worker[105581]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_for_inconsistencies finished in 0.496 seconds
May 09 14:09:34.766811 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:09:34.766811 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:09:34.766811 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:09:34.796248 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:34.796593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:34.797295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:09:34.797295 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:34.797295 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:09:34.797295 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:09:34.797295 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:34.797295 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:09:34.797295 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:34.797295 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:09:34.797952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:34.797952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:09:34.797952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:34.797952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:09:34.797952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:09:34.797952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:09:34.797952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:09:34.797952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:34.797952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:34.797952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:34.797952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:09:34.797952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:34.798618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:34.798618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:34.798618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:09:34.798618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:34.798618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:09:34.798618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:09:34.798618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:34.798618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:09:34.798618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:34.798618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:09:34.798618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:09:34.798618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:34.798618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:09:34.799166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:34.799166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:09:34.799166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:34.799166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:09:34.799166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:09:34.799166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:09:34.799166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:09:34.799166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:09:34.799166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:34.799166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:09:34.799166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:09:34.799166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:09:34.799166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:09:34.799166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:09:34.799166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:34.799838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:34.799838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:09:34.799838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:09:34.799838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:09:34.799838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:09:34.799838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:34.799838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:34.799838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:34.799838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:09:34.799838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:34.799838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:34.799838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:34.799838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:09:34.799838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:34.799838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:09:34.799838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:09:34.799838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:34.799838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:09:34.799838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:34.800629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:09:34.800629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:09:34.800629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:34.800629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:09:34.800629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:34.800629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:09:34.977443 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:09:34.978091 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:09:34.978300 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:09:34.979452 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:34.979792 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:34.980055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:34.980944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:09:34.980944 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:34.980944 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:09:34.980944 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:09:34.980944 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:34.980944 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:09:34.980944 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:34.980944 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:09:34.982673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:34.982673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:09:34.982673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:34.982673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:09:34.982673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:09:34.982673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:09:34.982673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:09:34.982673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:34.982673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:34.982673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:34.982673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:09:34.982673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:34.983530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:34.983530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:34.983530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:09:34.983530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:34.983530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:09:34.983530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:09:34.983530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:34.983530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:09:34.983530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:34.983530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:09:34.983530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:09:34.983530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:34.983530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:09:34.984665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:34.984665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:09:34.984665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:34.984665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:09:34.984665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:09:34.984665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:09:34.984665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:09:34.984665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:09:34.984665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:34.984665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:09:34.984665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:09:34.984665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:09:34.984665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:09:34.984665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:09:34.984665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:34.986087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:34.986087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:09:34.986087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:09:34.986087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:09:34.986087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:09:34.986087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:34.986087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:34.986087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:34.986087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:09:34.986087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:34.986087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:34.986087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:34.986087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:09:34.986087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:34.986087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:09:34.986087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:09:34.986087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:34.986087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:09:34.986087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:34.987434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:09:34.987434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:09:34.987434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:34.987434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:09:34.987434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:34.987434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:09:34.987434 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:09:34.987434 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:09:34.987434 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:09:34.987434 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:09:34.988545 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:34.988545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:34.988545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:09:34.988545 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:34.988545 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:09:34.988545 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:09:34.988545 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:34.988545 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:09:34.988545 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:34.988545 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:09:34.988545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:34.988545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:09:34.988545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:09:34.988545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:09:34.989293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:34.989293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:09:34.989293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:34.989293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:09:34.989293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:34.989293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:09:34.989293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:34.989293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:09:34.989293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:34.989293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:09:34.989293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:09:34.989293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:34.989293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:09:34.989293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:34.989293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:09:34.989293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:09:34.989293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:34.989293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:09:34.990098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:34.990098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:09:34.990098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:34.990098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:09:34.990098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:34.990098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:09:34.990098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:09:34.990098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:09:34.990098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:09:34.990098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:09:34.990098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:09:34.990098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:09:34.991645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:34.991645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:34.991645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:34.991645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:09:34.991645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:34.991645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:34.991645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:34.991645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:09:34.991645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:34.991645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:09:34.991645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:09:34.991645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:34.991645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:09:34.991645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:34.992883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:09:34.992883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:09:34.992883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:34.992883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:09:34.992883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:34.992883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:09:34.992883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:34.992883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:09:34.992883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:09:34.992883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:09:34.992883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:09:34.992883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:09:34.992883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:34.992883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:09:34.993677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:09:34.993677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:09:34.993677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:09:34.993677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:09:34.993677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:34.993677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:34.993677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:09:34.993677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:09:34.993677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:09:34.993677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:09:34.993677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:09:34.993677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:09:34.993677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:09:34.993677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:34.993677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:34.993677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:34.993677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:09:34.993677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:34.993677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:34.994249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:34.994249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:09:34.994249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:34.994249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:09:34.994249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:09:34.994249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:34.994249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:09:34.994249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:34.994249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:09:34.994249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:09:34.994249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:34.994249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:09:34.994249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:34.994249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:09:34.994249 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:09:34.994561 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:09:34.994561 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:09:34.994561 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:09:34.994561 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:34.994561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:34.994561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:09:34.994561 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:34.994561 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:09:34.994561 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:09:34.994776 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:34.994776 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:09:34.994776 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:34.994776 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:09:34.994776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:34.994776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:09:34.994776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:09:34.994776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:09:34.994776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:34.994776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:09:34.994776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:34.994776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:09:34.994776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:34.994776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:09:34.994776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:34.994776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:09:34.995047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:34.995047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:09:34.995047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:09:34.995047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:34.995047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:09:34.995047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:34.995047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:09:34.995047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:09:34.995047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:34.995047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:09:34.995047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:34.995047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:09:34.995259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:34.995259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:09:34.995259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:34.995259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:09:34.995259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:09:34.995259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:09:34.995259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:09:34.995259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:09:34.995259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:09:34.995259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:34.995259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:34.995999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:34.995999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:09:34.995999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:34.995999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:34.995999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:34.995999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:09:34.995999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:34.995999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:09:34.995999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:09:34.995999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:34.995999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:09:34.995999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:34.995999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:09:34.995999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:09:34.996672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:34.996672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:09:34.996672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:34.996672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:09:34.996672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:34.996672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:09:34.996672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:09:34.996672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:09:34.996672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:09:34.996672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:09:34.996672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:34.996672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:09:34.996672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:09:34.996672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:09:34.997301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:09:34.997301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:09:34.997301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:34.997301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:34.997301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:09:34.997301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:09:34.997301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:09:34.997301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:09:34.997301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:09:34.997301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:09:34.997301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:34.997301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:34.997301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:34.997301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:09:34.997301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:34.997301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:34.997301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:34.997301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:09:34.997301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:34.997968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:09:34.997968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:09:34.997968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:34.997968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:09:34.997968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:34.997968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:09:34.997968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:09:34.997968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:34.997968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:09:34.997968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:34.997968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:09:34.998866 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:09:34.999077 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:09:34.999161 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:09:34.999274 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:09:34.999612 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:34.999969 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:35.000115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:35.000620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:09:35.000620 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:35.000620 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:09:35.000620 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:09:35.000620 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:35.000620 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:09:35.000620 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:35.000620 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:09:35.001391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:35.001391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:09:35.001391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:35.001391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:09:35.001391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:09:35.001391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:09:35.001391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:09:35.001391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:35.001391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:35.001391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:35.001391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:09:35.001391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:35.001646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:35.001646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:35.001646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:09:35.001646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:35.001646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:09:35.001646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:09:35.001646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:35.001646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:09:35.001646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:35.001646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:09:35.001646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:09:35.001646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:35.001646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:09:35.001941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:35.001941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:09:35.002346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:35.002346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:09:35.002346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:09:35.002346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:09:35.002346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:09:35.002346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:09:35.002346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:35.002346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:09:35.002346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:09:35.002346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:09:35.002346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:09:35.002346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:09:35.002346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:35.002346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:35.002346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:09:35.002346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:09:35.002712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:09:35.002712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:09:35.002712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:35.002712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:35.002712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:35.002712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:09:35.002712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:35.002712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:35.002712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:35.002712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:09:35.002712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:35.002712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:09:35.002712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:09:35.002712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:35.002712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:09:35.002712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:35.002712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:09:35.002712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:09:35.002712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:35.003034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:09:35.003034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:35.003034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:09:39.765963 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:09:39.766702 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:09:39.766856 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:09:39.768045 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:09:39.803323 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:39.803855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:39.804604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:09:39.804604 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:39.804604 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:09:39.804604 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:09:39.804604 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:39.804604 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:09:39.804604 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:39.804604 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:09:39.805321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:39.805321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:09:39.805321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:39.805321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:09:39.805321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:09:39.805321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:09:39.805321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:09:39.805321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:39.805321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:39.805321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:39.805321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:09:39.805321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:39.805975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:39.805975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:39.805975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:09:39.805975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:39.805975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:09:39.805975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:09:39.805975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:39.805975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:09:39.805975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:39.805975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:09:39.805975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:09:39.805975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:39.805975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:09:39.806537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:39.806537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:09:39.806537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:39.806537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:09:39.806537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:09:39.806537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:09:39.806537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:09:39.806537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:09:39.806537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:39.806537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:09:39.806537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:09:39.806537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:09:39.806537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:09:39.806537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:09:39.806537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:39.807204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:39.807204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:09:39.807204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:09:39.807204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:09:39.807204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:09:39.807204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:39.807204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:39.807204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:39.807204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:09:39.807204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:39.807204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:39.807204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:39.807204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:09:39.807204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:39.807204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:09:39.807204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:09:39.807204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:39.807204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:09:39.807204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:39.807990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:09:39.807990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:09:39.807990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:39.807990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:09:39.807990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:39.807990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:09:39.978080 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:09:39.978505 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:09:39.978634 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:09:39.979503 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:39.979800 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:39.980004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:39.980711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:09:39.980711 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:39.980711 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:09:39.980711 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:09:39.980711 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:39.980711 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:09:39.980711 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:39.980711 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:09:39.981680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:39.981680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:09:39.981680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:39.981680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:09:39.981680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:09:39.981680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:09:39.981680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:09:39.981680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:39.981680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:39.981680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:39.981680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:09:39.981680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:39.982102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:39.982102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:39.982102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:09:39.982102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:39.982102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:09:39.982102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:09:39.982102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:39.982102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:09:39.982102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:39.982102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:09:39.982102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:09:39.982102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:39.982102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:09:39.982485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:39.982485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:09:39.982696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:39.982696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:09:39.982696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:09:39.982696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:09:39.982696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:09:39.982696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:09:39.982696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:39.982696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:09:39.982696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:09:39.982696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:09:39.982696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:09:39.982696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:09:39.982696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:39.982696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:39.982696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:09:39.982696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:09:39.983176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:09:39.983176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:09:39.983176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:39.983176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:39.983176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:39.983176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:09:39.983176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:39.983176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:39.983176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:39.983176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:09:39.983176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:39.983176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:09:39.983176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:09:39.983176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:39.983176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:09:39.983176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:39.983176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:09:39.983176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:09:39.983176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:39.983740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:09:39.983740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:39.983740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:09:39.985301 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:09:39.985555 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:09:39.985659 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:09:39.985772 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:09:39.986141 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:39.986380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:39.986913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:09:39.986913 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:39.986913 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:09:39.986913 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:09:39.986913 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:39.986913 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:09:39.986913 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:39.986913 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:09:39.987636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:39.987636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:09:39.987636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:09:39.987636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:09:39.987636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:39.987636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:09:39.987636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:39.987636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:09:39.987636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:39.987636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:09:39.987636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:39.987636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:09:39.987636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:39.987636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:09:39.987636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:09:39.988113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:39.988113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:09:39.988113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:39.988113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:09:39.988113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:09:39.988113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:39.988113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:09:39.988113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:39.988113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:09:39.988493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:39.988493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:09:39.988493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:39.988493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:09:39.988493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:09:39.988493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:09:39.988493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:09:39.988493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:09:39.988493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:09:39.988493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:09:39.988493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:39.988493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:39.989317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:39.989317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:09:39.989317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:39.989317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:39.989317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:39.989317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:09:39.989317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:39.989317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:09:39.989317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:09:39.989317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:39.989317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:09:39.989317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:39.989317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:09:39.989317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:09:39.990113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:39.990113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:09:39.990113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:39.990113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:09:39.990113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:39.990113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:09:39.990113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:09:39.990113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:09:39.990113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:09:39.990113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:09:39.990113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:39.990113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:09:39.990113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:09:39.990113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:09:39.990862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:09:39.990862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:09:39.990862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:39.990862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:39.990862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:09:39.990862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:09:39.990862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:09:39.990862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:09:39.990862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:09:39.990862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:09:39.990862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:09:39.990862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:39.990862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:39.990862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:39.990862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:09:39.990862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:39.990862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:39.990862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:39.990862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:09:39.991448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:39.991448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:09:39.991448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:09:39.991448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:39.991448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:09:39.991448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:39.991448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:09:39.991448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:09:39.991448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:39.991448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:09:39.991448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:39.991448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:09:39.992622 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:09:39.992838 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:09:39.992948 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:09:39.993061 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:09:39.993431 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:39.993739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:39.994274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:09:39.994274 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:39.994274 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:09:39.994274 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:09:39.994274 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:39.994274 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:09:39.994274 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:39.994274 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:09:39.994971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:39.994971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:09:39.994971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:09:39.994971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:09:39.994971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:39.994971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:09:39.994971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:39.994971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:09:39.994971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:39.994971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:09:39.994971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:39.994971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:09:39.994971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:39.994971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:09:39.994971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:09:39.996080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:39.996080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:09:39.996080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:39.996080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:09:39.996080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:09:39.996080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:39.996080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:09:39.996080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:39.996080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:09:39.996080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:39.996080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:09:39.996080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:39.996080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:09:39.996080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:09:39.996504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:09:39.996504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:09:39.996504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:09:39.996504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:09:39.996504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:39.996504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:39.996504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:39.996504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:09:39.996504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:39.996504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:39.996504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:39.996504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:09:39.996504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:39.996913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:09:39.996913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:09:39.996913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:39.996913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:09:39.996913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:39.996913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:09:39.996913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:09:39.996913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:39.996913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:09:39.996913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:39.996913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:09:39.996913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:39.997254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:09:39.997254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:09:39.997254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:09:39.997254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:09:39.997254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:09:39.997254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:39.997254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:09:39.997254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:09:39.997254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:09:39.997254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:09:39.997254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:09:39.997254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:39.997254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:39.997254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:09:39.997254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:09:39.997254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:09:39.997254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:09:39.997254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:09:39.997254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:09:39.997825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:39.997825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:39.997825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:39.997825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:09:39.997825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:39.997825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:39.997825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:39.997825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:09:39.997825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:39.997825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:09:39.997825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:09:39.997825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:39.997825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:09:39.997825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:39.997825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:09:39.997825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:09:39.997825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:39.997825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:09:39.997825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:39.997825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:09:39.999179 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:09:39.999430 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:09:39.999527 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:09:39.999664 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:09:39.999990 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:40.000246 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:40.000781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:40.001038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:09:40.001038 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:40.001038 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:09:40.001038 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:09:40.001038 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:40.001038 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:09:40.001038 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:40.001038 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:09:40.002147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:40.002147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:09:40.002147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:40.002147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:09:40.002147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:09:40.002147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:09:40.002147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:09:40.002147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:40.002147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:40.002147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:40.002147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:09:40.002147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:40.002548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:40.002548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:40.002548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:09:40.002548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:40.002548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:09:40.002548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:09:40.002548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:40.002548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:09:40.002548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:40.002548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:09:40.002548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:09:40.002548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:40.002548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:09:40.003103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:40.003103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:09:40.003103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:40.003103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:09:40.003103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:09:40.003103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:09:40.003103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:09:40.003103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:09:40.003103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:40.003103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:09:40.003103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:09:40.003103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:09:40.003103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:09:40.003103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:09:40.003103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:40.003721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:40.003721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:09:40.003721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:09:40.003721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:09:40.003721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:09:40.003721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:40.003721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:40.003721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:40.003721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:09:40.003721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:40.003721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:40.003721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:40.003721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:09:40.003721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:40.003721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:09:40.003721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:09:40.003721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:40.003721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:09:40.003721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:40.004227 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:09:40.004227 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:09:40.004227 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:40.004227 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:09:40.004227 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:40.004227 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:09:44.766682 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:09:44.767615 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:09:44.767615 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:09:44.768887 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:09:44.805632 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:44.805632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:44.805881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:09:44.805881 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:44.805881 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:09:44.805881 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:09:44.805881 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:44.805881 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:09:44.805881 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:44.805881 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:09:44.807116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:44.807116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:09:44.807116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:44.807116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:09:44.807116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:09:44.807116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:09:44.807116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:09:44.807116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:44.807116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:44.807116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:44.807116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:09:44.807116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:44.807517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:44.807517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:44.807517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:09:44.807517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:44.807517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:09:44.807517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:09:44.807517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:44.807517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:09:44.807517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:44.807517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:09:44.807517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:09:44.807517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:44.807517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:09:44.807880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:44.807880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:09:44.808173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:44.808173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:09:44.808173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:09:44.808173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:09:44.808173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:09:44.808173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:09:44.808173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:44.808173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:09:44.808173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:09:44.808173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:09:44.808173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:09:44.808173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:09:44.808173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:44.808173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:44.808173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:09:44.808173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:09:44.809080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:09:44.809080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:09:44.809080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:44.809080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:44.809080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:44.809080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:09:44.809080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:44.809080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:44.809080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:44.809080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:09:44.809080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:44.809080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:09:44.809080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:09:44.809080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:44.809080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:09:44.809080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:44.809080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:09:44.809080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:09:44.809080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:44.810234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:09:44.810234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:44.810234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:09:44.978644 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:09:44.979748 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:09:44.979968 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:09:44.983102 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:44.983102 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:44.983102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:44.983985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:09:44.983985 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:44.983985 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:09:44.983985 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:09:44.983985 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:44.983985 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:09:44.983985 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:44.983985 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:09:44.985276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:44.985276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:09:44.985276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:44.985276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:09:44.985276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:09:44.985276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:09:44.985276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:09:44.985276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:44.985276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:44.985276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:44.985276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:09:44.985276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:44.986075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:44.986075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:44.986075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:09:44.986075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:44.986075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:09:44.986075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:09:44.986075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:44.986075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:09:44.986075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:44.986075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:09:44.986075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:09:44.986075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:44.986075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:09:44.986518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:44.986518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:09:44.987610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:44.987610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:09:44.987610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:09:44.987610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:09:44.987610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:09:44.987610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:09:44.987610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:44.987610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:09:44.987610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:09:44.987610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:09:44.987610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:09:44.987610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:09:44.987610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:44.987610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:44.987610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:09:44.987610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:09:44.988041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:09:44.988041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:09:44.988041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:44.988041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:44.988041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:44.988041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:09:44.988041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:44.988041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:44.988041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:44.988041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:09:44.988041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:44.988041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:09:44.988041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:09:44.988041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:44.988041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:09:44.988041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:44.988041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:09:44.988041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:09:44.988041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:44.988573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:09:44.988573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:44.988573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:09:44.988573 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:09:44.988573 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:09:44.988573 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:09:44.988573 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:09:44.992071 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:44.994207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:44.995201 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:09:44.995535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:09:44.995535 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:44.995535 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:09:44.995535 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:09:44.995535 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:44.995535 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:09:44.995535 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:44.995535 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:09:44.996748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:44.996748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:09:44.996748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:09:44.996748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:09:44.996748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:44.996748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:09:44.996748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:44.996748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:09:44.996748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:44.996748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:09:44.996748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:44.996748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:09:44.996748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:44.996748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:09:44.996748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:09:44.997385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:44.997385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:09:44.997385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:44.997385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:09:44.997385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:09:44.997385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:44.997385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:09:44.997385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:44.997385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:09:44.997385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.008 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:44.997385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:09:44.997385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:44.997385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:09:44.997385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:09:44.997969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:09:44.997969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:09:44.997969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:09:44.997969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:09:44.997969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:09:44.997969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:44.997969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:44.997969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:44.997969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:09:44.997969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:44.997969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:44.997969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:44.997969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:09:44.998385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:44.998385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:09:44.998385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:09:44.998385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:44.998385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:09:44.998385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:44.998385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:09:44.998385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:09:44.998385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:44.998385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:09:44.998385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:44.998385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:09:44.998618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:44.998618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:09:44.998618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:09:44.998618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:09:44.998618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:09:44.998618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:09:44.998618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:44.998618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:09:44.998618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:09:44.998618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:09:44.998618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:09:44.998618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:09:44.998618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:44.998618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:44.998618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:09:44.998618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:09:44.998902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:09:44.998902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:09:44.998902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:09:44.998902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:09:44.998902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:09:44.998902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:44.998902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:44.998902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:44.998902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:09:44.998902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:44.998902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:44.998902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:44.998902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:09:44.998902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:44.998902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:09:44.998902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:09:44.998902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:44.998902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:09:44.999240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:44.999240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:09:44.999240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:09:44.999240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:44.999240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:09:44.999240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:44.999240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:09:44.999240 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:09:44.999240 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:09:44.999240 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:09:45.000516 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:09:45.000516 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:45.000516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:45.000714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:09:45.000714 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:45.000714 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:09:45.000714 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:09:45.000714 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:45.000714 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:09:45.000714 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:45.000714 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:09:45.001535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:45.001535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:09:45.001535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:09:45.001535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:09:45.001535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:45.001535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:09:45.001535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:45.001535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:09:45.001535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:45.001535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:09:45.001535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:45.001535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:09:45.001535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:45.001535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:09:45.001535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:09:45.001961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:45.001961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:09:45.001961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:45.001961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:09:45.001961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:09:45.001961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:45.001961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:09:45.001961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:45.001961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:09:45.002575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:45.002575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:09:45.002575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:45.002575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:09:45.002575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:09:45.002575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:09:45.002575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:09:45.002575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:09:45.002575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:09:45.002575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:45.002575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:45.002803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:45.002803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:09:45.002803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:45.002803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:45.002803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:45.002803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:09:45.002803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:45.002803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:09:45.002803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:09:45.002803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:45.002803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:09:45.002803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:45.002803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:09:45.002803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:09:45.003050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:45.003050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:09:45.003050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:45.003050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:09:45.004300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:45.004300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:09:45.004300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:09:45.004300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:09:45.004300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:09:45.004300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:09:45.004300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:45.004300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:09:45.004300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:09:45.004300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:09:45.004300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:09:45.004300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:09:45.004300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:45.004300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:45.004300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:09:45.004300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:09:45.004300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:09:45.004644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:09:45.004644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:09:45.004644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:09:45.004644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:45.004644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:45.004644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:45.004644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:09:45.004644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:45.004644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:45.004644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:45.004644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:09:45.004644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:45.004644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:09:45.004644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:09:45.004644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:45.004644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:09:45.004644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:45.004644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:09:45.004644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:09:45.004965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:45.004965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:09:45.004965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:45.004965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:09:45.004965 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:09:45.004965 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:09:45.004965 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:09:45.006326 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:45.006326 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:45.006326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:45.006488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:09:45.006488 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:45.006488 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:09:45.006488 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:09:45.006488 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:45.006488 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:09:45.006488 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:45.006488 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:09:45.007138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:45.007138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:09:45.007138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:45.007138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:09:45.007138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:09:45.007138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:09:45.007138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:09:45.007138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:45.007138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:45.007138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:45.007138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:09:45.007138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:45.007384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:45.007384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:45.007384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:09:45.007384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:45.007384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:09:45.007384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:09:45.007384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:45.007384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:09:45.007384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:45.007384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:09:45.007384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:09:45.007384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:45.007384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:09:45.007635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:45.007635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:09:45.007921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:45.007921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:09:45.007921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:09:45.007921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:09:45.007921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:09:45.007921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:09:45.007921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:45.007921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:09:45.007921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:09:45.007921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:09:45.007921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:09:45.007921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:09:45.007921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:45.007921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:45.007921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:09:45.007921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:09:45.008607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:09:45.008607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:09:45.008607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:45.008607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:45.008607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:45.008607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:09:45.008607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:45.008607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:45.008607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:45.008607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:09:45.008607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:45.008607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:09:45.008607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:09:45.008607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:45.008607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:09:45.008607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:45.008607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:09:45.008607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:09:45.008607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:45.009143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:09:45.009143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:45.009143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:09:49.767240 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:09:49.768022 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:09:49.768094 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:09:49.769688 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:09:49.806491 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:49.807720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:49.807977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:09:49.807977 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:49.807977 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:09:49.807977 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:09:49.807977 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:49.807977 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:09:49.807977 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:49.807977 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:09:49.808773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:49.808773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:09:49.808773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:49.808773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:09:49.808773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:09:49.808773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:09:49.808773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:09:49.808773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:49.808773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:49.808773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:49.808773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:09:49.808773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:49.809309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:49.809309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:49.809309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:09:49.809309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:49.809309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:09:49.809309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:09:49.809309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:49.809309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:09:49.809309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:49.809309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:09:49.809309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:09:49.809309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:49.809309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:09:49.809968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:49.809968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:09:49.809968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:49.809968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:09:49.809968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:09:49.809968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:09:49.809968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:09:49.809968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:09:49.809968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:49.809968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:09:49.809968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:09:49.809968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:09:49.809968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:09:49.809968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:09:49.809968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:49.810683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:49.810683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:09:49.810683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:09:49.810683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:09:49.810683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:09:49.810683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:49.810683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:49.810683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:49.810683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:09:49.810683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:49.810683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:49.810683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:49.810683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:09:49.810683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:49.810683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:09:49.810683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:09:49.810683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:49.810683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:09:49.810683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:49.811566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:09:49.811566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:09:49.811566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:49.811566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:09:49.811566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:49.811566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:09:49.979369 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:09:49.979998 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:09:49.980196 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:09:49.982780 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:49.982780 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:49.982780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:49.983042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:09:49.983042 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:49.983042 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:09:49.983042 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:09:49.983042 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:49.983042 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:09:49.983042 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:49.983042 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:09:49.984875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:49.984875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:09:49.984875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:49.984875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:09:49.984875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:09:49.984875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:09:49.984875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:09:49.984875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:49.984875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:49.984875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:49.984875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:09:49.984875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:49.985241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:49.985241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:49.985241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:09:49.985241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:49.985241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:09:49.985241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:09:49.985241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:49.985241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:09:49.985241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:49.985241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:09:49.985241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:09:49.985241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:49.985241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:09:49.985643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:49.985643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:09:49.986838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:49.986838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:09:49.986838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:09:49.986838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:09:49.986838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:09:49.986838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:09:49.986838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:49.986838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:09:49.986838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:09:49.986838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:09:49.986838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:09:49.986838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:09:49.986838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:49.986838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:49.986838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:09:49.986838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:09:49.987289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:09:49.987289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:09:49.987289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:49.987289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:49.987289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:49.987289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:09:49.987289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:49.987289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:49.987289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:49.987289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:09:49.987289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:49.987289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:09:49.987289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:09:49.987289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:49.987289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:09:49.987289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:49.987289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:09:49.987289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:09:49.987289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:49.987811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:09:49.987811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:49.987811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:09:49.989506 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:09:49.989786 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:09:49.989878 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:09:49.990054 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:09:49.990503 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:49.990813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:49.991601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:09:49.991601 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:49.991601 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:09:49.991601 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:09:49.991601 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:49.991601 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:09:49.991601 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:49.991601 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:09:49.992334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:49.992334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:09:49.992334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:09:49.992334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:09:49.992334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:49.992334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:09:49.992334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:49.992334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:09:49.992334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:49.992334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:09:49.992334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:49.992334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:09:49.992334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:49.992334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:09:49.992334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:09:49.992773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:49.992773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:09:49.992773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:49.992773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:09:49.992773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:09:49.992773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:49.992773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:09:49.992773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:49.992773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:09:49.993307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:49.993307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:09:49.993307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:49.993307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:09:49.993307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:09:49.993307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:09:49.993307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:09:49.993307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:09:49.993307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:09:49.993307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:09:49.993307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:49.993307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:49.993723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:49.993723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:09:49.993723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:49.993723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:49.993723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:49.993723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:09:49.993723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:49.993723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:09:49.993723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:09:49.993723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:49.993723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:09:49.993723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:49.993723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:09:49.993723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:09:49.994092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:49.994092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:09:49.994092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:49.994092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:09:49.994501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:49.994501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:09:49.994501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:09:49.994501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:09:49.994501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:09:49.994501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:09:49.994501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:49.994501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:09:49.994501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:09:49.994501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:09:49.994501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:09:49.994501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:09:49.994501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:49.994501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:49.994501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:09:49.994501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:09:49.995101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:09:49.995101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:09:49.995101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:09:49.995101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:09:49.995101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:09:49.995101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:49.995101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:49.995101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:49.995101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:09:49.995101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:49.995101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:49.995101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:49.995101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:09:49.995101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:49.995101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:09:49.995101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:09:49.995101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:49.995101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:09:49.999095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:49.999095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:09:49.999095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:09:49.999095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:49.999095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:09:49.999095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:49.999095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:09:49.999095 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:09:49.999095 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:09:49.999095 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:09:49.999417 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:09:49.999719 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:50.000071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:50.000668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:09:50.000668 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:50.000668 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:09:50.000668 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:09:50.000668 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:50.000668 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:09:50.000668 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:50.000668 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:09:50.001464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:50.001464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:09:50.001464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:09:50.001464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:09:50.001464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:50.001464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:09:50.001464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:50.001464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:09:50.001464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:50.001464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:09:50.001464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:50.001464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:09:50.001464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:50.001464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:09:50.001464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:09:50.001749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:50.001749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:09:50.001749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:50.001749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:09:50.001749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:09:50.001749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:50.001749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:09:50.001749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:50.001749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:09:50.002592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:50.002592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:09:50.002592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:50.002592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:09:50.002592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:09:50.002592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:09:50.002592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:09:50.002592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:09:50.002592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:09:50.002592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:50.002592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:50.002823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:50.002823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:09:50.002823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:50.002823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:50.002823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:50.002823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:09:50.002823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:50.002823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:09:50.002823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:09:50.002823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:50.002823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:09:50.002823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:50.002823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:09:50.002823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:09:50.003073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:50.003073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:09:50.003073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:50.003073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:09:50.003787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:50.003787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:09:50.003787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:09:50.003787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:09:50.003787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:09:50.003787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:09:50.003787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:50.003787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:09:50.003787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:09:50.003787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:09:50.003787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:09:50.003787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:09:50.003787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:50.003787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:50.003787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:09:50.003787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:09:50.003787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:09:50.004141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:09:50.004141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:09:50.004141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:09:50.004141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:50.004141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:50.004141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:50.004141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:09:50.004141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:50.004141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:50.004141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:50.004141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:09:50.004141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:50.004141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:09:50.004141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:09:50.004141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:50.004141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:09:50.004141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:50.004141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:09:50.004141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:09:50.004525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:50.004525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:09:50.004525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:50.004525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:09:50.004607 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:09:50.004799 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:09:50.004905 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:09:50.005055 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:09:50.005840 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:50.005840 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:50.005840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:50.005995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:09:50.005995 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:50.005995 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:09:50.005995 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:09:50.005995 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:50.005995 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:09:50.005995 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:50.005995 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:09:50.006840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:50.006840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:09:50.006840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:50.006840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:09:50.006840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:09:50.006840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:09:50.006840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:09:50.006840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:50.006840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:50.006840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:50.006840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:09:50.006840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:50.007088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:50.007088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:50.007088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:09:50.007088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:50.007088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:09:50.007088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:09:50.007088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:50.007088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:09:50.007088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:50.007088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:09:50.007088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:09:50.007088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:50.007088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:09:50.007336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:50.007336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:09:50.007856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:50.007856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:09:50.007856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:09:50.007856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:09:50.007856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:09:50.007856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:09:50.007856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:50.007856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:09:50.007856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:09:50.007856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:09:50.007856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:09:50.007856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:09:50.007856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:50.007856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:50.007856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:09:50.007856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:09:50.008155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:09:50.008155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:09:50.008155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:50.008155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:50.008155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:50.008155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:09:50.008155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:50.008155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:50.008155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:50.008155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:09:50.008155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:50.008155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:09:50.008155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:09:50.008155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:50.008155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:09:50.008155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:50.008155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:09:50.008155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:09:50.008155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:50.008537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:09:50.008537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:50.008537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:09:54.767892 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:09:54.768924 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:09:54.768924 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:09:54.769232 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:09:54.813677 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:54.813677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:54.813677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:09:54.813677 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:54.813677 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:09:54.813677 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:09:54.813677 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:54.813677 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:09:54.813677 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:54.813677 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:09:54.814041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.045 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:54.814041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:09:54.814041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:54.814041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:09:54.814041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:09:54.814041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:09:54.814041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:09:54.814041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:54.814041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:54.814041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:54.814041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:09:54.814041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:54.814299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:54.814299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:54.814299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:09:54.814299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:54.814299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:09:54.814299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:09:54.814299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:54.814299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:09:54.814299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:54.814299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:09:54.814299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:09:54.814299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:54.814299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:09:54.814559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:54.814559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:09:54.815232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:54.815232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:09:54.815232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:09:54.815232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:09:54.815232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:09:54.815232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:09:54.815232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:54.815232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:09:54.815232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:09:54.815232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:09:54.815232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:09:54.815232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:09:54.815232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:54.815232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:54.815232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:09:54.815232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:09:54.815539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:09:54.815539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:09:54.815539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:54.815539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:54.815539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:54.815539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:09:54.815539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:54.815539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:54.815539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:54.815539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:09:54.815539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:54.815539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:09:54.815539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:09:54.815539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:54.815539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:09:54.815539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:54.815539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:09:54.815539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:09:54.815539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:54.815893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:09:54.815893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:54.815893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:09:54.980001 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:09:54.980689 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:09:54.980940 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:09:54.981904 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:54.982208 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:54.982483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:54.983253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:09:54.983253 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:54.983253 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:09:54.983253 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:09:54.983253 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:54.983253 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:09:54.983253 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:54.983253 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:09:54.984727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:54.984727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:09:54.984727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:54.984727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:09:54.984727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:09:54.984727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:09:54.984727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:09:54.984727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:54.984727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:54.984727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:54.984727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:09:54.984727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:54.985550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:54.985550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:54.985550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:09:54.985550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:54.985550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:09:54.985550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:09:54.985550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:54.985550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:09:54.985550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:54.985550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:09:54.985550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:09:54.985550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:54.985550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:09:54.986380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:54.986380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:09:54.986562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:54.986562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:09:54.986562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:09:54.986562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:09:54.986562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:09:54.986562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:09:54.986562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:54.986562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:09:54.986562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:09:54.986562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:09:54.986562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:09:54.986562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:09:54.986562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:54.986562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:54.986562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:09:54.986562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:09:54.987251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:09:54.987251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:09:54.987251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:54.987251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:54.987251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:54.987251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:09:54.987251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:54.987251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:54.987251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:54.987251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:09:54.987251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:54.987251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:09:54.987251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:09:54.987251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:54.987251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:09:54.987251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:54.987251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:09:54.987251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:09:54.987251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:54.988099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:09:54.988099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:54.988099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:09:54.989883 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:09:54.990193 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:09:54.990371 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:09:54.990591 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:09:54.991019 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:54.991314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:54.991978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:09:54.991978 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:54.991978 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:09:54.991978 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:09:54.991978 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:54.991978 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:09:54.991978 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:54.991978 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:09:54.992748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:54.992748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:09:54.992748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:09:54.992748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:09:54.992748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:54.992748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:09:54.992748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:54.992748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:09:54.992748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:54.992748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:09:54.992748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:54.992748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:09:54.992748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:54.992748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:09:54.992748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:09:54.993228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:54.993228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:09:54.993228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:54.993228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:09:54.993228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:09:54.993228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:54.993228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:09:54.993228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:54.993228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:09:54.993582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:54.993582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:09:54.993582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:54.993582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:09:54.993582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:09:54.993582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:09:54.993582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:09:54.993582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:09:54.993582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:09:54.993582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:09:54.993582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:54.993582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:54.993933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:54.993933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:09:54.993933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:54.993933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:54.993933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:54.993933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:09:54.993933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:54.993933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:09:54.993933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:09:54.993933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:54.993933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:09:54.993933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:54.993933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:09:54.993933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:09:54.994297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:54.994297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:09:54.994297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:54.994297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:09:54.994547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:54.994547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:09:54.994547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:09:54.994547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:09:54.994547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:09:54.994547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:09:54.994547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:54.994547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:09:54.994547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:09:54.994547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:09:54.994547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:09:54.994547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:09:54.994547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:54.994547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:54.994547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:09:54.994547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:09:54.995006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:09:54.995006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:09:54.995006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:09:54.995006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:09:54.995006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:09:54.995006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:54.995006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:54.995006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:54.995006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:09:54.995006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:54.995006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:54.995006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:54.995006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:09:54.995006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:54.995006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:09:54.995006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:09:54.995006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:54.995006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:09:54.995512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:54.995512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:09:54.995512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:09:54.995512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:54.995512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:09:54.995512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:54.995512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:09:54.998998 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:09:54.999236 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:09:54.999345 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:09:54.999501 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:09:54.999892 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:55.000250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:55.001155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:09:55.001155 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:55.001155 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:09:55.001155 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:09:55.001155 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:55.001155 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:09:55.001155 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:55.001155 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:09:55.002238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:55.002238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:09:55.002238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:09:55.002238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:09:55.002238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:55.002238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:09:55.002238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:55.002238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:09:55.002238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:55.002238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:09:55.002238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:55.002238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:09:55.002238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:55.002238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:09:55.002238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:09:55.002657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:55.002657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:09:55.002657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:55.002657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:09:55.002657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:09:55.002657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:55.002657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:09:55.002657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:55.002657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:09:55.004020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:55.004020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:09:55.004020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:55.004020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:09:55.004020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:09:55.004020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:09:55.004020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:09:55.004020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:09:55.004020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:09:55.004020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:55.004020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:55.004337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:55.004337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:09:55.004337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:55.004337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:55.004337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:55.004337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:09:55.004337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:55.004337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:09:55.004337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:09:55.004337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:55.004337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:09:55.004337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:55.004337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:09:55.004337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:09:55.004758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:55.004758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:09:55.004758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:55.004758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:09:55.008306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:55.008306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:09:55.008306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:09:55.008306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:09:55.008306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:09:55.008306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:09:55.008306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:55.008306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:09:55.008306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:09:55.008306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:09:55.008306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:09:55.008306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:09:55.008306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:55.008306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:55.008306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:09:55.008306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:09:55.008306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:09:55.008832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:09:55.008832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:09:55.008832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:09:55.008832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:55.008832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:55.008832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:55.008832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:09:55.008832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:55.008832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:55.008832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:55.008832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:09:55.008832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:55.008832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:09:55.008832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:09:55.008832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:55.008832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:09:55.008832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:55.008832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:09:55.008832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:09:55.009339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:55.009339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:09:55.009339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:55.009339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:09:55.009339 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:09:55.009339 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:09:55.009339 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:09:55.009339 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:09:55.009590 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:55.009590 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:55.009821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:55.010430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:09:55.010430 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:55.010430 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:09:55.010430 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:09:55.010430 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:55.010430 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:09:55.010430 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:55.010430 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:09:55.010989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:55.010989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:09:55.010989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:55.010989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:09:55.010989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:09:55.010989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:09:55.010989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:09:55.010989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:55.010989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:55.010989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:55.010989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:09:55.010989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:55.011292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:55.011292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:55.011292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:09:55.011292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:55.011292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:09:55.011292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:09:55.011292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:55.011292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:09:55.011292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:55.011292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:09:55.011292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:09:55.011292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:55.011292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:09:55.011656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:55.011656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:09:55.011656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:55.011656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:09:55.011656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:09:55.011656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:09:55.011656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:09:55.011656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:09:55.011656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:55.011656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:09:55.011656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:09:55.011656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:09:55.011656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:09:55.011656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:09:55.011656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:55.012059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:55.012059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:09:55.012059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:09:55.012059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:09:55.012059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:09:55.012059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:55.012059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:55.012059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:55.012059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:09:55.012059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:55.012059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:55.012059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:55.012059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:09:55.012059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:55.012059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:09:55.012059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:09:55.012059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:55.012059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:09:55.012059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:55.012667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:09:55.012667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:09:55.012667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:55.012667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:09:55.012667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:55.012667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:09:59.768517 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:09:59.769315 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:09:59.769511 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:09:59.770796 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:09:59.821862 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:59.822305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:59.823229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:09:59.823229 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:59.823229 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:09:59.823229 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:09:59.823229 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:59.823229 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:09:59.823229 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:59.823229 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:09:59.824097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.054 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:59.824097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:09:59.824097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:59.824097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:09:59.824097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:09:59.824097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:09:59.824097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:09:59.824097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:59.824097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:59.824097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:59.824097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:09:59.824097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:59.825465 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:59.825465 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:59.825465 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:09:59.825465 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:59.825465 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:09:59.825465 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:09:59.825465 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:59.825465 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:09:59.825465 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:59.825465 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:09:59.825465 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:09:59.825465 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:59.825465 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:09:59.826012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:59.826012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:09:59.826012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:59.826012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:09:59.826012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:09:59.826012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:09:59.826012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:09:59.826012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:09:59.826012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:59.826012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:09:59.826012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:09:59.826012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:09:59.826012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:09:59.826012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:09:59.826012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:59.826716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:59.826716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:09:59.826716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:09:59.826716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:09:59.826716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:09:59.826716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:59.826716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:59.826716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:59.826716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:09:59.826716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:59.826716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:59.826716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:59.826716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:09:59.826716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:59.826716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:09:59.826716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:09:59.826716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:59.826716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:09:59.826716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:59.830479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:09:59.830479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:09:59.830479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:59.830479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:09:59.830479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:59.830479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:09:59.980903 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:09:59.981706 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:09:59.981706 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:09:59.982869 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:59.983237 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:59.983542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:59.984484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:09:59.984484 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:59.984484 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:09:59.984484 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:09:59.984484 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:59.984484 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:09:59.984484 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:59.984484 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:09:59.986028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:59.986028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:09:59.986028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:59.986028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:09:59.986028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:09:59.986028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:09:59.986028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:09:59.986028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:59.986028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:59.986028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:59.986028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:09:59.986028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:59.987458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:59.987458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:59.987458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:09:59.987458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:59.987458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:09:59.987458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:09:59.987458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:59.987458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:09:59.987458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:59.987458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:09:59.987458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:09:59.987458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:59.987458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:09:59.988689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:59.988689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:09:59.988689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:59.988689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:09:59.988689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:09:59.988689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:09:59.988689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:09:59.988689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:09:59.988689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:59.988689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:09:59.988689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:09:59.988689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:09:59.988689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:09:59.988689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:09:59.988689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:59.989697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:59.989697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:09:59.989697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:09:59.989697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:09:59.989697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:09:59.989697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:59.989697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:59.989697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:59.989697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:09:59.989697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:59.989697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:59.989697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:59.989697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:09:59.989697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:59.989697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:09:59.989697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:09:59.989697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:59.989697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:09:59.989697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:59.990340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:09:59.990340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:09:59.990340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:59.990340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:09:59.990340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:59.990340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:09:59.990340 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:09:59.991497 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:09:59.991497 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:09:59.991497 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:09:59.991497 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:09:59.991786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:59.992536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:09:59.992536 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:59.992536 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:09:59.992536 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:09:59.992536 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:59.992536 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:09:59.992536 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:59.992536 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:09:59.993280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:59.993280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:09:59.993280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:09:59.993280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:09:59.993280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:59.993280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:09:59.993280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:59.993280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:09:59.993280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:59.993280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:09:59.993280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:59.993280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:09:59.993280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:59.993280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:09:59.993280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:09:59.994305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:59.994305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:09:59.994305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:59.994305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:09:59.994305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:09:59.994305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:59.994305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:09:59.994305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:59.994305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:09:59.994305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:59.994305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:09:59.994305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:59.994305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:09:59.994305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:09:59.995162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:09:59.995162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:09:59.995162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:09:59.995162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:09:59.995162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:09:59.995162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:59.995162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:59.995162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:59.995162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:09:59.995162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:59.995162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:09:59.995162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:59.995162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:09:59.995782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:59.995782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:09:59.995782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:09:59.995782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:59.995782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:09:59.995782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:59.995782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:09:59.995782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:09:59.995782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:59.995782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:09:59.995782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:59.995782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:09:59.996095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:59.996095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:09:59.996095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:09:59.996095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:09:59.996095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:09:59.996095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:09:59.996095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:59.996095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:09:59.996095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:09:59.996095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:09:59.996095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:09:59.996095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:09:59.996095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:09:59.996095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:09:59.996095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:09:59.996095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:09:59.996530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:09:59.996530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:09:59.996530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:09:59.996530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:09:59.996530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:09:59.996530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:59.996530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:59.996530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:09:59.996530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:09:59.996530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:09:59.996530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:09:59.996530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:09:59.996530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:09:59.996530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:09:59.996530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:09:59.996530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:09:59.996530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:09:59.996530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:09:59.997020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:09:59.997020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:09:59.997020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:09:59.997020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:09:59.997020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:09:59.997020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:09:59.997020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:09:59.999427 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:09:59.999699 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:09:59.999801 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:09:59.999935 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:10:00.000392 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:10:00.000919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:00.001726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:10:00.001726 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:00.001726 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:10:00.001726 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:10:00.001726 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:00.001726 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:10:00.001726 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:00.001726 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:10:00.002426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:00.002426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:10:00.002426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:10:00.002426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:10:00.002426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:00.002426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:10:00.002426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:00.002426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:10:00.002426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:00.002426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:10:00.002426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:00.002426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:10:00.002426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:00.002426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:10:00.002426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:10:00.002724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:00.002724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:10:00.002724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:00.002724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:10:00.002724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:10:00.002724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:00.002724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:10:00.002724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:00.002724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:10:00.003298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:00.003298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:10:00.003298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:00.003298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:10:00.003298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:10:00.003298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:10:00.003298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:10:00.003298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:10:00.003298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:10:00.003298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:00.003298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:00.003849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:00.003849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:10:00.003849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:00.003849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:00.003849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:00.003849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:10:00.003849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:00.003849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:10:00.003849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:10:00.003849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:00.003849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:10:00.003849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:00.003849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:10:00.003849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:10:00.004350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:00.004350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:10:00.004350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:00.004350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:10:00.004350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:00.004350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:10:00.004350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:10:00.004350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:10:00.004350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:10:00.004350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:10:00.004350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:00.004350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:10:00.004350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:10:00.004350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:10:00.004632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:10:00.004632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:10:00.004632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:00.004632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:00.004632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:10:00.004632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:10:00.004632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:10:00.004632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:10:00.004632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:10:00.004632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:10:00.004632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:00.004632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:00.004632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:00.004632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:10:00.004632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:00.004632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:00.004632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:00.004632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:10:00.004632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:00.004973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:10:00.004973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:10:00.004973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:00.004973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:10:00.004973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:00.004973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:10:00.004973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:10:00.004973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:00.004973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:10:00.004973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:00.004973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:10:00.008842 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:10:00.009117 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:10:00.009216 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:10:00.009368 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:10:00.009810 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:10:00.010166 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:10:00.010417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:00.011126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:10:00.011126 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:00.011126 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:10:00.011126 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:10:00.011126 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:00.011126 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:10:00.011126 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:00.011126 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:10:00.011937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:00.011937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:10:00.011937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:00.011937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:10:00.011937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:10:00.011937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:10:00.011937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:10:00.011937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:00.011937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:00.011937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:00.011937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:10:00.011937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:00.012280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:00.012280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:00.012280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:10:00.012280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:00.012280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:10:00.012280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:10:00.012280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:00.012280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:10:00.012280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:00.012280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:10:00.012280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:10:00.012280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:00.012280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:10:00.012614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:00.012614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:10:00.012933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:00.012933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:10:00.012933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:10:00.012933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:10:00.012933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:10:00.012933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:10:00.012933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:00.012933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:10:00.012933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:10:00.012933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:10:00.012933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:10:00.012933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:10:00.012933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:00.012933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:00.012933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:10:00.012933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:10:00.014247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:10:00.014247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:10:00.014247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:00.014247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:00.014247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:00.014247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:10:00.014247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:00.014247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:00.014247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:00.014247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:10:00.014247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:00.014247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:10:00.014247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:10:00.014247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:00.014247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:10:00.014247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:00.014247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:10:00.014247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:10:00.014247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:00.014728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:10:00.014728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:00.014728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:10:04.769234 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:10:04.770477 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:10:04.770477 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:10:04.771443 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:10:04.806339 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:10:04.806739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:04.807598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:10:04.807598 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:04.807598 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:10:04.807598 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:10:04.807598 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:04.807598 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:10:04.807598 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:04.807598 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:10:04.808323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:04.808323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:10:04.808323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:04.808323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:10:04.808323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:10:04.808323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:10:04.808323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:10:04.808323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:04.808323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:04.808323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:04.808323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:10:04.808323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:04.809030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:04.809030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:04.809030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:10:04.809030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:04.809030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:10:04.809030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:10:04.809030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:04.809030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:10:04.809030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:04.809030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:10:04.809030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:10:04.809030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:04.809030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:10:04.809756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:04.809756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:10:04.809756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:04.809756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:10:04.809756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:10:04.809756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:10:04.809756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:10:04.809756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:10:04.809756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:04.809756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:10:04.809756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:10:04.809756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:10:04.809756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:10:04.809756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:10:04.809756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:04.810473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:04.810473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:10:04.810473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:10:04.810473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:10:04.810473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:10:04.810473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:04.810473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:04.810473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:04.810473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:10:04.810473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:04.810473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:04.810473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:04.810473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:10:04.810473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:04.810473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:10:04.810473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:10:04.810473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:04.810473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:10:04.810473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:04.810971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:10:04.810971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:10:04.810971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:04.810971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:10:04.810971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:04.810971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:10:04.981446 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:10:04.982083 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:10:04.982251 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:10:04.983428 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:10:04.983783 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:10:04.984019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:04.984925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:10:04.984925 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:04.984925 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:10:04.984925 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:10:04.984925 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:04.984925 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:10:04.984925 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:04.984925 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:10:04.986522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:04.986522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:10:04.986522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:04.986522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:10:04.986522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:10:04.986522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:10:04.986522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:10:04.986522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:04.986522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:04.986522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:04.986522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:10:04.986522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:04.987231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:04.987231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:04.987231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:10:04.987231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:04.987231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:10:04.987231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:10:04.987231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:04.987231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:10:04.987231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:04.987231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:10:04.987231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:10:04.987231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:04.987231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:10:04.987817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:04.987817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:10:04.988258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:04.988258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:10:04.988258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:10:04.988258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:10:04.988258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:10:04.988258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:10:04.988258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:04.988258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:10:04.988258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:10:04.988258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:10:04.988258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:10:04.988258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:10:04.988258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:04.988258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:04.988258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:10:04.988258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:10:04.989011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:10:04.989011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:10:04.989011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:04.989011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:04.989011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:04.989011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:10:04.989011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:04.989011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:04.989011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:04.989011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:10:04.989011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:04.989011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:10:04.989011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:10:04.989011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:04.989011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:10:04.989011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:04.989011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:10:04.989011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:10:04.989011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:04.989951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:10:04.989951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:04.989951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:10:04.990729 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:10:04.991011 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:10:04.991091 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:10:04.991237 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:10:04.991728 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:10:04.992029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:04.992752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:10:04.992752 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:04.992752 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:10:04.992752 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:10:04.992752 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:04.992752 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:10:04.992752 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:04.992752 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:10:04.993522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:04.993522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:10:04.993522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:10:04.993522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:10:04.993522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:04.993522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:10:04.993522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:04.993522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:10:04.993522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:04.993522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:10:04.993522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:04.993522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:10:04.993522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:04.993522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:10:04.993522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:10:04.994506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:04.994506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:10:04.994506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:04.994506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:10:04.994506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:10:04.994506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:04.994506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:10:04.994506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:04.994506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:10:04.994506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:04.994506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:10:04.994506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:04.994506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:10:04.994506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:10:04.995026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:10:04.995026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:10:04.995026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:10:04.995026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:10:04.995026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:10:04.995026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:04.995026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:04.995026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:04.995026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:10:04.995026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:04.995026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:04.995026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:04.995026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:10:04.995487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:04.995487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:10:04.995487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:10:04.995487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:04.995487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:10:04.995487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:04.995487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:10:04.995487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:10:04.995487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:04.995487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:10:04.995487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:04.995487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:10:04.995838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:04.995838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:10:04.995838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:10:04.995838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:10:04.995838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:10:04.995838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:10:04.995838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:04.995838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:10:04.995838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:10:04.995838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:10:04.995838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:10:04.995838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:10:04.995838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:04.995838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:04.995838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:10:04.995838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:10:04.996320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:10:04.996320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:10:04.996320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:10:04.996320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:10:04.996320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:10:04.996320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:04.996320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:04.996320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:04.996320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:10:04.996320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:04.996320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:04.996320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:04.996320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:10:04.996320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:04.996320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:10:04.996320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:10:04.996320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:04.996320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:10:04.996919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:04.996919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:10:04.996919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:10:04.996919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:04.996919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:10:04.996919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:04.996919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:10:04.999949 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:10:05.000255 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:10:05.000364 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:10:05.000519 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:10:05.000850 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:10:05.001062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:05.001544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:10:05.001544 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:05.001544 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:10:05.001544 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:10:05.001544 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:05.001544 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:10:05.001544 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:05.001544 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:10:05.002276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:05.002276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:10:05.002276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:10:05.002276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:10:05.002276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:05.002276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:10:05.002276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:05.002276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:10:05.002276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:05.002276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:10:05.002276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:05.002276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:10:05.002276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:05.002276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:10:05.002276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:10:05.002719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:05.002719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:10:05.002719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:05.002719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:10:05.002719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:10:05.002719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:05.002719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:10:05.002719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:05.002719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:10:05.003118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:05.003118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:10:05.003118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:05.003118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:10:05.003118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:10:05.003118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:10:05.003118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:10:05.003118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:10:05.003118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:10:05.003118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:05.003118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:05.003457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:05.003457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:10:05.003457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:05.003457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:05.003457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:05.003457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:10:05.003457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:05.003457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:10:05.003457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:10:05.003457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:05.003457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:10:05.003457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:05.003457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:10:05.003457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:10:05.003873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:05.003873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:10:05.003873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:05.003873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:10:05.004243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:05.004243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:10:05.004243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:10:05.004243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:10:05.004243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:10:05.004243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:10:05.004243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:05.004243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:10:05.004243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:10:05.004243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:10:05.004243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:10:05.004243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:10:05.004243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:05.004243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:05.004243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:10:05.004243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:10:05.004243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:10:05.004734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:10:05.004734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:10:05.004734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:10:05.004734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:05.004734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:05.004734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:05.004734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:10:05.004734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:05.004734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:05.004734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:05.004734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:10:05.004734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:05.004734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:10:05.004734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:10:05.004734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:05.004734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:10:05.004734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:05.004734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:10:05.004734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:10:05.005195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:05.005195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:10:05.005195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:05.005195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:10:05.009180 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:10:05.009429 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:10:05.009534 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:10:05.009674 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:10:05.010060 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:10:05.010284 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:10:05.010501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:05.011240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:10:05.011240 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:05.011240 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:10:05.011240 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:10:05.011240 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:05.011240 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:10:05.011240 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:05.011240 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:10:05.012036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:05.012036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:10:05.012036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:05.012036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:10:05.012036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:10:05.012036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:10:05.012036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:10:05.012036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:05.012036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:05.012036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:05.012036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:10:05.012036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:05.012392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:05.012392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:05.012392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:10:05.012392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:05.012392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:10:05.012392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:10:05.012392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:05.012392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:10:05.012392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:05.012392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:10:05.012392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:10:05.012392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:05.012392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:10:05.012767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:05.012767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:10:05.012975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:05.012975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:10:05.012975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:10:05.012975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:10:05.012975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:10:05.012975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:10:05.012975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:05.012975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:10:05.012975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:10:05.012975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:10:05.012975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:10:05.012975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:10:05.012975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:05.012975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:05.012975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:10:05.012975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:10:05.013502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:10:05.013502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:10:05.013502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:05.013502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:05.013502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:05.013502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:10:05.013502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:05.013502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:05.013502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:05.013502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:10:05.013502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:05.013502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:10:05.013502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:10:05.013502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:05.013502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:10:05.013502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:05.013502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:10:05.013502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:10:05.013502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:05.014026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:10:05.014026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:05.014026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:10:09.769741 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:10:09.770884 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:10:09.770884 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:10:09.771477 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:10:09.807165 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:10:09.807617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:09.808519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:10:09.808519 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:09.808519 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:10:09.808519 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:10:09.808519 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:09.808519 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:10:09.808519 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:09.808519 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:10:09.809282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:09.809282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:10:09.809282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:09.809282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:10:09.809282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:10:09.809282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:10:09.809282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:10:09.809282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:09.809282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:09.809282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:09.809282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:10:09.809282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:09.809843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:09.809843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:09.809843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:10:09.809843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:09.809843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:10:09.809843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:10:09.809843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:09.809843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:10:09.809843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:09.809843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:10:09.809843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:10:09.809843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:09.809843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:10:09.810356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:09.810356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:10:09.810356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:09.810356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:10:09.810356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:10:09.810356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:10:09.810356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:10:09.810356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:10:09.810356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:09.810356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:10:09.810356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:10:09.810356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:10:09.810356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:10:09.810356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:10:09.810356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:09.810976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:09.810976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:10:09.810976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:10:09.810976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:10:09.810976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:10:09.810976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:09.810976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:09.810976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:09.810976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:10:09.810976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:09.810976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:09.810976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:09.810976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:10:09.810976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:09.810976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:10:09.810976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:10:09.810976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:09.810976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:10:09.810976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:09.811714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:10:09.811714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:10:09.811714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:09.811714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:10:09.811714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:09.811714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:10:09.981999 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:10:09.982438 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:10:09.982722 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:10:09.983879 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:10:09.984206 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:10:09.984480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:09.985360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:10:09.985360 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:09.985360 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:10:09.985360 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:10:09.985360 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:09.985360 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:10:09.985360 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:09.985360 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:10:09.987024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:09.987024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:10:09.987024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:09.987024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:10:09.987024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:10:09.987024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:10:09.987024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:10:09.987024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:09.987024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:09.987024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:09.987024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:10:09.987024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:09.987544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:09.987544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:09.987544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:10:09.987544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:09.987544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:10:09.987544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:10:09.987544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:09.987544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:10:09.987544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:09.987544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:10:09.987544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:10:09.987544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:09.987544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:10:09.988085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:09.988085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:10:09.990243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:09.990243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:10:09.990243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:10:09.990243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:10:09.990243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:10:09.990243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:10:09.990243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:09.990243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:10:09.990243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:10:09.990243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:10:09.990243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:10:09.990243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:10:09.990243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:09.990243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:09.990243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:10:09.990243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:10:09.990855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:10:09.990855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:10:09.990855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:09.990855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:09.990855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:09.990855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:10:09.990855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:09.990855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:09.990855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:09.990855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:10:09.990855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:09.990855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:10:09.990855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:10:09.990855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:09.990855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:10:09.990855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:09.990855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:10:09.990855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:10:09.990855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:09.991512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:10:09.991512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:09.991512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:10:09.991512 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:10:09.991512 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:10:09.991512 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:10:09.991678 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:10:09.992203 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:10:09.992463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:09.993194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:10:09.993194 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:09.993194 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:10:09.993194 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:10:09.993194 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:09.993194 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:10:09.993194 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:09.993194 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:10:09.993985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:09.993985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:10:09.993985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:10:09.993985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:10:09.993985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:09.993985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:10:09.993985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:09.993985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:10:09.993985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:09.993985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:10:09.993985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:09.993985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:10:09.993985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:09.993985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:10:09.993985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:10:09.994849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:09.994849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:10:09.994849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:09.994849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:10:09.994849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:10:09.994849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:09.994849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:10:09.994849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:09.994849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:10:09.994849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:09.994849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:10:09.994849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:09.994849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:10:09.994849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:10:09.995644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:10:09.995644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:10:09.995644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:10:09.995644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:10:09.995644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:10:09.995644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:09.995644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:09.995644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:09.995644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:10:09.995644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:09.995644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:09.995644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:09.995644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:10:09.996305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:09.996305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:10:09.996305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:10:09.996305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:09.996305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:10:09.996305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:09.996305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:10:09.996305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:10:09.996305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:09.996305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:10:09.996305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:09.996305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:10:09.996929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:09.996929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:10:09.996929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:10:09.996929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:10:09.996929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:10:09.996929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:10:09.996929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:09.996929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:10:09.996929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:10:09.996929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:10:09.996929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:10:09.996929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:10:09.996929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:09.996929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:09.996929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:10:09.996929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:10:09.997321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:10:09.997321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:10:09.997321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:10:09.997321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:10:09.997321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:10:09.997321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:09.997321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:09.997321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:09.997321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:10:09.997321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:09.997321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:09.997321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:09.997321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:10:09.997321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:09.997321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:10:09.997321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:10:09.997321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:09.997321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:10:09.997786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:09.997786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:10:09.997786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:10:09.997786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:09.997786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:10:09.997786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:09.997786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:10:10.000287 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:10:10.000509 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:10:10.000633 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:10:10.000739 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:10:10.001246 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:10:10.001652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:10.002350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:10:10.002350 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:10.002350 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:10:10.002350 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:10:10.002350 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:10.002350 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:10:10.002350 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:10.002350 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:10:10.003133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:10.003133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:10:10.003133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:10:10.003133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:10:10.003133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:10.003133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:10:10.003133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:10.003133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:10:10.003133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:10.003133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:10:10.003133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:10.003133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:10:10.003133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:10.003133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:10:10.003133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:10:10.003752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:10.003752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:10:10.003752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:10.003752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:10:10.003752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:10:10.003752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:10.003752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:10:10.003752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:10.003752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:10:10.004209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:10.004209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:10:10.004209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:10.004209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:10:10.004209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:10:10.004209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:10:10.004209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:10:10.004209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:10:10.004209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:10:10.004209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:10.004209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:10.004716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:10.004716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:10:10.004716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:10.004716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:10.004716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:10.004716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:10:10.004716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:10.004716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:10:10.004716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:10:10.004716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:10.004716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:10:10.004716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:10.004716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:10:10.004716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:10:10.005386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:10.005386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:10:10.005386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:10.005386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:10:10.005386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:10.005386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:10:10.005386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:10:10.005386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:10:10.005386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:10:10.005386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:10:10.005386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:10.005386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:10:10.005386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:10:10.005386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:10:10.005816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:10:10.005816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:10:10.005816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:10.005816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:10.005816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:10:10.005816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:10:10.005816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:10:10.005816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:10:10.005816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:10:10.005816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:10:10.005816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:10.005816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:10.005816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:10.005816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:10:10.005816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:10.005816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:10.005816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:10.005816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:10:10.005816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:10.006289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:10:10.006289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:10:10.006289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:10.006289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:10:10.006289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:10.006289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:10:10.006289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:10:10.006289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:10.006289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:10:10.006289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:10.006289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:10:10.009431 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:10:10.009652 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:10:10.009754 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:10:10.009915 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:10:10.010371 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:10:10.010747 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:10:10.010980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:10.011668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:10:10.011668 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:10.011668 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:10:10.011668 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:10:10.011668 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:10.011668 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:10:10.011668 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:10.011668 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:10:10.012579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:10.012579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:10:10.012579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:10.012579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:10:10.012579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:10:10.012579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:10:10.012579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:10:10.012579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:10.012579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:10.012579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:10.012579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:10:10.012579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:10.013250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:10.013250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:10.013250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:10:10.013250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:10.013250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:10:10.013250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:10:10.013250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:10.013250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:10:10.013250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:10.013250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:10:10.013250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:10:10.013250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:10.013250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:10:10.013947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:10.013947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:10:10.013947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:10.013947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:10:10.013947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:10:10.013947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:10:10.013947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:10:10.013947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:10:10.013947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:10.013947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:10:10.013947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:10:10.013947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:10:10.013947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:10:10.013947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:10:10.013947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:10.014733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:10.014733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:10:10.014733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:10:10.014733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:10:10.014733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:10:10.014733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:10.014733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:10.014733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:10.014733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:10:10.014733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:10.014733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:10.014733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:10.014733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:10:10.014733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:10.014733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:10:10.014733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:10:10.014733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:10.014733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:10:10.014733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:10.015858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:10:10.015858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:10:10.015858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:10.015858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:10:10.015858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:10.015858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:10:14.770312 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:10:14.771206 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:10:14.771206 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:10:14.772568 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:10:14.805598 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:10:14.805865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:14.806540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:10:14.806540 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:14.806540 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:10:14.806540 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:10:14.806540 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:14.806540 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:10:14.806540 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:14.806540 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:10:14.807426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:14.807426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:10:14.807426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:14.807426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:10:14.807426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:10:14.807426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:10:14.807426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:10:14.807426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:14.807426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:14.807426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:14.807426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:10:14.807426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:14.808117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:14.808117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:14.808117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:10:14.808117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:14.808117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:10:14.808117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:10:14.808117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:14.808117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:10:14.808117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:14.808117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:10:14.808117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:10:14.808117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:14.808117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:10:14.808738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:14.808738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:10:14.808738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:14.808738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:10:14.808738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:10:14.808738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:10:14.808738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:10:14.808738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:10:14.808738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:14.808738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:10:14.808738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:10:14.808738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:10:14.808738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:10:14.808738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:10:14.808738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:14.809435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:14.809435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:10:14.809435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:10:14.809435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:10:14.809435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:10:14.809435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:14.809435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:14.809435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:14.809435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:10:14.809435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:14.809435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:14.809435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:14.809435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:10:14.809435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:14.809435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:10:14.809435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:10:14.809435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:14.809435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:10:14.809435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:14.810271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:10:14.810271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:10:14.810271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:14.810271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:10:14.810271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:14.810271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:10:14.982736 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:10:14.983365 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:10:14.983636 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:10:14.984631 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:10:14.984906 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:10:14.985128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:14.985978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:10:14.985978 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:14.985978 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:10:14.985978 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:10:14.985978 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:14.985978 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:10:14.985978 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:14.985978 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:10:14.987515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:14.987515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:10:14.987515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:14.987515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:10:14.987515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:10:14.987515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:10:14.987515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:10:14.987515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:14.987515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:14.987515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:14.987515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:10:14.987515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:14.988176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:14.988176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:14.988176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:10:14.988176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:14.988176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:10:14.988176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:10:14.988176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:14.988176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:10:14.988176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:14.988176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:10:14.988176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:10:14.988176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:14.988176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:10:14.988854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:14.988854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:10:14.989739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:14.989739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:10:14.989739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:10:14.989739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:10:14.989739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:10:14.989739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:10:14.989739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:14.989739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:10:14.989739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:10:14.989739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:10:14.989739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:10:14.989739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:10:14.989739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:14.989739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:14.989739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:10:14.989739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:10:14.990590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:10:14.990590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:10:14.990590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:14.990590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:14.990590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:14.990590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:10:14.990590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:14.990590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:14.990590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:14.990590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:10:14.990590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:14.990590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:10:14.990590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:10:14.990590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:14.990590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:10:14.990590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:14.990590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:10:14.990590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:10:14.990590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:14.991148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:10:14.991148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:14.991148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:10:14.991519 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:10:14.991837 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:10:14.991981 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:10:14.992142 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:10:14.992568 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:10:14.992834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:14.993457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:10:14.993457 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:14.993457 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:10:14.993457 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:10:14.993457 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:14.993457 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:10:14.993457 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:14.993457 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:10:14.994320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:14.994320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:10:14.994320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:10:14.994320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:10:14.994320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:14.994320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:10:14.994320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:14.994320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:10:14.994320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:14.994320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:10:14.994320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:14.994320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:10:14.994320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:14.994320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:10:14.994320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:10:14.994758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:14.994758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:10:14.994758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:14.994758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:10:14.994758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:10:14.994758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:14.994758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:10:14.994758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:14.994758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:10:14.995371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:14.995371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:10:14.995371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:14.995371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:10:14.995371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:10:14.995371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:10:14.995371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:10:14.995371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:10:14.995371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:10:14.995371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:10:14.995371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:14.995371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:14.995736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:14.995736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:10:14.995736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:14.995736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:14.995736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:14.995736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:10:14.995736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:14.995736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:10:14.995736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:10:14.995736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:14.995736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:10:14.995736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:14.995736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:10:14.995736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:10:14.996086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:14.996086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:10:14.996086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:14.996086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:10:14.996695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:14.996695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:10:14.996695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:10:14.996695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:10:14.996695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:10:14.996695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:10:14.996695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:14.996695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:10:14.996695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:10:14.996695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:10:14.996695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:10:14.996695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:10:14.996695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:14.996695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:14.996695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:10:14.996695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:10:14.997190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:10:14.997190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:10:14.997190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:10:14.997190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:10:14.997190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:10:14.997190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:14.997190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:14.997190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:14.997190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:10:14.997190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:14.997190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:14.997190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:14.997190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:10:14.997190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:14.997190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:10:14.997190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:10:14.997190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:14.997190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:10:14.997955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:14.997955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:10:14.997955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:10:14.997955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:14.997955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:10:14.997955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:14.997955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:10:15.000790 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:10:15.001123 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:10:15.001247 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:10:15.001423 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:10:15.001771 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:10:15.002002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:15.002471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:10:15.002471 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:15.002471 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:10:15.002471 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:10:15.002471 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:15.002471 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:10:15.002471 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:15.002471 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:10:15.003277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:15.003277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:10:15.003277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:10:15.003277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:10:15.003277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:15.003277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:10:15.003277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:15.003277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:10:15.003277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:15.003277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:10:15.003277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:15.003277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:10:15.003277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:15.003277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:10:15.003277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:10:15.003707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:15.003707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:10:15.003707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:15.003707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:10:15.003707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:10:15.003707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:15.003707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:10:15.003707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:15.003707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:10:15.004381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:15.004381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:10:15.004381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:15.004381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:10:15.004381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:10:15.004381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:10:15.004381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:10:15.004381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:10:15.004381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:10:15.004381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:15.004381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:15.004721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:15.004721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:10:15.004721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:15.004721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:15.004721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:15.004721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:10:15.004721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:15.004721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:10:15.004721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:10:15.004721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:15.004721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:10:15.004721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:15.004721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:10:15.004721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:10:15.005172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:15.005172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:10:15.005172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:15.005172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:10:15.005978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:15.005978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:10:15.005978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:10:15.005978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:10:15.005978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:10:15.005978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:10:15.005978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:15.005978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:10:15.005978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:10:15.005978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:10:15.005978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:10:15.005978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:10:15.005978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:15.005978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:15.005978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:10:15.005978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:10:15.005978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:10:15.006538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:10:15.006538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:10:15.006538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:10:15.006538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:15.006538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:15.006538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:15.006538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:10:15.006538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:15.006538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:15.006538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:15.006538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:10:15.006538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:15.006538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:10:15.006538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:10:15.006538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:15.006538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:10:15.006538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:15.006538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:10:15.006538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:10:15.007056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:15.007056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:10:15.007056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:15.007056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:10:15.009882 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:10:15.010193 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:10:15.010332 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:10:15.010555 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:10:15.010859 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:10:15.011076 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:10:15.011215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:15.011672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:10:15.011672 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:15.011672 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:10:15.011672 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:10:15.011672 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:15.011672 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:10:15.011672 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:15.011672 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:10:15.012585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:15.012585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:10:15.012585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:15.012585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:10:15.012585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:10:15.012585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:10:15.012585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:10:15.012585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:15.012585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:15.012585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:15.012585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:10:15.012585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:15.012929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:15.012929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:15.012929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:10:15.012929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:15.012929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:10:15.012929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:10:15.012929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:15.012929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:10:15.012929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:15.012929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:10:15.012929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:10:15.012929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:15.012929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:10:15.013276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:15.013276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:10:15.013869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:15.013869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:10:15.013869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:10:15.013869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:10:15.013869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:10:15.013869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:10:15.013869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:15.013869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:10:15.013869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:10:15.013869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:10:15.013869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:10:15.013869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:10:15.013869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:15.013869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:15.013869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:10:15.013869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:10:15.014297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:10:15.014297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:10:15.014297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:15.014297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:15.014297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:15.014297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:10:15.014297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:15.014297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:15.014297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:15.014297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:10:15.014297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:15.014297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:10:15.014297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:10:15.014297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:15.014297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:10:15.014297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:15.014297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:10:15.014297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:10:15.014297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:15.014845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:10:15.014845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:15.014845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:10:19.771366 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:10:19.772183 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:10:19.772183 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:10:19.773292 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:10:19.808508 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:10:19.808920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:19.809771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:10:19.809771 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:19.809771 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:10:19.809771 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:10:19.809771 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:19.809771 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:10:19.809771 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:19.809771 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:10:19.810543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:19.810543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:10:19.810543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:19.810543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:10:19.810543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:10:19.810543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:10:19.810543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:10:19.810543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:19.810543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:19.810543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:19.810543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:10:19.810543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:19.811653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:19.811653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:19.811653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:10:19.811653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:19.811653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:10:19.811653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:10:19.811653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:19.811653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:10:19.811653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:19.811653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:10:19.811653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:10:19.811653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:19.811653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:10:19.812155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:19.812155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:10:19.812155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:19.812155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:10:19.812155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:10:19.812155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:10:19.812155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:10:19.812155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:10:19.812155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:19.812155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:10:19.812155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:10:19.812155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:10:19.812155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:10:19.812155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:10:19.812155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:19.812910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:19.812910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:10:19.812910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:10:19.812910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:10:19.812910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:10:19.812910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:19.812910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:19.812910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:19.812910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:10:19.812910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:19.812910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:19.812910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:19.812910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:10:19.812910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:19.812910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:10:19.812910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:10:19.812910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:19.812910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:10:19.812910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:19.814770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:10:19.814770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:10:19.814770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:19.814770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:10:19.814770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:19.814770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:10:19.983448 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:10:19.983909 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:10:19.984084 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:10:19.985214 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:10:19.985513 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:10:19.985739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:19.986631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:10:19.986631 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:19.986631 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:10:19.986631 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:10:19.986631 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:19.986631 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:10:19.986631 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:19.986631 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:10:19.987989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:19.987989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:10:19.987989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:19.987989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:10:19.987989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:10:19.987989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:10:19.987989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:10:19.987989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:19.987989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:19.987989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:19.987989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:10:19.987989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:19.988931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:19.988931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:19.988931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:10:19.988931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:19.988931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:10:19.988931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:10:19.988931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:19.988931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:10:19.988931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:19.988931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:10:19.988931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:10:19.988931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:19.988931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:10:19.989461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:19.989461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:10:19.989775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:19.989775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:10:19.989775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:10:19.989775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:10:19.989775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:10:19.989775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:10:19.989775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:19.989775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:10:19.989775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:10:19.989775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:10:19.989775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:10:19.989775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:10:19.989775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:19.989775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:19.989775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:10:19.989775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:10:19.991457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:10:19.991457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:10:19.991457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:19.991457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:19.991457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:19.991457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:10:19.991457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:19.991457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:19.991457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:19.991457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:10:19.991457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:19.991457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:10:19.991457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:10:19.991457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:19.991457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:10:19.991457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:19.991457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:10:19.991457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:10:19.991457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:19.993491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:10:19.993491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:19.993491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:10:19.993491 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:10:19.993491 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:10:19.993491 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:10:19.993491 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:10:19.993491 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:10:19.994078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:19.994078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:10:19.994078 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:19.994078 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:10:19.994078 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:10:19.994078 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:19.994078 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:10:19.994078 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:19.994078 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:10:19.994538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:19.994538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:10:19.994538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:10:19.994538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:10:19.994538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:19.994538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:10:19.994538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:19.994538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:10:19.994538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:19.994538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:10:19.994538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:19.994538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:10:19.994538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:19.994538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:10:19.994538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:10:19.995266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:19.995266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:10:19.995266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:19.995266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:10:19.995266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:10:19.995266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:19.995266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:10:19.995266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:19.995266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:10:19.995266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:19.995266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:10:19.995266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:19.995266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:10:19.995266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:10:19.995954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:10:19.995954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:10:19.995954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:10:19.995954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:10:19.995954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:10:19.995954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:19.995954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:19.995954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:19.995954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:10:19.995954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:19.995954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:19.995954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:19.995954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:10:19.996696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:19.996696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:10:19.996696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:10:19.996696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:19.996696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:10:19.996696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:19.996696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:10:19.996696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:10:19.996696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:19.996696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:10:19.996696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:19.996696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:10:19.997097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:19.997097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:10:19.997097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:10:19.997097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:10:19.997097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:10:19.997097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:10:19.997097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:19.997097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:10:19.997097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:10:19.997097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:10:19.997097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:10:19.997097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:10:19.997097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:19.997097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:19.997097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:10:19.997097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:10:19.997382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:10:19.997382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:10:19.997382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:10:19.997382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:10:19.997382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:10:19.997382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:19.997382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:19.997382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:19.997382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:10:19.997382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:19.997382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:19.997382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:19.997382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:10:19.997382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:19.997382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:10:19.997382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:10:19.997382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:19.997382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:10:19.997727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:19.997727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:10:19.997727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:10:19.997727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:19.997727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:10:19.997727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:19.997727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:10:20.001223 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:10:20.001494 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:10:20.001626 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:10:20.001751 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:10:20.002162 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:10:20.002398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:20.002966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:10:20.002966 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:20.002966 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:10:20.002966 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:10:20.002966 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:20.002966 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:10:20.002966 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:20.002966 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:10:20.003688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:20.003688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:10:20.003688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:10:20.003688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:10:20.003688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:20.003688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:10:20.003688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:20.003688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:10:20.003688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:20.003688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:10:20.003688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:20.003688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:10:20.003688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:20.003688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:10:20.003688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:10:20.004512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:20.004512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:10:20.004512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:20.004512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:10:20.004512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:10:20.004512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:20.004512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:10:20.004512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:20.004512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:10:20.004964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:20.004964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:10:20.004964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:20.004964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:10:20.004964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:10:20.004964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:10:20.004964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:10:20.004964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:10:20.004964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:10:20.004964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:20.004964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:20.005482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:20.005482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:10:20.005482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:20.005482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:20.005482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:20.005482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:10:20.005482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:20.005482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:10:20.005482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:10:20.005482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:20.005482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:10:20.005482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:20.005482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:10:20.005482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:10:20.006223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:20.006223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:10:20.006223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:20.006223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:10:20.006223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:20.006223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:10:20.006223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:10:20.006223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:10:20.006223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:10:20.006223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:10:20.006223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:20.006223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:10:20.006223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:10:20.006223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:10:20.006895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:10:20.006895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:10:20.006895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:20.006895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:20.006895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:10:20.006895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:10:20.006895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:10:20.006895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:10:20.006895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:10:20.006895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:10:20.006895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:20.006895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:20.006895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:20.006895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:10:20.006895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:20.006895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:20.006895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:20.006895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:10:20.006895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:20.007736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:10:20.007736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:10:20.007736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:20.007736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:10:20.007736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:20.007736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:10:20.007736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:10:20.007736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:20.007736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:10:20.007736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:20.007736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:10:20.010650 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:10:20.010941 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:10:20.011046 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:10:20.011173 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:10:20.011430 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:10:20.011639 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:10:20.011762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:20.012310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:10:20.012310 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:20.012310 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:10:20.012310 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:10:20.012310 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:20.012310 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:10:20.012310 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:20.012310 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:10:20.013136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:20.013136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:10:20.013136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:20.013136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:10:20.013136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:10:20.013136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:10:20.013136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:10:20.013136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:20.013136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:20.013136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:20.013136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:10:20.013136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:20.013715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:20.013715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:20.013715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:10:20.013715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:20.013715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:10:20.013715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:10:20.013715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:20.013715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:10:20.013715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:20.013715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:10:20.013715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:10:20.013715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:20.013715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:10:20.014103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:20.014103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:10:20.014103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:20.014103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:10:20.014103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:10:20.014103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:10:20.014103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:10:20.014103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:10:20.014103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:20.014103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:10:20.014103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:10:20.014103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:10:20.014103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:10:20.014103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:10:20.014103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:20.014637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:20.014637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:10:20.014637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:10:20.014637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:10:20.014637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:10:20.014637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:20.014637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:20.014637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:20.014637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:10:20.014637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:20.014637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:20.014637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:20.014637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:10:20.014637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:20.014637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:10:20.014637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:10:20.014637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:20.014637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:10:20.014637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:20.015165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:10:20.015165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:10:20.015165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:20.015165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:10:20.015165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:20.015165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:10:24.772081 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:10:24.772745 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:10:24.772745 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:10:24.773944 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:10:24.776575 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG dbcounter [-] [105581] Writing DB stats neutron:SELECT=490,neutron:UPDATE=3 {{(pid=105581) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 14:10:24.807131 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:10:24.807342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:24.808238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:10:24.808238 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:24.808238 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:10:24.808238 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:10:24.808238 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:24.808238 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:10:24.808238 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:24.808238 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:10:24.809859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:24.809859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:10:24.809859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:24.809859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:10:24.809859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:10:24.809859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:10:24.809859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:10:24.809859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:24.809859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:24.809859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:24.809859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:10:24.809859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:24.810547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:24.810547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:24.810547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:10:24.810547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:24.810547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:10:24.810547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:10:24.810547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:24.810547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:10:24.810547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:24.810547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:10:24.810547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:10:24.810547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:24.810547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:10:24.811271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:24.811271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:10:24.811633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:24.811633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:10:24.811633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:10:24.811633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:10:24.811633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:10:24.811633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:10:24.811633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:24.811633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:10:24.811633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:10:24.811633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:10:24.811633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:10:24.811633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:10:24.811633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:24.811633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:24.811633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:10:24.811633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:10:24.812369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:10:24.812369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:10:24.812369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:24.812369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:24.812369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:24.812369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:10:24.812369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:24.812369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:24.812369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:24.812369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:10:24.812369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:24.812369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:10:24.812369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:10:24.812369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:24.812369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:10:24.812369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:24.812369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:10:24.812369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:10:24.812369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:24.813173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:10:24.813173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:24.813173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:10:24.984321 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:10:24.984769 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:10:24.984953 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:10:24.985984 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:10:24.986238 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:10:24.986400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:24.987097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:10:24.987097 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:24.987097 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:10:24.987097 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:10:24.987097 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:24.987097 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:10:24.987097 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:24.987097 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:10:24.988596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:24.988596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:10:24.988596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:24.988596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:10:24.988596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:10:24.988596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:10:24.988596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:10:24.988596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:24.988596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:24.988596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:24.988596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:10:24.988596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:24.989155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:24.989155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:24.989155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:10:24.989155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:24.989155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:10:24.989155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:10:24.989155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:24.989155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:10:24.989155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:24.989155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:10:24.989155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:10:24.989155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:24.989155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:10:24.989752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:24.989752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:10:24.990277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:24.990277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:10:24.990277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:10:24.990277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:10:24.990277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:10:24.990277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:10:24.990277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:24.990277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:10:24.990277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:10:24.990277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:10:24.990277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:10:24.990277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:10:24.990277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:24.990277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:24.990277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:10:24.990277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:10:24.991093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:10:24.991093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:10:24.991093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:24.991093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:24.991093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:24.991093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:10:24.991093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:24.991093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:24.991093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:24.991093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:10:24.991093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:24.991093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:10:24.991093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:10:24.991093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:24.991093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:10:24.991093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:24.991093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:10:24.991093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:10:24.991093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:24.991943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:10:24.991943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:24.991943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:10:24.992428 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:10:24.992834 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:10:24.993006 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:10:24.993186 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:10:24.993578 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:10:24.993842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:24.994461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:10:24.994461 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:24.994461 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:10:24.994461 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:10:24.994461 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:24.994461 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:10:24.994461 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:24.994461 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:10:24.995649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:24.995649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:10:24.995649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:10:24.995649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:10:24.995649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:24.995649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:10:24.995649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:24.995649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:10:24.995649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:24.995649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:10:24.995649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:24.995649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:10:24.995649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:24.995649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:10:24.995649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:10:24.996194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:24.996194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:10:24.996194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:24.996194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:10:24.996194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:10:24.996194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:24.996194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:10:24.996194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:24.996194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:10:24.996982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:24.996982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:10:24.996982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:24.996982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:10:24.996982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:10:24.996982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:10:24.996982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:10:24.996982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:10:24.996982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:10:24.996982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:10:24.996982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:24.996982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:24.997315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:24.997315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:10:24.997315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:24.997315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:24.997315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:24.997315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:10:24.997315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:24.997315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:10:24.997315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:10:24.997315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:24.997315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:10:24.997315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:24.997315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:10:24.997315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:10:24.997731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:24.997731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:10:24.997731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:24.997731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:10:24.998426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:24.998426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:10:24.998426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:10:24.998426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:10:24.998426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:10:24.998426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:10:24.998426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:24.998426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:10:24.998426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:10:24.998426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:10:24.998426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:10:24.998426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:10:24.998426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:24.998426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:24.998426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:10:24.998426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:10:24.998843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:10:24.998843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:10:24.998843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:10:24.998843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:10:24.998843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:10:24.998843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:24.998843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:24.998843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:24.998843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:10:24.998843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:24.998843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:24.998843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:24.998843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:10:24.998843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:24.998843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:10:24.998843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:10:24.998843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:24.998843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:10:24.999314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:24.999314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:10:24.999314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:10:24.999314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:24.999314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:10:24.999314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:24.999314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:10:25.001596 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:10:25.001873 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:10:25.002004 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:10:25.002125 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:10:25.002470 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:10:25.002692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:25.003159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:10:25.003159 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:25.003159 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:10:25.003159 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:10:25.003159 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:25.003159 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:10:25.003159 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:25.003159 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:10:25.003914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:25.003914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:10:25.003914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:10:25.003914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:10:25.003914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:25.003914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:10:25.003914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:25.003914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:10:25.003914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:25.003914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:10:25.003914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:25.003914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:10:25.003914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:25.003914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:10:25.003914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:10:25.004325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:25.004325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:10:25.004325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:25.004325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:10:25.004325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:10:25.004325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:25.004325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:10:25.004325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:25.004325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:10:25.004814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:25.004814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:10:25.004814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:25.004814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:10:25.004814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:10:25.004814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:10:25.004814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:10:25.004814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:10:25.004814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:10:25.004814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:25.004814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:25.005123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:25.005123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:10:25.005123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:25.005123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:25.005123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:25.005123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:10:25.005123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:25.005123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:10:25.005123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:10:25.005123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:25.005123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:10:25.005123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:25.005123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:10:25.005123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:10:25.005501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:25.005501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:10:25.005501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:25.005501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:10:25.005895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:25.005895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:10:25.005895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:10:25.005895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:10:25.005895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:10:25.005895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:10:25.005895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:25.005895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:10:25.005895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:10:25.005895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:10:25.005895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:10:25.005895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:10:25.005895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:25.005895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:25.005895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:10:25.005895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:10:25.005895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:10:25.006368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:10:25.006368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:10:25.006368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:10:25.006368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:25.006368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:25.006368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:25.006368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:10:25.006368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:25.006368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:25.006368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:25.006368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:10:25.006368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:25.006368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:10:25.006368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:10:25.006368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:25.006368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:10:25.006368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:25.006368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:10:25.006368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:10:25.006914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:25.006914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:10:25.006914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:25.006914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:10:25.011185 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:10:25.011499 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:10:25.011607 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:10:25.011766 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:10:25.012233 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:10:25.012741 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:10:25.012966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:25.013766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:10:25.013766 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:25.013766 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:10:25.013766 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:10:25.013766 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:25.013766 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:10:25.013766 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:25.013766 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:10:25.014620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:25.014620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:10:25.014620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:25.014620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:10:25.014620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:10:25.014620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:10:25.014620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:10:25.014620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:25.014620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:25.014620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:25.014620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:10:25.014620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:25.015232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:25.015232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:25.015232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:10:25.015232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:25.015232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:10:25.015232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:10:25.015232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:25.015232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:10:25.015232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:25.015232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:10:25.015232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:10:25.015232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:25.015232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:10:25.015829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:25.015829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:10:25.015829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:25.015829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:10:25.015829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:10:25.015829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:10:25.015829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:10:25.015829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:10:25.015829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:25.015829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:10:25.015829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:10:25.015829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:10:25.015829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:10:25.015829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:10:25.015829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:25.016553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:25.016553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:10:25.016553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:10:25.016553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:10:25.016553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:10:25.016553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:25.016553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:25.016553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:25.016553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:10:25.016553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:25.016553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:25.016553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:25.016553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:10:25.016553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:25.016553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:10:25.016553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:10:25.016553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:25.016553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:10:25.016553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:25.017390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:10:25.017390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:10:25.017390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:25.017390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:10:25.017390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:25.017390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:10:29.772724 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:10:29.773512 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:10:29.773512 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:10:29.774699 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:10:29.807378 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:10:29.807815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:29.808768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:10:29.808768 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:29.808768 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:10:29.808768 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:10:29.808768 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:29.808768 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:10:29.808768 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:29.808768 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:10:29.809833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:29.809833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:10:29.809833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:29.809833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:10:29.809833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:10:29.809833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:10:29.809833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:10:29.809833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:29.809833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:29.809833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:29.809833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:10:29.809833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:29.810181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:29.810181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:29.810181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:10:29.810181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:29.810181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:10:29.810181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:10:29.810181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:29.810181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:10:29.810181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:29.810181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:10:29.810181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:10:29.810181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:29.810181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:10:29.810533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:29.810533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:10:29.810905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:29.810905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:10:29.810905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:10:29.810905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:10:29.810905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:10:29.810905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:10:29.810905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:29.810905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:10:29.810905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:10:29.810905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:10:29.810905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:10:29.810905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:10:29.810905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:29.810905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:29.810905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:10:29.810905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:10:29.811498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:10:29.811498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:10:29.811498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:29.811498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:29.811498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:29.811498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:10:29.811498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:29.811498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:29.811498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:29.811498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:10:29.811498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:29.811498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:10:29.811498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:10:29.811498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:29.811498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:10:29.811498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:29.811498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:10:29.811498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:10:29.811498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:29.812217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:10:29.812217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:29.812217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:10:29.984955 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:10:29.985621 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:10:29.985814 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:10:29.987031 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:10:29.987425 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:10:29.987696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:29.988648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:10:29.988648 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:29.988648 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:10:29.988648 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:10:29.988648 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:29.988648 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:10:29.988648 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:29.988648 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:10:29.990294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:29.990294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:10:29.990294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:29.990294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:10:29.990294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:10:29.990294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:10:29.990294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:10:29.990294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:29.990294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:29.990294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:29.990294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:10:29.990294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:29.990647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:29.990647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:29.990647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:10:29.990647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:29.990647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:10:29.990647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:10:29.990647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:29.990647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:10:29.990647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:29.990647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:10:29.990647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:10:29.990647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:29.990647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:10:29.990978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:29.990978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:10:29.992371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:29.992371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:10:29.992371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:10:29.992371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:10:29.992371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:10:29.992371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:10:29.992371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:29.992371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:10:29.992371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:10:29.992371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:10:29.992371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:10:29.992371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:10:29.992371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:29.992371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:29.992371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:10:29.992371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:10:29.992790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:10:29.992790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:10:29.992790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:29.992790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:29.992790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:29.992790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:10:29.992790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:29.992790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:29.992790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:29.992790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:10:29.992790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:29.992790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:10:29.992790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:10:29.992790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:29.992790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:10:29.992790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:29.992790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:10:29.992790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:10:29.992790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:29.993352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:10:29.993352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:29.993352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:10:29.993352 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:10:29.993352 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:10:29.993352 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:10:29.993352 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:10:29.993907 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:10:29.994147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:29.994834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:10:29.994834 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:29.994834 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:10:29.994834 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:10:29.994834 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:29.994834 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:10:29.994834 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:29.994834 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:10:29.995596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:29.995596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:10:29.995596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:10:29.995596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:10:29.995596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:29.995596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:10:29.995596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:29.995596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:10:29.995596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:29.995596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:10:29.995596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:29.995596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:10:29.995596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:29.995596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:10:29.995596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:10:29.996395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:29.996395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:10:29.996395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:29.996395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:10:29.996395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:10:29.996395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:29.996395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:10:29.996395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:29.996395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:10:29.996395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:29.996395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:10:29.996395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:29.996395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:10:29.996395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:10:29.997155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:10:29.997155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:10:29.997155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:10:29.997155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:10:29.997155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:10:29.997155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:29.997155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:29.997155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:29.997155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:10:29.997155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:29.997155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:29.997155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:29.997155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:10:29.997681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:29.997681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:10:29.997681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:10:29.997681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:29.997681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:10:29.997681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:29.997681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:10:29.997681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:10:29.997681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:29.997681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:10:29.997681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:29.997681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:10:29.997983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:29.997983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:10:29.997983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:10:29.997983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:10:29.997983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:10:29.997983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:10:29.997983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:29.997983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:10:29.997983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:10:29.997983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:10:29.997983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:10:29.997983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:10:29.997983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:29.997983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:29.997983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:10:29.997983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:10:29.998358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:10:29.998358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:10:29.998358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:10:29.998358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:10:29.998358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:10:29.998358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:29.998358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:29.998358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:29.998358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:10:29.998358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:29.998358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:29.998358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:29.998358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:10:29.998358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:29.998358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:10:29.998358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:10:29.998358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:29.998358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:10:29.998804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:29.998804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:10:29.998804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:10:29.998804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:29.998804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:10:29.998804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:29.998804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:10:30.002025 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:10:30.002294 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:10:30.002393 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:10:30.002540 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:10:30.002965 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:10:30.003339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:30.004033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:10:30.004033 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:30.004033 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:10:30.004033 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:10:30.004033 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:30.004033 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:10:30.004033 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:30.004033 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:10:30.004780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:30.004780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:10:30.004780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:10:30.004780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:10:30.004780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:30.004780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:10:30.004780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:30.004780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:10:30.004780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:30.004780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:10:30.004780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:30.004780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:10:30.004780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:30.004780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:10:30.004780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:10:30.005160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:30.005160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:10:30.005160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:30.005160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:10:30.005160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:10:30.005160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:30.005160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:10:30.005160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:30.005160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:10:30.005732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:30.005732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:10:30.005732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:30.005732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:10:30.005732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:10:30.005732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:10:30.005732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:10:30.005732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:10:30.005732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:10:30.005732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:30.005732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:30.006315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:30.006315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:10:30.006315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:30.006315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:30.006315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:30.006315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:10:30.006315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:30.006315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:10:30.006315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:10:30.006315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:30.006315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:10:30.006315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:30.006315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:10:30.006315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:10:30.007012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:30.007012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:10:30.007012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:30.007012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:10:30.007012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:30.007012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:10:30.007012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:10:30.007012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:10:30.007012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:10:30.007012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:10:30.007012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:30.007012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:10:30.007012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:10:30.007012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:10:30.007388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:10:30.007388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:10:30.007388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:30.007388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:30.007388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:10:30.007388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:10:30.007388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:10:30.007388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:10:30.007388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:10:30.007388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:10:30.007388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:30.007388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:30.007388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:30.007388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:10:30.007388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:30.007388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:30.007388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:30.007388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:10:30.007388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:30.007897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:10:30.007897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:10:30.007897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:30.007897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:10:30.007897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:30.007897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:10:30.007897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:10:30.007897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:30.007897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:10:30.007897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:30.007897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:10:30.011561 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:10:30.011809 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:10:30.011909 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:10:30.012076 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:10:30.012512 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:10:30.012843 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:10:30.013061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:30.013752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:10:30.013752 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:30.013752 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:10:30.013752 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:10:30.013752 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:30.013752 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:10:30.013752 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:30.013752 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:10:30.014573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:30.014573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:10:30.014573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:30.014573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:10:30.014573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:10:30.014573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:10:30.014573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:10:30.014573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:30.014573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:30.014573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:30.014573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:10:30.014573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:30.014878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:30.014878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:30.014878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:10:30.014878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:30.014878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:10:30.014878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:10:30.014878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:30.014878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:10:30.014878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:30.014878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:10:30.014878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:10:30.014878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:30.014878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:10:30.015194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:30.015194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:10:30.015598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:30.015598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:10:30.015598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:10:30.015598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:10:30.015598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:10:30.015598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:10:30.015598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:30.015598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:10:30.015598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:10:30.015598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:10:30.015598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:10:30.015598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:10:30.015598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:30.015598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:30.015598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:10:30.015598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:10:30.016447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:10:30.016447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:10:30.016447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:30.016447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:30.016447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:30.016447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:10:30.016447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:30.016447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:30.016447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:30.016447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:10:30.016447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:30.016447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:10:30.016447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:10:30.016447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:30.016447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:10:30.016447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:30.016447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:10:30.016447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:10:30.016447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:30.017229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:10:30.017229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:30.017229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:10:34.773545 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:10:34.774379 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:10:34.774379 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:10:34.775653 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:10:34.808861 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:10:34.809277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:34.810159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:10:34.810159 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:34.810159 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:10:34.810159 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:10:34.810159 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:34.810159 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:10:34.810159 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:34.810159 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:10:34.811443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:34.811443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:10:34.811443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:34.811443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:10:34.811443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:10:34.811443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:10:34.811443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:10:34.811443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:34.811443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:34.811443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:34.811443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:10:34.811443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:34.812063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:34.812063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:34.812063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:10:34.812063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:34.812063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:10:34.812063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:10:34.812063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:34.812063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:10:34.812063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:34.812063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:10:34.812063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:10:34.812063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:34.812063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:10:34.812690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:34.812690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:10:34.812690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:34.812690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:10:34.812690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:10:34.812690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:10:34.812690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:10:34.812690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:10:34.812690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:34.812690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:10:34.812690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:10:34.812690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:10:34.812690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:10:34.812690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:10:34.812690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:34.813365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:34.813365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:10:34.813365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:10:34.813365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:10:34.813365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:10:34.813365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:34.813365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:34.813365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:34.813365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:10:34.813365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:34.813365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:34.813365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:34.813365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:10:34.813365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:34.813365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:10:34.813365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:10:34.813365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:34.813365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:10:34.813365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:34.814202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:10:34.814202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:10:34.814202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:34.814202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:10:34.814202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:34.814202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:10:34.985710 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:10:34.986158 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:10:34.986344 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:10:34.987355 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:10:34.987727 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:10:34.987964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:34.988844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:10:34.988844 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:34.988844 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:10:34.988844 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:10:34.988844 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:34.988844 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:10:34.988844 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:34.988844 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:10:34.990291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:34.990291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:10:34.990291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:34.990291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:10:34.990291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:10:34.990291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:10:34.990291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:10:34.990291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:34.990291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:34.990291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:34.990291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:10:34.990291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:34.990882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:34.990882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:34.990882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:10:34.990882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:34.990882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:10:34.990882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:10:34.990882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:34.990882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:10:34.990882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:34.990882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:10:34.990882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:10:34.990882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:34.990882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:10:34.991434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:34.991434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:10:34.993160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:34.993160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:10:34.993160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:10:34.993160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:10:34.993160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:10:34.993160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:10:34.993160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:34.993160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:10:34.993160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:10:34.993160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:10:34.993160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:10:34.993160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:10:34.993160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:34.993160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:34.993160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:10:34.993160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:10:34.993957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:10:34.993957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:10:34.993957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:34.993957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:34.993957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:34.993957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:10:34.993957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:34.993957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:34.993957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:34.993957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:10:34.993957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:34.993957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:10:34.993957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:10:34.993957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:34.993957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:10:34.993957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:34.993957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:10:34.993957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:10:34.993957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:34.994833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:10:34.994833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:34.994833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:10:34.994833 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:10:34.994833 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:10:34.994833 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:10:34.994833 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:10:34.995244 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:10:34.995244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:34.995617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:10:34.995617 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:34.995617 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:10:34.995617 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:10:34.995617 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:34.995617 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:10:34.995617 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:34.995617 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:10:34.996351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:34.996351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:10:34.996351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:10:34.996351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:10:34.996351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:34.996351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:10:34.996351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:34.996351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:10:34.996351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:34.996351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:10:34.996351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:34.996351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:10:34.996351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:34.996351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:10:34.996351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:10:34.996846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:34.996846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:10:34.996846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:34.996846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:10:34.996846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:10:34.996846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:34.996846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:10:34.996846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:34.996846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:10:34.997226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:34.997226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:10:34.997226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:34.997226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:10:34.997226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:10:34.997226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:10:34.997226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:10:34.997226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:10:34.997226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:10:34.997226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:10:34.997226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:34.997226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:34.997604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:34.997604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:10:34.997604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:34.997604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:34.997604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:34.997604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:10:34.997604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:34.997604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:10:34.997604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:10:34.997604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:34.997604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:10:34.997604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:34.997604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:10:34.997604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:10:34.997951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:34.997951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:10:34.997951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:34.997951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:10:34.998460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:34.998460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:10:34.998460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:10:34.998460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:10:34.998460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:10:34.998460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:10:34.998460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:34.998460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:10:34.998460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:10:34.998460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:10:34.998460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:10:34.998460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:10:34.998460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:34.998460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:34.998460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:10:34.998460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:10:34.998874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:10:34.998874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:10:34.998874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:10:34.998874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:10:34.998874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:10:34.998874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:34.998874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:34.998874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:34.998874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:10:34.998874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:34.998874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:34.998874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:34.998874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:10:34.998874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:34.998874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:10:34.998874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:10:34.998874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:34.998874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:10:34.999349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:34.999349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:10:34.999349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:10:34.999349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:34.999349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:10:34.999349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:34.999349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:10:35.002508 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:10:35.002814 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:10:35.002931 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:10:35.003054 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:10:35.003533 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:10:35.003886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:35.004625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:10:35.004625 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:35.004625 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:10:35.004625 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:10:35.004625 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:35.004625 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:10:35.004625 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:35.004625 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:10:35.005360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:35.005360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:10:35.005360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:10:35.005360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:10:35.005360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:35.005360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:10:35.005360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:35.005360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:10:35.005360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:35.005360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:10:35.005360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:35.005360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:10:35.005360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:35.005360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:10:35.005360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:10:35.005785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:35.005785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:10:35.005785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:35.005785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:10:35.005785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:10:35.005785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:35.005785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:10:35.005785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:35.005785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:10:35.006298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:35.006298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:10:35.006298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:35.006298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:10:35.006298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:10:35.006298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:10:35.006298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:10:35.006298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:10:35.006298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:10:35.006298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:35.006298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:35.006638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:35.006638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:10:35.006638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:35.006638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:35.006638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:35.006638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:10:35.006638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:35.006638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:10:35.006638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:10:35.006638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:35.006638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:10:35.006638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:35.006638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:10:35.006638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:10:35.006973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:35.006973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:10:35.006973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:35.006973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:10:35.007338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:35.007338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:10:35.007338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:10:35.007338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:10:35.007338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:10:35.007338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:10:35.007338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:35.007338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:10:35.007338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:10:35.007338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:10:35.007338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:10:35.007338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:10:35.007338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:35.007338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:35.007338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:10:35.007338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:10:35.007338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:10:35.007801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:10:35.007801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:10:35.007801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:10:35.007801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:35.007801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:35.007801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:35.007801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:10:35.007801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:35.007801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:35.007801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:35.007801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:10:35.007801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:35.007801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:10:35.007801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:10:35.007801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:35.007801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:10:35.007801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:35.007801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:10:35.007801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:10:35.008393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:35.008393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:10:35.008393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:35.008393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:10:35.011954 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:10:35.012293 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:10:35.012417 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:10:35.012633 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:10:35.013028 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:10:35.013370 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:10:35.013626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:35.014312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:10:35.014312 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:35.014312 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:10:35.014312 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:10:35.014312 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:35.014312 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:10:35.014312 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:35.014312 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:10:35.015193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:35.015193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:10:35.015193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:35.015193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:10:35.015193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:10:35.015193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:10:35.015193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:10:35.015193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:35.015193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:35.015193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:35.015193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:10:35.015193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:35.015553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:35.015553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:35.015553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:10:35.015553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:35.015553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:10:35.015553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:10:35.015553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:35.015553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:10:35.015553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:35.015553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:10:35.015553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:10:35.015553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:35.015553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:10:35.015886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:35.015886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:10:35.016416 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:35.016416 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:10:35.016416 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:10:35.016416 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:10:35.016416 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:10:35.016416 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:10:35.016416 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:35.016416 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:10:35.016416 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:10:35.016416 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:10:35.016416 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:10:35.016416 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:10:35.016416 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:35.016416 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:35.016416 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:10:35.016416 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:10:35.016847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:10:35.016847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:10:35.016847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:35.016847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:35.016847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:35.016847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:10:35.016847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:35.016847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:35.016847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:35.016847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:10:35.016847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:35.016847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:10:35.016847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:10:35.016847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:35.016847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:10:35.016847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:35.016847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:10:35.016847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:10:35.016847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:35.017328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:10:35.017328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:35.017328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:10:39.774319 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:10:39.775006 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:10:39.775202 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:10:39.776491 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:10:39.813879 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:10:39.814089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:39.815050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:10:39.815050 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:39.815050 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:10:39.815050 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:10:39.815050 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:39.815050 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:10:39.815050 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:39.815050 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:10:39.816250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:39.816250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:10:39.816250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:39.816250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:10:39.816250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:10:39.816250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:10:39.816250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:10:39.816250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:39.816250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:39.816250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:39.816250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:10:39.816250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:39.816865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:39.816865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:39.816865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:10:39.816865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:39.816865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:10:39.816865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:10:39.816865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:39.816865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:10:39.816865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:39.816865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:10:39.816865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:10:39.816865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:39.816865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:10:39.817452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:39.817452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:10:39.817452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:39.817452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:10:39.817452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:10:39.817452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:10:39.817452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:10:39.817452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:10:39.817452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:39.817452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:10:39.817452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:10:39.817452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:10:39.817452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:10:39.817452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:10:39.817452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:39.818104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:39.818104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:10:39.818104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:10:39.818104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:10:39.818104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:10:39.818104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:39.818104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:39.818104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:39.818104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:10:39.818104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:39.818104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:39.818104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:39.818104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:10:39.818104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:39.818104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:10:39.818104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:10:39.818104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:39.818104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:10:39.818104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:39.818906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:10:39.818906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:10:39.818906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:39.818906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:10:39.818906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:39.818906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:10:39.986395 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:10:39.987072 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:10:39.987186 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:10:39.988693 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:10:39.988865 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:10:39.989010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:39.990461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:10:39.990461 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:39.990461 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:10:39.990461 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:10:39.990461 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:39.990461 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:10:39.990461 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:39.990461 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:10:39.991314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:39.991314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:10:39.991314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:39.991314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:10:39.991314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:10:39.991314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:10:39.991314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:10:39.991314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:39.991314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:39.991314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:39.991314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:10:39.991314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:39.991869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:39.991869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:39.991869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:10:39.991869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:39.991869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:10:39.991869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:10:39.991869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:39.991869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:10:39.991869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:39.991869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:10:39.991869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:10:39.991869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:39.991869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:10:39.992493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:39.992493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:10:39.993098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:39.993098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:10:39.993098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:10:39.993098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:10:39.993098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:10:39.993098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:10:39.993098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:39.993098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:10:39.993098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:10:39.993098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:10:39.993098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:10:39.993098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:10:39.993098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:39.993098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:39.993098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:10:39.993098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:10:39.993830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:10:39.993830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:10:39.993830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:39.993830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:39.993830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:39.993830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:10:39.993830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:39.993830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:39.993830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:39.993830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:10:39.993830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:39.993830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:10:39.993830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:10:39.993830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:39.993830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:10:39.993830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:39.993830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:10:39.993830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:10:39.993830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:39.994798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:10:39.994798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:39.994798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:10:39.995181 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:10:39.995754 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:10:39.995933 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:10:39.996137 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:10:39.997111 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:10:39.997384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:39.997949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:10:39.997949 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:39.997949 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:10:39.997949 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:10:39.997949 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:39.997949 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:10:39.997949 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:39.997949 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:10:39.998502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:39.998502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:10:39.998502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:10:39.998502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:10:39.998502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:39.998502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:10:39.998502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:39.998502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:10:39.998502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:39.998502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:10:39.998502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:39.998502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:10:39.998502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:39.998502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:10:39.998502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:10:39.999242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:39.999242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:10:39.999242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:39.999242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:10:39.999242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:10:39.999242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:39.999242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:10:39.999242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:39.999242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:10:39.999242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:39.999242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:10:39.999242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:39.999242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:10:39.999242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:10:39.999926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:10:39.999926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:10:39.999926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:10:39.999926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:10:39.999926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:10:39.999926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:39.999926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:39.999926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:39.999926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:10:39.999926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:39.999926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:39.999926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:39.999926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:10:40.000610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:40.000610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:10:40.000610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:10:40.000610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:40.000610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:10:40.000610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:40.000610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:10:40.000610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:10:40.000610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:40.000610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:10:40.000610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:40.000610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:10:40.001129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:40.001129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:10:40.001129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:10:40.001129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:10:40.001129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:10:40.001129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:10:40.001129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:40.001129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:10:40.001129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:10:40.001129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:10:40.001129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:10:40.001129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:10:40.001129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:40.001129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:40.001129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:10:40.001129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:10:40.001706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:10:40.001706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:10:40.001706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:10:40.001706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:10:40.001706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:10:40.001706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:40.001706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:40.001706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:40.001706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:10:40.001706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:40.001706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:40.001706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:40.001706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:10:40.001706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:40.001706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:10:40.001706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:10:40.001706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:40.001706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:10:40.002158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:40.002158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:10:40.002158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:10:40.002158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:40.002158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:10:40.002158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:40.002158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:10:40.002834 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:10:40.003029 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:10:40.003099 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:10:40.003193 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:10:40.003536 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:10:40.003761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:40.004222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:10:40.004222 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:40.004222 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:10:40.004222 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:10:40.004222 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:40.004222 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:10:40.004222 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:40.004222 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:10:40.004780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:40.004780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:10:40.004780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:10:40.004780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:10:40.004780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:40.004780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:10:40.004780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:40.004780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:10:40.004780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:40.004780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:10:40.004780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:40.004780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:10:40.004780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:40.004780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:10:40.004780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:10:40.005494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:40.005494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:10:40.005494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:40.005494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:10:40.005494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:10:40.005494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:40.005494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:10:40.005494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:40.005494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:10:40.005494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:40.005494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:10:40.005494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:40.005494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:10:40.005494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:10:40.005860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:10:40.005860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:10:40.005860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:10:40.005860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:10:40.005860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:40.005860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:40.005860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:40.005860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:10:40.005860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:40.005860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:40.005860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:40.005860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:10:40.005860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:40.006238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:10:40.006238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:10:40.006238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:40.006238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:10:40.006238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:40.006238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:10:40.006238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:10:40.006238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:40.006238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:10:40.006238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:40.006238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:10:40.006238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:40.006573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:10:40.006573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:10:40.006573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:10:40.006573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:10:40.006573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:10:40.006573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:40.006573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:10:40.006573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:10:40.006573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:10:40.006573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:10:40.006573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:10:40.006573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:40.006573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:40.006573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:10:40.006573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:10:40.006573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:10:40.006573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:10:40.006573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:10:40.006573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:10:40.007080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:40.007080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:40.007080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:40.007080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:10:40.007080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:40.007080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:40.007080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:40.007080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:10:40.007080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:40.007080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:10:40.007080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:10:40.007080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:40.007080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:10:40.007080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:40.007080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:10:40.007080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:10:40.007080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:40.007080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:10:40.007080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:40.007080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:10:40.012547 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:10:40.012798 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:10:40.012908 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:10:40.013061 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:10:40.013373 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:10:40.013652 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:10:40.013804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:40.014288 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:10:40.014288 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:40.014288 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:10:40.014288 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:10:40.014288 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:40.014288 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:10:40.014288 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:40.014288 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:10:40.015143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:40.015143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:10:40.015143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:40.015143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:10:40.015143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:10:40.015143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:10:40.015143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:10:40.015143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:40.015143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:40.015143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:40.015143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:10:40.015143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:40.015490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:40.015490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:40.015490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:10:40.015490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:40.015490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:10:40.015490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:10:40.015490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:40.015490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:10:40.015490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:40.015490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:10:40.015490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:10:40.015490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:40.015490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:10:40.015876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:40.015876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:10:40.016131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:40.016131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:10:40.016131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:10:40.016131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:10:40.016131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:10:40.016131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:10:40.016131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:40.016131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:10:40.016131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:10:40.016131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:10:40.016131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:10:40.016131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:10:40.016131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:40.016131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:40.016131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:10:40.016131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:10:40.016590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:10:40.016590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:10:40.016590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:40.016590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:40.016590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:40.016590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:10:40.016590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:40.016590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:40.016590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:40.016590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:10:40.016590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:40.016590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:10:40.016590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:10:40.016590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:40.016590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:10:40.016590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:40.016590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:10:40.016590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:10:40.016590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:40.017079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:10:40.017079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:40.017079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:10:44.774945 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:10:44.775839 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:10:44.775920 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:10:44.777262 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:10:44.814816 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:10:44.815244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:44.816127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:10:44.816127 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:44.816127 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:10:44.816127 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:10:44.816127 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:44.816127 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:10:44.816127 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:44.816127 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:10:44.816920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:44.816920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:10:44.816920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:44.816920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:10:44.816920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:10:44.816920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:10:44.816920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:10:44.816920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:44.816920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:44.816920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:44.816920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:10:44.816920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:44.817574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:44.817574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:44.817574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:10:44.817574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:44.817574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:10:44.817574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:10:44.817574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:44.817574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:10:44.817574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:44.817574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:10:44.817574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:10:44.817574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:44.817574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:10:44.818150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:44.818150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:10:44.818150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:44.818150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:10:44.818150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:10:44.818150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:10:44.818150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:10:44.818150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:10:44.818150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:44.818150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:10:44.818150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:10:44.818150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:10:44.818150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:10:44.818150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:10:44.818150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:44.818885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:44.818885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:10:44.818885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:10:44.818885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:10:44.818885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:10:44.818885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:44.818885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:44.818885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:44.818885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:10:44.818885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:44.818885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:44.818885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:44.818885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:10:44.818885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:44.818885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:10:44.818885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:10:44.818885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:44.818885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:10:44.818885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:44.819771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:10:44.819771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:10:44.819771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:44.819771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:10:44.819771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:44.819771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:10:44.987113 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:10:44.987784 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:10:44.987972 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:10:44.989167 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:10:44.989576 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:10:44.989862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:44.990796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:10:44.990796 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:44.990796 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:10:44.990796 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:10:44.990796 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:44.990796 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:10:44.990796 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:44.990796 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:10:44.992309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:44.992309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:10:44.992309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:44.992309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:10:44.992309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:10:44.992309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:10:44.992309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:10:44.992309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:44.992309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:44.992309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:44.992309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:10:44.992309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:44.992908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:44.992908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:44.992908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:10:44.992908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:44.992908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:10:44.992908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:10:44.992908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:44.992908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:10:44.992908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:44.992908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:10:44.992908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:10:44.992908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:44.992908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:10:44.993474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:44.993474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:10:44.994022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:44.994022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:10:44.994022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:10:44.994022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:10:44.994022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:10:44.994022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:10:44.994022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:44.994022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:10:44.994022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:10:44.994022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:10:44.994022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:10:44.994022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:10:44.994022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:44.994022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:44.994022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:10:44.994022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:10:44.994764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:10:44.994764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:10:44.994764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:44.994764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:44.994764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:44.994764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:10:44.994764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:44.994764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:44.994764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:44.994764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:10:44.994764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:44.994764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:10:44.994764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:10:44.994764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:44.994764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:10:44.994764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:44.994764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:10:44.994764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:10:44.994764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:44.995605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:10:44.995605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:44.995605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:10:44.995605 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:10:44.995839 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:10:44.995971 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:10:44.996159 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:10:44.996663 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:10:44.996944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:44.997649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:10:44.997649 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:44.997649 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:10:44.997649 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:10:44.997649 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:44.997649 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:10:44.997649 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:44.997649 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:10:44.998433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:44.998433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:10:44.998433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:10:44.998433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:10:44.998433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:44.998433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:10:44.998433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:44.998433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:10:44.998433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:44.998433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:10:44.998433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:44.998433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:10:44.998433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:44.998433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:10:44.998433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:10:44.999132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:44.999132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:10:44.999132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:44.999132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:10:44.999132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:10:44.999132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:44.999132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:10:44.999132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:44.999132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:10:44.999563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:44.999563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:10:44.999563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:44.999563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:10:44.999563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:10:44.999563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:10:44.999563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:10:44.999563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:10:44.999563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:10:44.999563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:10:44.999563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:44.999563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:45.000117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:45.000117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:10:45.000117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:45.000117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:45.000117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:45.000117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:10:45.000117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:45.000117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:10:45.000117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:10:45.000117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:45.000117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:10:45.000117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:45.000117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:10:45.000117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:10:45.000734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:45.000734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:10:45.000734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:45.000734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:10:45.000734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:45.000734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:10:45.000734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:10:45.000734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:10:45.000734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:10:45.000734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:10:45.000734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:45.000734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:10:45.000734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:10:45.000734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:10:45.001352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:10:45.001352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:10:45.001352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:45.001352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:45.001352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:10:45.001352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:10:45.001352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:10:45.001352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:10:45.001352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:10:45.001352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:10:45.001352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:10:45.001352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:45.001352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:45.001352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:45.001352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:10:45.001352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:45.001352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:45.001352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:45.001352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:10:45.002267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:45.002267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:10:45.002267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:10:45.002267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:45.002267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:10:45.002267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:45.002267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:10:45.002267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:10:45.002267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:45.002267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:10:45.002267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:45.002267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:10:45.003084 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:10:45.003306 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:10:45.003431 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:10:45.003566 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:10:45.003892 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:10:45.004107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:45.004582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:10:45.004582 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:45.004582 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:10:45.004582 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:10:45.004582 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:45.004582 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:10:45.004582 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:45.004582 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:10:45.005267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:45.005267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:10:45.005267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:10:45.005267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:10:45.005267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:45.005267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:10:45.005267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:45.005267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:10:45.005267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:45.005267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:10:45.005267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:45.005267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:10:45.005267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:45.005267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:10:45.005267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:10:45.005704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:45.005704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:10:45.005704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:45.005704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:10:45.005704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:10:45.005704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:45.005704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:10:45.005704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:45.005704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:10:45.006181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:45.006181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:10:45.006181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:45.006181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:10:45.006181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:10:45.006181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:10:45.006181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:10:45.006181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:10:45.006181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:10:45.006181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:45.006181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:45.006520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:45.006520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:10:45.006520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:45.006520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:45.006520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:45.006520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:10:45.006520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:45.006520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:10:45.006520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:10:45.006520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:45.006520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:10:45.006520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:45.006520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:10:45.006520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:10:45.006895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:45.006895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:10:45.006895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:45.006895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:10:45.007213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:45.007213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:10:45.007213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:10:45.007213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:10:45.007213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:10:45.007213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:10:45.007213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:45.007213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:10:45.007213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:10:45.007213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:10:45.007213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:10:45.007213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:10:45.007213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:45.007213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:45.007213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:10:45.007213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:10:45.007213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:10:45.007704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:10:45.007704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:10:45.007704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:10:45.007704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:45.007704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:45.007704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:45.007704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:10:45.007704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:45.007704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:45.007704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:45.007704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:10:45.007704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:45.007704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:10:45.007704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:10:45.007704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:45.007704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:10:45.007704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:45.007704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:10:45.007704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:10:45.008186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:45.008186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:10:45.008186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:45.008186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:10:45.012815 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:10:45.013038 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:10:45.013142 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:10:45.013285 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:10:45.013719 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:10:45.014084 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:10:45.014299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:45.015124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:10:45.015124 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:45.015124 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:10:45.015124 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:10:45.015124 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:45.015124 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:10:45.015124 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:45.015124 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:10:45.016412 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:45.016412 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:10:45.016412 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:45.016412 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:10:45.016412 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:10:45.016412 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:10:45.016412 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:10:45.016412 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:45.016412 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:45.016412 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:45.016412 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:10:45.016412 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:45.016760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:45.016760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:45.016760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:10:45.016760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:45.016760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:10:45.016760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:10:45.016760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:45.016760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:10:45.016760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:45.016760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:10:45.016760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:10:45.016760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:45.016760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:10:45.017110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:45.017110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:10:45.017957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:45.017957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:10:45.017957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:10:45.017957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:10:45.017957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:10:45.017957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:10:45.017957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:45.017957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:10:45.017957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:10:45.017957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:10:45.017957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:10:45.017957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:10:45.017957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:45.017957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:45.017957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:10:45.017957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:10:45.018418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:10:45.018418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:10:45.018418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:45.018418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:45.018418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:45.018418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:10:45.018418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:45.018418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:45.018418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:45.018418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:10:45.018418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:45.018418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:10:45.018418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:10:45.018418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:45.018418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:10:45.018418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:45.018418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:10:45.018418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:10:45.018418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:45.019106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:10:45.019106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:45.019106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:10:49.775645 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:10:49.776652 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:10:49.776652 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:10:49.777758 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:10:49.816914 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:10:49.817212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:49.818075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:10:49.818075 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:49.818075 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:10:49.818075 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:10:49.818075 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:49.818075 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:10:49.818075 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:49.818075 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:10:49.819208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:49.819208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:10:49.819208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:49.819208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:10:49.819208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:10:49.819208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:10:49.819208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:10:49.819208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:49.819208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:49.819208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:49.819208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:10:49.819208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:49.820730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:49.820730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:49.820730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:10:49.820730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:49.820730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:10:49.820730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:10:49.820730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:49.820730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:10:49.820730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:49.820730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:10:49.820730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:10:49.820730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:49.820730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:10:49.821557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:49.821557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:10:49.821557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:49.821557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:10:49.821557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:10:49.821557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:10:49.821557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:10:49.821557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:10:49.821557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:49.821557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:10:49.821557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:10:49.821557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:10:49.821557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:10:49.821557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:10:49.821557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:49.822003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:49.822003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:10:49.822003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:10:49.822003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:10:49.822003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:10:49.822003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:49.822003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:49.822003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:49.822003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:10:49.822003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:49.822003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:49.822003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:49.822003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:10:49.822003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:49.822003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:10:49.822003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:10:49.822003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:49.822003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:10:49.822003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:49.822739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:10:49.822739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:10:49.822739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:49.822739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:10:49.822739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:49.822739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:10:49.987772 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:10:49.988510 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:10:49.988728 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:10:49.989913 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:10:49.990299 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:10:49.990641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:49.991568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:10:49.991568 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:49.991568 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:10:49.991568 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:10:49.991568 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:49.991568 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:10:49.991568 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:49.991568 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:10:49.993126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:49.993126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:10:49.993126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:49.993126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:10:49.993126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:10:49.993126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:10:49.993126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:10:49.993126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:49.993126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:49.993126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:49.993126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:10:49.993126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:49.993807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:49.993807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:49.993807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:10:49.993807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:49.993807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:10:49.993807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:10:49.993807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:49.993807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:10:49.993807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:49.993807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:10:49.993807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:10:49.993807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:49.993807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:10:49.994334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:49.994334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:10:49.995178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:49.995178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:10:49.995178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:10:49.995178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:10:49.995178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:10:49.995178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:10:49.995178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:49.995178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:10:49.995178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:10:49.995178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:10:49.995178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:10:49.995178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:10:49.995178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:49.995178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:49.995178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:10:49.995178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:10:49.995891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:10:49.995891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:10:49.995891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:49.995891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:49.995891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:49.995891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:10:49.995891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:49.995891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:49.995891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:49.995891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:10:49.995891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:49.995891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:10:49.995891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:10:49.995891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:49.995891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:10:49.995891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:49.995891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:10:49.995891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:10:49.995891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:49.996768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:10:49.996768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:49.996768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:10:49.996768 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:10:49.996768 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:10:49.996768 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:10:49.996768 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:10:49.996768 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:10:49.997112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:49.997854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:10:49.997854 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:49.997854 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:10:49.997854 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:10:49.997854 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:49.997854 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:10:49.997854 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:49.997854 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:10:49.998736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:49.998736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:10:49.998736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:10:49.998736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:10:49.998736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:49.998736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:10:49.998736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:49.998736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:10:49.998736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:49.998736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:10:49.998736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:49.998736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:10:49.998736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:49.998736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:10:49.998736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:10:49.999607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:49.999607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:10:49.999607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:49.999607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:10:49.999607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:10:49.999607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:49.999607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:10:49.999607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:49.999607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:10:49.999607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:49.999607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:10:49.999607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:49.999607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:10:49.999607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:10:50.000609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:10:50.000609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:10:50.000609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:10:50.000609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:10:50.000609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:10:50.000609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:50.000609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:50.000609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:50.000609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:10:50.000609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:50.000609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:50.000609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:50.000609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:10:50.001263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:50.001263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:10:50.001263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:10:50.001263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:50.001263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:10:50.001263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:50.001263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:10:50.001263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:10:50.001263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:50.001263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:10:50.001263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:50.001263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:10:50.001871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:50.001871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:10:50.001871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:10:50.001871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:10:50.001871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:10:50.001871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:10:50.001871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:50.001871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:10:50.001871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:10:50.001871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:10:50.001871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:10:50.001871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:10:50.001871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:50.001871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:50.001871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:10:50.001871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:10:50.002781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:10:50.002781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:10:50.002781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:10:50.002781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:10:50.002781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:10:50.002781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:50.002781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:50.002781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:50.002781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:10:50.002781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:50.002781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:50.002781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:50.002781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:10:50.002781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:50.002781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:10:50.002781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:10:50.002781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:50.002781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:10:50.003479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:50.003479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:10:50.003479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:10:50.003479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:50.003479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:10:50.003479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:50.003479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:10:50.003479 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:10:50.003819 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:10:50.004070 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:10:50.004191 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:10:50.004640 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:10:50.004996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:50.005714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:10:50.005714 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:50.005714 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:10:50.005714 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:10:50.005714 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:50.005714 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:10:50.005714 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:50.005714 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:10:50.006389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:50.006389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:10:50.006389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:10:50.006389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:10:50.006389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:50.006389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:10:50.006389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:50.006389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:10:50.006389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:50.006389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:10:50.006389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:50.006389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:10:50.006389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:50.006389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:10:50.006389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:10:50.006798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:50.006798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:10:50.006798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:50.006798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:10:50.006798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:10:50.006798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:50.006798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:10:50.006798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:50.006798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:10:50.007267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:50.007267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:10:50.007267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:50.007267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:10:50.007267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:10:50.007267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:10:50.007267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:10:50.007267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:10:50.007267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:10:50.007267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:50.007267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:50.007574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:50.007574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:10:50.007574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:50.007574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:50.007574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:50.007574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:10:50.007574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:50.007574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:10:50.007574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:10:50.007574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:50.007574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:10:50.007574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:50.007574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:10:50.007574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:10:50.007813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:50.007813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:10:50.007813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:50.007813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:10:50.008301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:50.008301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:10:50.008301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:10:50.008301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:10:50.008301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:10:50.008301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:10:50.008301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:50.008301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:10:50.008301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:10:50.008301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:10:50.008301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:10:50.008301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:10:50.008301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:50.008301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:50.008301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:10:50.008301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:10:50.008301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:10:50.008635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:10:50.008635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:10:50.008635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:10:50.008635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:50.008635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:50.008635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:50.008635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:10:50.008635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:50.008635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:50.008635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:50.008635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:10:50.008635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:50.008635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:10:50.008635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:10:50.008635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:50.008635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:10:50.008635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:50.008635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:10:50.008635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:10:50.008960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:50.008960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:10:50.008960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:50.008960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:10:50.013165 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:10:50.013415 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:10:50.013547 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:10:50.013701 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:10:50.013988 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:10:50.014207 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:10:50.014349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:50.014867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:10:50.014867 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:50.014867 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:10:50.014867 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:10:50.014867 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:50.014867 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:10:50.014867 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:50.014867 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:10:50.015681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:50.015681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:10:50.015681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:50.015681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:10:50.015681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:10:50.015681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:10:50.015681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:10:50.015681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:50.015681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:50.015681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:50.015681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:10:50.015681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:50.016143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:50.016143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:50.016143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:10:50.016143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:50.016143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:10:50.016143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:10:50.016143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:50.016143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:10:50.016143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:50.016143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:10:50.016143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:10:50.016143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:50.016143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:10:50.016466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:50.016466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:10:50.016758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:50.016758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:10:50.016758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:10:50.016758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:10:50.016758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:10:50.016758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:10:50.016758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:50.016758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:10:50.016758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:10:50.016758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:10:50.016758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:10:50.016758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:10:50.016758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:50.016758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:50.016758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:10:50.016758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:10:50.017668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:10:50.017668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:10:50.017668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:50.017668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:50.017668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:50.017668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:10:50.017668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:50.017668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:50.017668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:50.017668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:10:50.017668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:50.017668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:10:50.017668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:10:50.017668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:50.017668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:10:50.017668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:50.017668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:10:50.017668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:10:50.017668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:50.018631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:10:50.018631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:50.018631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:10:54.776196 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:10:54.777008 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:10:54.777008 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:10:54.777689 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:10:54.809929 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:10:54.810356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:54.811369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:10:54.811369 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:54.811369 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:10:54.811369 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:10:54.811369 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:54.811369 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:10:54.811369 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:54.811369 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:10:54.812564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:54.812564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:10:54.812564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:54.812564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:10:54.812564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:10:54.812564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:10:54.812564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:10:54.812564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:54.812564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:54.812564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:54.812564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:10:54.812564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:54.813134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:54.813134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:54.813134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:10:54.813134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:54.813134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:10:54.813134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:10:54.813134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:54.813134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:10:54.813134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:54.813134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:10:54.813134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:10:54.813134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:54.813134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:10:54.813710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:54.813710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:10:54.814347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:54.814347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:10:54.814347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:10:54.814347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:10:54.814347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:10:54.814347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:10:54.814347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:54.814347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:10:54.814347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:10:54.814347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:10:54.814347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:10:54.814347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:10:54.814347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:54.814347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:54.814347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:10:54.814347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:10:54.815335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:10:54.815335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:10:54.815335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:54.815335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:54.815335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:54.815335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:10:54.815335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:54.815335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:54.815335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:54.815335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:10:54.815335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:54.815335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:10:54.815335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:10:54.815335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:54.815335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:10:54.815335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:54.815335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:10:54.815335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:10:54.815335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:54.816468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:10:54.816468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:54.816468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:10:54.988626 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:10:54.989068 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:10:54.989305 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:10:54.990364 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:10:54.990738 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:10:54.990920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:54.991842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:10:54.991842 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:54.991842 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:10:54.991842 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:10:54.991842 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:54.991842 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:10:54.991842 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:54.991842 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:10:54.993331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:54.993331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:10:54.993331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:54.993331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:10:54.993331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:10:54.993331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:10:54.993331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:10:54.993331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:54.993331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:54.993331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:54.993331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:10:54.993331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:54.993959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:54.993959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:54.993959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:10:54.993959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:54.993959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:10:54.993959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:10:54.993959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:54.993959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:10:54.993959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:54.993959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:10:54.993959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:10:54.993959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:54.993959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:10:54.994512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:54.994512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:10:54.995155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:54.995155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:10:54.995155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:10:54.995155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:10:54.995155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:10:54.995155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:10:54.995155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:54.995155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:10:54.995155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:10:54.995155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:10:54.995155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:10:54.995155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:10:54.995155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:54.995155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:54.995155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:10:54.995155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:10:54.995952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:10:54.995952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:10:54.995952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:54.995952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:54.995952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:54.995952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:10:54.995952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:54.995952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:54.995952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:54.995952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:10:54.995952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:54.995952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:10:54.995952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:10:54.995952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:54.995952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:10:54.995952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:54.995952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:10:54.995952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:10:54.995952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:54.996827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:10:54.996827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:54.996827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:10:54.996827 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:10:54.996827 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:10:54.996827 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:10:54.997201 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:10:54.997393 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:10:54.997690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:54.998292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:10:54.998292 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:54.998292 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:10:54.998292 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:10:54.998292 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:54.998292 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:10:54.998292 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:54.998292 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:10:54.998987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:54.998987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:10:54.998987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:10:54.998987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:10:54.998987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:54.998987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:10:54.998987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:54.998987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:10:54.998987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:54.998987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:10:54.998987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:54.998987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:10:54.998987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:54.998987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:10:54.998987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:10:54.999425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:54.999425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:10:54.999425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:54.999425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:10:54.999425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:10:54.999425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:54.999425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:10:54.999425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:54.999425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:10:54.999774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:54.999774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:10:54.999774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:54.999774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:10:54.999774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:10:54.999774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:10:54.999774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:10:54.999774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:10:54.999774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:10:54.999774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:10:54.999774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:54.999774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:55.000120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:55.000120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:10:55.000120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:55.000120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:55.000120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:55.000120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:10:55.000120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:55.000120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:10:55.000120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:10:55.000120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:55.000120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:10:55.000120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:55.000120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:10:55.000120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:10:55.000485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:55.000485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:10:55.000485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:55.000485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:10:55.000752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:55.000752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:10:55.000752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:10:55.000752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:10:55.000752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:10:55.000752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:10:55.000752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:55.000752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:10:55.000752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:10:55.000752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:10:55.000752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:10:55.000752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:10:55.000752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:55.000752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:55.000752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:10:55.000752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:10:55.001195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:10:55.001195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:10:55.001195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:10:55.001195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:10:55.001195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:10:55.001195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:55.001195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:55.001195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:55.001195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:10:55.001195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:55.001195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:55.001195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:55.001195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:10:55.001195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:55.001195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:10:55.001195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:10:55.001195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:55.001195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:10:55.001707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:55.001707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:10:55.001707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:10:55.001707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:55.001707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:10:55.001707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:55.001707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:10:55.003763 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:10:55.003990 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:10:55.004091 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:10:55.004214 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:10:55.004515 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:10:55.004725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:55.005196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:10:55.005196 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:55.005196 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:10:55.005196 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:10:55.005196 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:55.005196 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:10:55.005196 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:55.005196 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:10:55.005976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:55.005976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:10:55.005976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:10:55.005976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:10:55.005976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:55.005976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:10:55.005976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:55.005976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:10:55.005976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:55.005976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:10:55.005976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:55.005976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:10:55.005976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:55.005976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:10:55.005976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:10:55.006393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:55.006393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:10:55.006393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:55.006393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:10:55.006393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:10:55.006393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:55.006393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:10:55.006393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:55.006393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:10:55.006848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:55.006848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:10:55.006848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:55.006848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:10:55.006848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:10:55.006848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:10:55.006848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:10:55.006848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:10:55.006848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:10:55.006848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:55.006848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:55.007178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:55.007178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:10:55.007178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:55.007178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:55.007178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:55.007178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:10:55.007178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:55.007178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:10:55.007178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:10:55.007178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:55.007178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:10:55.007178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:55.007178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:10:55.007178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:10:55.007570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:55.007570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:10:55.007570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:55.007570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:10:55.008104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:55.008104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:10:55.008104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:10:55.008104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:10:55.008104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:10:55.008104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:10:55.008104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:55.008104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:10:55.008104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:10:55.008104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:10:55.008104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:10:55.008104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:10:55.008104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:55.008104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:55.008104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:10:55.008104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:10:55.008104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:10:55.008572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:10:55.008572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:10:55.008572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:10:55.008572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:55.008572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:55.008572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:55.008572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:10:55.008572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:55.008572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:55.008572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:55.008572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:10:55.008572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:55.008572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:10:55.008572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:10:55.008572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:55.008572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:10:55.008572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:55.008572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:10:55.008572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:10:55.009070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:55.009070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:10:55.009070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:55.009070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:10:55.013504 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:10:55.013751 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:10:55.013864 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:10:55.014009 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:10:55.014278 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:10:55.014512 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:10:55.014639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:55.015091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:10:55.015091 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:55.015091 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:10:55.015091 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:10:55.015091 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:55.015091 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:10:55.015091 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:55.015091 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:10:55.015871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:55.015871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:10:55.015871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:55.015871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:10:55.015871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:10:55.015871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:10:55.015871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:10:55.015871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:55.015871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:55.015871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:55.015871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:10:55.015871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:55.016203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:55.016203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:55.016203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:10:55.016203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:55.016203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:10:55.016203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:10:55.016203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:55.016203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:10:55.016203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:55.016203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:10:55.016203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:10:55.016203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:55.016203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:10:55.016560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:55.016560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:10:55.016797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:55.016797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:10:55.016797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:10:55.016797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:10:55.016797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:10:55.016797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:10:55.016797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:55.016797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:10:55.016797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:10:55.016797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:10:55.016797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:10:55.016797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:10:55.016797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:55.016797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:55.016797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:10:55.016797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:10:55.017235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:10:55.017235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:10:55.017235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:55.017235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:55.017235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:55.017235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:10:55.017235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:55.017235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:55.017235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:55.017235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:10:55.017235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:55.017235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:10:55.017235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:10:55.017235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:55.017235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:10:55.017235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:55.017235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:10:55.017235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:10:55.017235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:55.017728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:10:55.017728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:55.017728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:10:59.776862 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:10:59.780363 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:10:59.780363 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:10:59.780363 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:10:59.813008 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:10:59.813291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:59.816612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:10:59.816612 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:59.816612 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:10:59.816612 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:10:59.816612 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:59.816612 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:10:59.816612 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:59.816612 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:10:59.816612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:59.816612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:10:59.816612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:59.816612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:10:59.816612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:10:59.816612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:10:59.816612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:10:59.817128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:59.817128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:59.817128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:59.817128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:10:59.817128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:59.817128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:59.817128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:59.817128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:10:59.817128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:59.817128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:10:59.817128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:10:59.817128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:59.817128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:10:59.817128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:59.817561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:10:59.817561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:10:59.817561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:59.817561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:10:59.817561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:59.817561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:10:59.817561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:59.817561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:10:59.817561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:10:59.817561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:10:59.817561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:10:59.817561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:10:59.817561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:59.817561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:10:59.817980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:10:59.817980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:10:59.817980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:10:59.817980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:10:59.817980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:59.817980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:59.817980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:10:59.817980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:10:59.817980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:10:59.817980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:10:59.817980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:59.817980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:59.817980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:59.817980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:10:59.817980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:59.817980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:59.817980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:59.817980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:10:59.817980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:59.817980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:10:59.817980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:10:59.818559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:59.818559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:10:59.818559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:59.818559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:10:59.818559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:10:59.818559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:59.818559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:10:59.818559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:59.818559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:10:59.989282 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:10:59.990873 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:10:59.990873 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:10:59.991266 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:10:59.991533 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:10:59.991728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:59.992460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:10:59.992460 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:59.992460 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:10:59.992460 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:10:59.992460 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:59.992460 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:10:59.992460 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:59.992460 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:10:59.995692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:59.995692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:10:59.995692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:59.995692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:10:59.995692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:10:59.995692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:10:59.995692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:10:59.995692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:59.995692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:59.995692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:59.995692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:10:59.995692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:59.996274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:10:59.996274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:59.996274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:10:59.996274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:59.996274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:10:59.996274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:10:59.996274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:59.996274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:10:59.996274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:59.996274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:10:59.996274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:10:59.996274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:59.996274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:10:59.996866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:59.996866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:10:59.997790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:59.997790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:10:59.997790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:10:59.997790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:10:59.997790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:10:59.997790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:10:59.997790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:59.997790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:10:59.997790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:10:59.997790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:10:59.997790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:10:59.997790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:10:59.997790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:10:59.997790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:10:59.997790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:10:59.997790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:10:59.998595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:10:59.998595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:10:59.998595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:59.998595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:59.998595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:10:59.998595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:10:59.998595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:10:59.998595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:10:59.998595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:10:59.998595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:10:59.998595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:10:59.998595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:10:59.998595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:10:59.998595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:10:59.998595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:10:59.998595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:10:59.998595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:10:59.998595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:10:59.998595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:10:59.999504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:10:59.999504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:10:59.999504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:10:59.999974 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:11:00.000552 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:11:00.000552 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:11:00.000552 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:11:00.000867 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:11:00.001060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:00.003376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:11:00.003376 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:00.003376 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:11:00.003376 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:11:00.003376 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:00.003376 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:11:00.003376 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:00.003376 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:11:00.003376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:00.003376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:11:00.003376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:11:00.003376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:11:00.003376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:00.003376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:11:00.003376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:00.003376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:11:00.003376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:00.003376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:11:00.003824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:00.003824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:11:00.003824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:00.003824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:11:00.003824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:11:00.003824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:00.003824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:11:00.003824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:00.003824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:11:00.003824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:11:00.003824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:00.003824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:11:00.003824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:00.003824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:11:00.003824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:00.004129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:11:00.004129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:00.004129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:11:00.004129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:11:00.004129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:11:00.004129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:11:00.004129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:11:00.004129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:11:00.004129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:11:00.004129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:00.004129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:00.004129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:00.004129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:11:00.004419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:00.004419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:00.004419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:00.004419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:11:00.004419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:00.004419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:11:00.004419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:11:00.004419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:00.004419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:11:00.004419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:00.004419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:11:00.004419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:11:00.004419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:00.004419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:11:00.004685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:00.004685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:11:00.004685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:00.004685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:11:00.004685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:11:00.004685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:11:00.004685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:11:00.004685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:11:00.004685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:00.004685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:11:00.004685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:11:00.004685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:11:00.004685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:11:00.004685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:11:00.004685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:00.004980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:00.004980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:11:00.004980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:11:00.004980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:11:00.004980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:11:00.004980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:11:00.004980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:11:00.004980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:11:00.004980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:00.004980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:00.004980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:00.004980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:11:00.004980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:00.004980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:00.004980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:00.004980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:11:00.004980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:00.004980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:11:00.004980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:11:00.005346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:00.005346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:11:00.005346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:00.005346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:11:00.005346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:11:00.005346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:00.005346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:11:00.005346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:00.005346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:11:00.005346 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:11:00.006557 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:11:00.006917 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:11:00.007084 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:11:00.007678 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:11:00.007906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:00.010368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:11:00.010368 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:00.010368 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:11:00.010368 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:11:00.010368 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:00.010368 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:11:00.010368 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:00.010368 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:11:00.010368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:00.010368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:11:00.010368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:11:00.010368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:11:00.010368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:00.010368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:11:00.010368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:00.010368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:11:00.010368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:00.010368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:11:00.010795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:00.010795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:11:00.010795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:00.010795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:11:00.010795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:11:00.010795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:00.010795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:11:00.010795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:00.010795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:11:00.010795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:11:00.010795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:00.010795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:11:00.010795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:00.010795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:11:00.010795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:00.011129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:11:00.011129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:00.011129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:11:00.011129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:11:00.011129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:11:00.011129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:11:00.011129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:11:00.011129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:11:00.011129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:00.011129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:00.011129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:00.011129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:11:00.011129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:00.011420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:00.011420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:00.011420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:11:00.011420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:00.011420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:11:00.011420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:11:00.011420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:00.011420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:11:00.011420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:00.011420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:11:00.011420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:11:00.011420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:00.011420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:11:00.011698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:00.011698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:11:00.011698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:00.011698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:11:00.011698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:11:00.011698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:11:00.011698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:11:00.011698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:11:00.011698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:00.011698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:11:00.011698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:11:00.011698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:11:00.011698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:11:00.011698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:11:00.011698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:00.012011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:00.012011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:11:00.012011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:11:00.012011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:11:00.012011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:11:00.012011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:11:00.012011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:11:00.012011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:00.012011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:00.012011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:00.012011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:11:00.012011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:00.012011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:00.012011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:00.012011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:11:00.012011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:00.012011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:11:00.012011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:11:00.012011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:00.012385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:11:00.012385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:00.012385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:11:00.012385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:11:00.012385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:00.012385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:11:00.012385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:00.012385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:11:00.013888 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:11:00.014324 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:11:00.014324 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:11:00.014324 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:11:00.014761 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:11:00.015316 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:11:00.015617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:00.017595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:11:00.017595 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:00.017595 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:11:00.017595 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:11:00.017595 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:00.017595 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:11:00.017595 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:00.017595 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:11:00.017595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:00.017595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:11:00.017595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:00.017595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:11:00.017595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:11:00.017595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:11:00.017595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:11:00.018188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:00.018188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:00.018188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:00.018188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:11:00.018188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:00.018188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:00.018188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:00.018188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:11:00.018188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:00.018188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:11:00.018188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:11:00.018188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:00.018188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:11:00.018188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:00.018558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:11:00.018558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:11:00.018558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:00.018558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:11:00.018558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:00.018558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:11:00.018558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:00.018558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:11:00.018558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:11:00.018558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:11:00.018558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:11:00.018558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:11:00.018558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:00.018558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:11:00.018919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:11:00.018919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:11:00.018919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:11:00.018919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:11:00.018919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:00.018919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:00.018919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:11:00.018919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:11:00.018919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:11:00.018919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:11:00.018919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:00.018919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:00.018919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:00.018919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:11:00.018919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:00.018919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:00.018919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:00.018919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:11:00.018919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:00.018919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:11:00.019310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:11:00.019310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:00.019310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:11:00.019310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:00.019310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:11:00.019310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:11:00.019310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:00.019310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:11:00.019310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:00.019310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:11:04.778239 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:11:04.779186 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:11:04.779186 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:11:04.780345 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:11:04.811195 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:11:04.811643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:04.812502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:11:04.812502 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:04.812502 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:11:04.812502 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:11:04.812502 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:04.812502 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:11:04.812502 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:04.812502 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:11:04.813257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:04.813257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:11:04.813257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:04.813257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:11:04.813257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:11:04.813257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:11:04.813257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:11:04.813257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:04.813257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:04.813257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:04.813257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:11:04.813257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:04.813959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:04.813959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:04.813959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:11:04.813959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:04.813959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:11:04.813959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:11:04.813959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:04.813959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:11:04.813959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:04.813959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:11:04.813959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:11:04.813959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:04.813959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:11:04.814679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:04.814679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:11:04.814679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:04.814679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:11:04.814679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:11:04.814679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:11:04.814679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:11:04.814679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:11:04.814679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:04.814679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:11:04.814679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:11:04.814679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:11:04.814679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:11:04.814679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:11:04.814679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:04.815502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:04.815502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:11:04.815502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:11:04.815502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:11:04.815502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:11:04.815502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:04.815502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:04.815502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:04.815502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:11:04.815502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:04.815502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:04.815502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:04.815502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:11:04.815502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:04.815502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:11:04.815502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:11:04.815502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:04.815502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:11:04.815502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:04.816455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:11:04.816455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:11:04.816455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:04.816455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:11:04.816455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:04.816455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:11:04.990394 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:11:04.991371 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:11:04.991649 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:11:04.992749 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:11:04.993016 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:11:04.993239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:04.994197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:11:04.994197 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:04.994197 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:11:04.994197 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:11:04.994197 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:04.994197 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:11:04.994197 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:04.994197 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:11:04.997553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:04.997553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:11:04.997553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:04.997553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:11:04.997553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:11:04.997553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:11:04.997553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:11:04.997553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:04.997553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:04.997553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:04.997553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:11:04.997553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:04.998318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:04.998318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:04.998318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:11:04.998318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:04.998318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:11:04.998318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:11:04.998318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:04.998318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:11:04.998318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:04.998318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:11:04.998318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:11:04.998318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:04.998318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:11:04.998926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:04.998926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:11:04.999325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:04.999325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:11:04.999325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:11:04.999325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:11:04.999325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:11:04.999325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:11:04.999325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:04.999325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:11:04.999325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:11:04.999325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:11:04.999325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:11:04.999325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:11:04.999325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:04.999325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:04.999325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:11:04.999325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:11:05.000075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:11:05.000075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:11:05.000075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:05.000075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:05.000075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:05.000075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:11:05.000075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:05.000075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:05.000075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:05.000075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:11:05.000075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:05.000075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:11:05.000075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:11:05.000075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:05.000075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:11:05.000075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:05.000075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:11:05.000075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:11:05.000075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:05.000928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:11:05.000928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:05.000928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:11:05.000928 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:11:05.000928 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:11:05.000928 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:11:05.000928 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:11:05.001319 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:11:05.001671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:05.002399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:11:05.002399 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:05.002399 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:11:05.002399 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:11:05.002399 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:05.002399 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:11:05.002399 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:05.002399 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:11:05.003143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:05.003143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:11:05.003143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:11:05.003143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:11:05.003143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:05.003143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:11:05.003143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:05.003143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:11:05.003143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:05.003143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:11:05.003143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:05.003143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:11:05.003143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:05.003143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:11:05.003143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:11:05.003883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:05.003883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:11:05.003883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:05.003883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:11:05.003883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:11:05.003883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:05.003883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:11:05.003883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:05.003883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:11:05.004295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:05.004295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:11:05.004295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:05.004295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:11:05.004295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:11:05.004295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:11:05.004295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:11:05.004295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:11:05.004295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:11:05.004295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:11:05.004295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:05.004295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:05.004892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:05.004892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:11:05.004892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:05.004892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:05.004892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:05.004892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:11:05.004892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:05.004892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:11:05.004892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:11:05.004892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:05.004892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:11:05.004892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:05.004892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:11:05.004892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:11:05.005499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:05.005499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:11:05.005499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:05.005499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:11:05.005499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:05.005499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:11:05.005499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:11:05.005499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:11:05.005499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:11:05.005499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:11:05.005499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:05.005499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:11:05.005499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:11:05.005499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:11:05.006123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:11:05.006123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:11:05.006123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:05.006123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:05.006123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:11:05.006123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:11:05.006123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:11:05.006123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:11:05.006123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:11:05.006123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:11:05.006123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:11:05.006123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:05.006123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:05.006123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:05.006123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:11:05.006123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:05.006123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:05.006123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:05.006123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:11:05.007034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:05.007034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:11:05.007034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:11:05.007034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:05.007034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:11:05.007034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:05.007034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:11:05.007034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:11:05.007034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:05.007034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:11:05.007034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:05.007034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:11:05.007034 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:11:05.007034 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:11:05.007478 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:11:05.007478 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:11:05.007478 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:11:05.007581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:05.007942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:11:05.007942 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:05.007942 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:11:05.007942 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:11:05.007942 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:05.007942 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:11:05.007942 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:05.007942 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:11:05.008462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:05.008462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:11:05.008462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:11:05.008462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:11:05.008462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:05.008462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:11:05.008462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:05.008462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:11:05.008462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:05.008462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:11:05.008462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:05.008462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:11:05.008462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:05.008462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:11:05.008462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:11:05.008854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:05.008854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:11:05.008854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:05.008854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:11:05.008854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:11:05.008854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:05.008854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:11:05.008854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:05.008854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:11:05.009094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:05.009094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:11:05.009094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:05.009094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:11:05.009094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:11:05.009094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:11:05.009094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:11:05.009094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:11:05.009094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:11:05.009094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:05.009094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:05.009427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:05.009427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:11:05.009427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:05.009427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:05.009427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:05.009427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:11:05.009427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:05.009427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:11:05.009427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:11:05.009427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:05.009427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:11:05.009427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:05.009427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:11:05.009427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:11:05.009802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:05.009802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:11:05.009802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:05.009802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:11:05.009802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:05.009802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:11:05.009802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:11:05.009802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:11:05.009802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:11:05.009802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:11:05.009802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:05.009802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:11:05.009802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:11:05.009802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:11:05.010187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:11:05.010187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:11:05.010187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:05.010187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:05.010187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:11:05.010187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:11:05.010187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:11:05.010187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:11:05.010187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:11:05.010187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:11:05.010187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:05.010187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:05.010187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:05.010187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:11:05.010187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:05.010187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:05.010187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:05.010187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:11:05.010187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:05.010745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:11:05.010745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:11:05.010745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:05.010745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:11:05.010745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:05.010745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:11:05.010745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:11:05.010745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:05.010745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:11:05.010745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:05.010745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:11:05.014196 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:11:05.014370 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:11:05.014492 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:11:05.014597 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:11:05.014874 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:11:05.015093 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:11:05.015242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:05.015678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:11:05.015678 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:05.015678 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:11:05.015678 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:11:05.015678 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:05.015678 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:11:05.015678 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:05.015678 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:11:05.016211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:05.016211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:11:05.016211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:05.016211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:11:05.016211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:11:05.016211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:11:05.016211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:11:05.016211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:05.016211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:05.016211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:05.016211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:11:05.016211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:05.016562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:05.016562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:05.016562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:11:05.016562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:05.016562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:11:05.016562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:11:05.016562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:05.016562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:11:05.016562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:05.016562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:11:05.016562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:11:05.016562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:05.016562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:11:05.016925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:05.016925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:11:05.016925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:05.016925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:11:05.016925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:11:05.016925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:11:05.016925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:11:05.016925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:11:05.016925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:05.016925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:11:05.016925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:11:05.016925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:11:05.016925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:11:05.016925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:11:05.016925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:05.017319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:05.017319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:11:05.017319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:11:05.017319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:11:05.017319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:11:05.017319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:05.017319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:05.017319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:05.017319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:11:05.017319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:05.017319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:05.017319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:05.017319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:11:05.017319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:05.017319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:11:05.017319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:11:05.017319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:05.017319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:11:05.017319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:05.017841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:11:05.017841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:11:05.017841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:05.017841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:11:05.017841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:05.017841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:11:09.778879 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:11:09.779540 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:11:09.779735 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:11:09.781003 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:11:09.814197 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:11:09.814525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:09.815323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:11:09.815323 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:09.815323 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:11:09.815323 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:11:09.815323 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:09.815323 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:11:09.815323 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:09.815323 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:11:09.816062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:09.816062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:11:09.816062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:09.816062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:11:09.816062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:11:09.816062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:11:09.816062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:11:09.816062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:09.816062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:09.816062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:09.816062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:11:09.816062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:09.816904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:09.816904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:09.816904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:11:09.816904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:09.816904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:11:09.816904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:11:09.816904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:09.816904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:11:09.816904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:09.816904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:11:09.816904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:11:09.816904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:09.816904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:11:09.817471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:09.817471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:11:09.817471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:09.817471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:11:09.817471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:11:09.817471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:11:09.817471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:11:09.817471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:11:09.817471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:09.817471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:11:09.817471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:11:09.817471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:11:09.817471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:11:09.817471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:11:09.817471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:09.817759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:09.817759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:11:09.817759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:11:09.817759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:11:09.817759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:11:09.817759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:09.817759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:09.817759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:09.817759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:11:09.817759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:09.817759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:09.817759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:09.817759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:11:09.817759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:09.817759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:11:09.817759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:11:09.817759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:09.817759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:11:09.817759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:09.818115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:11:09.818115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:11:09.818115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:09.818115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:11:09.818115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:09.818115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:11:09.991345 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:11:09.991954 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:11:09.992148 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:11:09.993385 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:11:09.993768 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:11:09.994037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:09.994981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:11:09.994981 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:09.994981 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:11:09.994981 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:11:09.994981 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:09.994981 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:11:09.994981 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:09.994981 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:11:09.996577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:09.996577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:11:09.996577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:09.996577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:11:09.996577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:11:09.996577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:11:09.996577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:11:09.996577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:09.996577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:09.996577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:09.996577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:11:09.996577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:09.997126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:09.997126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:09.997126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:11:09.997126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:09.997126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:11:09.997126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:11:09.997126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:09.997126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:11:09.997126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:09.997126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:11:09.997126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:11:09.997126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:09.997126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:11:09.997748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:09.997748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:11:09.998263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:09.998263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:11:09.998263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:11:09.998263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:11:09.998263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:11:09.998263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:11:09.998263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:09.998263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:11:09.998263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:11:09.998263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:11:09.998263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:11:09.998263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:11:09.998263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:09.998263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:09.998263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:11:09.998263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:11:09.998945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:11:09.998945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:11:09.998945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:09.998945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:09.998945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:09.998945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:11:09.998945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:09.998945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:09.998945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:09.998945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:11:09.998945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:09.998945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:11:09.998945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:11:09.998945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:09.998945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:11:09.998945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:09.998945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:11:09.998945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:11:09.998945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:09.999768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:11:09.999768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:09.999768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:11:10.000890 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:11:10.001296 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:11:10.001440 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:11:10.001572 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:11:10.002413 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:11:10.002774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:10.003552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:11:10.003552 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:10.003552 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:11:10.003552 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:11:10.003552 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:10.003552 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:11:10.003552 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:10.003552 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:11:10.004301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:10.004301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:11:10.004301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:11:10.004301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:11:10.004301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:10.004301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:11:10.004301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:10.004301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:11:10.004301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:10.004301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:11:10.004301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:10.004301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:11:10.004301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:10.004301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:11:10.004301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:11:10.004713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:10.004713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:11:10.004713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:10.004713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:11:10.004713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:11:10.004713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:10.004713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:11:10.004713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:10.004713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:11:10.005145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:10.005145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:11:10.005145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:10.005145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:11:10.005145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:11:10.005145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:11:10.005145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:11:10.005145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:11:10.005145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:11:10.005145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:11:10.005145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:10.005145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:10.005481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:10.005481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:11:10.005481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:10.005481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:10.005481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:10.005481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:11:10.005481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:10.005481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:11:10.005481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:11:10.005481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:10.005481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:11:10.005481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:10.005481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:11:10.005481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:11:10.005845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:10.005845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:11:10.005845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:10.005845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:11:10.006150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:10.006150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:11:10.006150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:11:10.006150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:11:10.006150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:11:10.006150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:11:10.006150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:10.006150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:11:10.006150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:11:10.006150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:11:10.006150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:11:10.006150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:11:10.006150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:10.006150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:10.006150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:11:10.006150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:11:10.006580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:11:10.006580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:11:10.006580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:11:10.006580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:11:10.006580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:11:10.006580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:10.006580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:10.006580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:10.006580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:11:10.006580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:10.006580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:10.006580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:10.006580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:11:10.006580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:10.006580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:11:10.006580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:11:10.006580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:10.006580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:11:10.007043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:10.007043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:11:10.007043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:11:10.007043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:10.007043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:11:10.007043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:10.007043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:11:10.007043 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:11:10.007298 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:11:10.007418 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:11:10.007557 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:11:10.008042 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:11:10.008489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:10.009235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:11:10.009235 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:10.009235 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:11:10.009235 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:11:10.009235 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:10.009235 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:11:10.009235 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:10.009235 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:11:10.009970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:10.009970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:11:10.009970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:11:10.009970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:11:10.009970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:10.009970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:11:10.009970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:10.009970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:11:10.009970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:10.009970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:11:10.009970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:10.009970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:11:10.009970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:10.009970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:11:10.009970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:11:10.010364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:10.010364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:11:10.010364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:10.010364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:11:10.010364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:11:10.010364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:10.010364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:11:10.010364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:10.010364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:11:10.010818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:10.010818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:11:10.010818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:10.010818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:11:10.010818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:11:10.010818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:11:10.010818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:11:10.010818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:11:10.010818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:11:10.010818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:10.010818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:10.011126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:10.011126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:11:10.011126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:10.011126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:10.011126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:10.011126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:11:10.011126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:10.011126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:11:10.011126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:11:10.011126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:10.011126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:11:10.011126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:10.011126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:11:10.011126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:11:10.011508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:10.011508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:11:10.011508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:10.011508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:11:10.011963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:10.011963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:11:10.011963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:11:10.011963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:11:10.011963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:11:10.011963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:11:10.011963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:10.011963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:11:10.011963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:11:10.011963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:11:10.011963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:11:10.011963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:11:10.011963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:10.011963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:10.011963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:11:10.011963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:11:10.011963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:11:10.012390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:11:10.012390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:11:10.012390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:11:10.012390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:10.012390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:10.012390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:10.012390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:11:10.012390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:10.012390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:10.012390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:10.012390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:11:10.012390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:10.012390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:11:10.012390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:11:10.012390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:10.012390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:11:10.012390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:10.012390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:11:10.012390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:11:10.012909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:10.012909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:11:10.012909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:10.012909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:11:10.014505 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:11:10.014764 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:11:10.014893 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:11:10.015054 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:11:10.015531 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:11:10.015922 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:11:10.016174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:10.016975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:11:10.016975 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:10.016975 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:11:10.016975 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:11:10.016975 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:10.016975 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:11:10.016975 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:10.016975 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:11:10.017773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:10.017773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:11:10.017773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:10.017773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:11:10.017773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:11:10.017773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:11:10.017773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:11:10.017773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:10.017773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:10.017773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:10.017773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:11:10.017773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:10.018120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:10.018120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:10.018120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:11:10.018120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:10.018120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:11:10.018120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:11:10.018120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:10.018120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:11:10.018120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:10.018120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:11:10.018120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:11:10.018120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:10.018120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:11:10.018485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:10.018485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:11:10.018717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:10.018717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:11:10.018717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:11:10.018717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:11:10.018717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:11:10.018717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:11:10.018717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:10.018717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:11:10.018717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:11:10.018717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:11:10.018717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:11:10.018717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:11:10.018717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:10.018717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:10.018717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:11:10.018717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:11:10.019143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:11:10.019143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:11:10.019143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:10.019143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:10.019143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:10.019143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:11:10.019143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:10.019143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:10.019143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:10.019143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:11:10.019143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:10.019143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:11:10.019143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:11:10.019143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:10.019143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:11:10.019143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:10.019143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:11:10.019143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:11:10.019143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:10.019669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:11:10.019669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:10.019669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:11:14.779644 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:11:14.780384 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:11:14.780551 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:11:14.781798 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:11:14.816815 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:11:14.817438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:14.818593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:11:14.818593 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:14.818593 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:11:14.818593 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:11:14.818593 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:14.818593 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:11:14.818593 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:14.818593 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:11:14.820419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:14.820419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:11:14.820419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:14.820419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:11:14.820419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:11:14.820419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:11:14.820419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:11:14.820419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:14.820419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:14.820419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:14.820419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:11:14.820419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:14.820800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:14.820800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:14.820800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:11:14.820800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:14.820800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:11:14.820800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:11:14.820800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:14.820800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:11:14.820800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:14.820800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:11:14.820800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:11:14.820800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:14.820800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:11:14.821158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:14.821158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:11:14.822879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:14.822879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:11:14.822879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:11:14.822879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:11:14.822879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:11:14.822879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:11:14.822879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:14.822879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:11:14.822879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:11:14.822879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:11:14.822879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:11:14.822879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:11:14.822879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:14.822879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:14.822879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:11:14.822879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:11:14.823386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:11:14.823386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:11:14.823386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:14.823386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:14.823386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:14.823386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:11:14.823386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:14.823386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:14.823386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:14.823386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:11:14.823386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:14.823386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:11:14.823386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:11:14.823386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:14.823386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:11:14.823386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:14.823386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:11:14.823386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:11:14.823386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:14.823929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:11:14.823929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:14.823929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:11:14.992120 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:11:14.992840 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:11:14.993024 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:11:14.995312 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:11:14.995932 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:11:14.996319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:14.997650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:11:14.997650 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:14.997650 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:11:14.997650 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:11:14.997650 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:14.997650 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:11:14.997650 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:14.997650 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:11:14.999065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:14.999065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:11:14.999065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:14.999065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:11:14.999065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:11:14.999065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:11:14.999065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:11:14.999065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:14.999065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:14.999065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:14.999065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:11:14.999065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:14.999704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:14.999704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:14.999704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:11:14.999704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:14.999704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:11:14.999704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:11:14.999704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:14.999704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:11:14.999704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:14.999704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:11:14.999704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:11:14.999704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:14.999704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:11:15.000254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:15.000254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:11:15.000807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:15.000807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:11:15.000807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:11:15.000807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:11:15.000807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:11:15.000807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:11:15.000807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:15.000807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:11:15.000807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:11:15.000807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:11:15.000807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:11:15.000807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:11:15.000807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:15.000807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:15.000807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:11:15.000807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:11:15.001574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:11:15.001574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:11:15.001574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:15.001574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:15.001574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:15.001574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:11:15.001574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:15.001574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:15.001574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:15.001574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:11:15.001574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:15.001574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:11:15.001574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:11:15.001574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:15.001574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:11:15.001574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:15.001574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:11:15.001574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:11:15.001574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:15.002370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:11:15.002370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:15.002370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:11:15.002370 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:11:15.002370 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:11:15.002370 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:11:15.002370 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:11:15.002962 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:11:15.003240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:15.003949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:11:15.003949 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:15.003949 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:11:15.003949 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:11:15.003949 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:15.003949 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:11:15.003949 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:15.003949 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:11:15.004745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:15.004745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:11:15.004745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:11:15.004745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:11:15.004745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:15.004745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:11:15.004745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:15.004745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:11:15.004745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:15.004745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:11:15.004745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:15.004745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:11:15.004745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:15.004745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:11:15.004745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:11:15.005484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:15.005484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:11:15.005484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:15.005484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:11:15.005484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:11:15.005484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:15.005484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:11:15.005484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:15.005484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:11:15.005484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:15.005484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:11:15.005484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:15.005484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:11:15.005484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:11:15.005878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:11:15.005878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:11:15.005878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:11:15.005878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:11:15.005878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:11:15.005878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:15.005878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:15.005878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:15.005878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:11:15.005878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:15.005878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:15.005878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:15.005878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:11:15.006243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:15.006243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:11:15.006243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:11:15.006243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:15.006243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:11:15.006243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:15.006243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:11:15.006243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:11:15.006243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:15.006243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:11:15.006243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:15.006243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:11:15.006567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:15.006567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:11:15.006567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:11:15.006567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:11:15.006567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:11:15.006567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:11:15.006567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:15.006567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:11:15.006567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:11:15.006567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:11:15.006567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:11:15.006567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:11:15.006567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:15.006567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:15.006567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:11:15.006567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:11:15.006995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:11:15.006995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:11:15.006995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:11:15.006995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:11:15.006995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:11:15.006995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:15.006995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:15.006995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:15.006995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:11:15.006995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:15.006995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:15.006995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:15.006995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:11:15.006995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:15.006995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:11:15.006995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:11:15.006995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:15.006995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:11:15.009438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:15.009438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:11:15.009438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:11:15.009438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:15.009438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:11:15.009438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:15.009438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:11:15.009438 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:11:15.009438 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:11:15.009438 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:11:15.009740 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:11:15.009740 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:11:15.009740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:15.010337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:11:15.010337 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:15.010337 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:11:15.010337 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:11:15.010337 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:15.010337 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:11:15.010337 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:15.010337 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:11:15.011944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:15.011944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:11:15.011944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:11:15.011944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:11:15.011944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:15.011944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:11:15.011944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:15.011944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:11:15.011944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:15.011944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:11:15.011944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:15.011944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:11:15.011944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:15.011944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:11:15.011944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:11:15.012487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:15.012487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:11:15.012487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:15.012487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:11:15.012487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:11:15.012487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:15.012487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:11:15.012487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:15.012487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:11:15.012801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:15.012801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:11:15.012801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:15.012801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:11:15.012801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:11:15.012801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:11:15.012801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:11:15.012801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:11:15.012801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:11:15.012801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:15.012801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:15.013139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:15.013139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:11:15.013139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:15.013139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:15.013139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:15.013139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:11:15.013139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:15.013139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:11:15.013139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:11:15.013139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:15.013139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:11:15.013139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:15.013139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:11:15.013139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:11:15.013541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:15.013541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:11:15.013541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:15.013541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:11:15.013841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:15.013841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:11:15.013841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:11:15.013841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:11:15.013841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:11:15.013841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:11:15.013841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:15.013841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:11:15.013841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:11:15.013841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:11:15.013841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:11:15.013841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:11:15.013841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:15.013841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:15.013841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:11:15.013841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:11:15.013841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:11:15.014299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:11:15.014299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:11:15.014299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:11:15.014299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:15.014299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:15.014299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:15.014299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:11:15.014299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:15.014299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:15.014299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:15.014299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:11:15.014299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:15.014299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:11:15.014299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:11:15.014299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:15.014299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:11:15.014299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:15.014299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:11:15.014299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:11:15.014847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:15.014847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:11:15.014847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:15.014847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:11:15.014963 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:11:15.015222 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:11:15.015318 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:11:15.015519 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:11:15.016309 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:11:15.016504 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:11:15.016621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:15.017072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:11:15.017072 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:15.017072 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:11:15.017072 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:11:15.017072 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:15.017072 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:11:15.017072 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:15.017072 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:11:15.017968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:15.017968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:11:15.017968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:15.017968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:11:15.017968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:11:15.017968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:11:15.017968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:11:15.017968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:15.017968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:15.017968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:15.017968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:11:15.017968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:15.018284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:15.018284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:15.018284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:11:15.018284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:15.018284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:11:15.018284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:11:15.018284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:15.018284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:11:15.018284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:15.018284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:11:15.018284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:11:15.018284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:15.018284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:11:15.018617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:15.018617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:11:15.018919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:15.018919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:11:15.018919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:11:15.018919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:11:15.018919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:11:15.018919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:11:15.018919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:15.018919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:11:15.018919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:11:15.018919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:11:15.018919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:11:15.018919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:11:15.018919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:15.018919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:15.018919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:11:15.018919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:11:15.019315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:11:15.019315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:11:15.019315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:15.019315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:15.019315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:15.019315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:11:15.019315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:15.019315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:15.019315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:15.019315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:11:15.019315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:15.019315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:11:15.019315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:11:15.019315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:15.019315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:11:15.019315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:15.019315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:11:15.019315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:11:15.019315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:15.020515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:11:15.020515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:15.020515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:11:19.780352 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:11:19.781352 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:11:19.781494 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:11:19.782810 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:11:19.821024 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:11:19.821491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:19.822580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:11:19.822580 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:19.822580 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:11:19.822580 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:11:19.822580 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:19.822580 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:11:19.822580 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:19.822580 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:11:19.823772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:19.823772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:11:19.823772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:19.823772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:11:19.823772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:11:19.823772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:11:19.823772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:11:19.823772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:19.823772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:19.823772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:19.823772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:11:19.823772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:19.824185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:19.824185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:19.824185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:11:19.824185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:19.824185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:11:19.824185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:11:19.824185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:19.824185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:11:19.824185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:19.824185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:11:19.824185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:11:19.824185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:19.824185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:11:19.824659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:19.824659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:11:19.824831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:19.824831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:11:19.824831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:11:19.824831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:11:19.824831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:11:19.824831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:11:19.824831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:19.824831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:11:19.824831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:11:19.824831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:11:19.824831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:11:19.824831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:11:19.824831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:19.824831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:19.824831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:11:19.824831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:11:19.825269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:11:19.825269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:11:19.825269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:19.825269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:19.825269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:19.825269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:11:19.825269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:19.825269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:19.825269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:19.825269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:11:19.825269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:19.825269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:11:19.825269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:11:19.825269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:19.825269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:11:19.825269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:19.825269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:11:19.825269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:11:19.825269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:19.825910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:11:19.825910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:19.825910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:11:19.993105 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:11:19.993874 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:11:19.994079 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:11:19.995434 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:11:19.995907 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:11:19.996178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:19.997202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:11:19.997202 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:19.997202 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:11:19.997202 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:11:19.997202 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:19.997202 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:11:19.997202 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:19.997202 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:11:19.998830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:19.998830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:11:19.998830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:19.998830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:11:19.998830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:11:19.998830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:11:19.998830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:11:19.998830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:19.998830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:19.998830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:19.998830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:11:19.998830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:19.999428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:19.999428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:19.999428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:11:19.999428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:19.999428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:11:19.999428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:11:19.999428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:19.999428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:11:19.999428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:19.999428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:11:19.999428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:11:19.999428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:19.999428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:11:20.000048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:20.000048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:11:20.000450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:20.000450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:11:20.000450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:11:20.000450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:11:20.000450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:11:20.000450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:11:20.000450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:20.000450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:11:20.000450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:11:20.000450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:11:20.000450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:11:20.000450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:11:20.000450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:20.000450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:20.000450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:11:20.000450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:11:20.001258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:11:20.001258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:11:20.001258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:20.001258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:20.001258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:20.001258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:11:20.001258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:20.001258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:20.001258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:20.001258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:11:20.001258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:20.001258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:11:20.001258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:11:20.001258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:20.001258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:11:20.001258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:20.001258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:11:20.001258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:11:20.001258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:20.002229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:11:20.002229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:20.002229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:11:20.002229 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:11:20.002579 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:11:20.002667 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:11:20.002785 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:11:20.003368 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:11:20.003772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:20.004595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:11:20.004595 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:20.004595 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:11:20.004595 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:11:20.004595 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:20.004595 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:11:20.004595 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:20.004595 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:11:20.005350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:20.005350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:11:20.005350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:11:20.005350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:11:20.005350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:20.005350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:11:20.005350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:20.005350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:11:20.005350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:20.005350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:11:20.005350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:20.005350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:11:20.005350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:20.005350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:11:20.005350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:11:20.006113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:20.006113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:11:20.006113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:20.006113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:11:20.006113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:11:20.006113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:20.006113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:11:20.006113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:20.006113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:11:20.006502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:20.006502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:11:20.006502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:20.006502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:11:20.006502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:11:20.006502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:11:20.006502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:11:20.006502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:11:20.006502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:11:20.006502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:11:20.006502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:20.006502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:20.006902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:20.006902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:11:20.006902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:20.006902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:20.006902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:20.006902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:11:20.006902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:20.006902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:11:20.006902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:11:20.006902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:20.006902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:11:20.006902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:20.006902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:11:20.006902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:11:20.007283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:20.007283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:11:20.007283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:20.007283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:11:20.007283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:20.007283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:11:20.007283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:11:20.007283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:11:20.007283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:11:20.007283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:11:20.007283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:20.007283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:11:20.007283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:11:20.007283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:11:20.007709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:11:20.007709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:11:20.007709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:20.007709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:20.007709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:11:20.007709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:11:20.007709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:11:20.007709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:11:20.007709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:11:20.007709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:11:20.007709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:11:20.007709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:20.007709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:20.007709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:20.007709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:11:20.007709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:20.007709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:20.007709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:20.007709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:11:20.008336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:20.008336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:11:20.008336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:11:20.008336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:20.008336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:11:20.008336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:20.008336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:11:20.008336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:11:20.008336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:20.008336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:11:20.008336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:20.008336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:11:20.008336 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:11:20.008738 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:11:20.008738 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:11:20.008825 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:11:20.009397 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:11:20.009801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:20.010710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:11:20.010710 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:20.010710 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:11:20.010710 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:11:20.010710 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:20.010710 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:11:20.010710 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:20.010710 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:11:20.011478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:20.011478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:11:20.011478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:11:20.011478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:11:20.011478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:20.011478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:11:20.011478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:20.011478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:11:20.011478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:20.011478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:11:20.011478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:20.011478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:11:20.011478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:20.011478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:11:20.011478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:11:20.011893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:20.011893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:11:20.011893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:20.011893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:11:20.011893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:11:20.011893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:20.011893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:11:20.011893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:20.011893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:11:20.012311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:20.012311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:11:20.012311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:20.012311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:11:20.012311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:11:20.012311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:11:20.012311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:11:20.012311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:11:20.012311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:11:20.012311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:20.012311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:20.012670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:20.012670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:11:20.012670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:20.012670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:20.012670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:20.012670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:11:20.012670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:20.012670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:11:20.012670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:11:20.012670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:20.012670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:11:20.012670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:20.012670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:11:20.012670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:11:20.013052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:20.013052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:11:20.013052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:20.013052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:11:20.013369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:20.013369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:11:20.013369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:11:20.013369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:11:20.013369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:11:20.013369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:11:20.013369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:20.013369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:11:20.013369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:11:20.013369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:11:20.013369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:11:20.013369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:11:20.013369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:20.013369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:20.013369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:11:20.013369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:11:20.013369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:11:20.013914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:11:20.013914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:11:20.013914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:11:20.013914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:20.013914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:20.013914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:20.013914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:11:20.013914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:20.013914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:20.013914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:20.013914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:11:20.013914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:20.013914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:11:20.013914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:11:20.013914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:20.013914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:11:20.013914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:20.013914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:11:20.013914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:11:20.014492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:20.014492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:11:20.014492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:20.014492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:11:20.015431 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:11:20.015805 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:11:20.015912 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:11:20.016065 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:11:20.016668 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:11:20.017027 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:11:20.017250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:20.018071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:11:20.018071 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:20.018071 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:11:20.018071 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:11:20.018071 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:20.018071 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:11:20.018071 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:20.018071 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:11:20.018938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:20.018938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:11:20.018938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:20.018938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:11:20.018938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:11:20.018938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:11:20.018938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:11:20.018938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:20.018938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:20.018938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:20.018938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:11:20.018938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:20.019320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:20.019320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:20.019320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:11:20.019320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:20.019320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:11:20.019320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:11:20.019320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:20.019320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:11:20.019320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:20.019320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:11:20.019320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:11:20.019320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:20.019320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:11:20.019721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:20.019721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:11:20.019891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:20.019891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:11:20.019891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:11:20.019891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:11:20.019891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:11:20.019891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:11:20.019891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:20.019891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:11:20.019891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:11:20.019891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:11:20.019891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:11:20.019891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:11:20.019891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:20.019891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:20.019891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:11:20.019891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:11:20.020355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:11:20.020355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:11:20.020355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:20.020355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:20.020355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:20.020355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:11:20.020355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:20.020355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:20.020355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:20.020355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:11:20.020355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:20.020355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:11:20.020355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:11:20.020355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:20.020355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:11:20.020355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:20.020355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:11:20.020355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:11:20.020355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:20.020942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:11:20.020942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:20.020942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:11:24.781128 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:11:24.785472 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:11:24.785472 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:11:24.785472 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:11:24.788355 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG dbcounter [-] [105581] Writing DB stats neutron:SELECT=240 {{(pid=105581) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 14:11:24.830847 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:11:24.831055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:24.833272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:11:24.833272 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:24.833272 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:11:24.833272 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:11:24.833272 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:24.833272 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:11:24.833272 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:24.833272 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:11:24.834238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.050 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:24.834238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:11:24.834238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:24.834238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:11:24.834238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:11:24.834238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:11:24.834238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:11:24.834238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:24.834238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:24.834238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:24.834238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:11:24.834238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:24.834578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:24.834578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:24.834578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:11:24.834578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:24.834578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:11:24.834578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:11:24.834578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:24.834578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:11:24.834578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:24.834578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:11:24.834578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:11:24.834578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:24.834578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:11:24.834850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:24.834850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:11:24.835847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:24.835847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:11:24.835847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:11:24.835847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:11:24.835847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:11:24.835847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:11:24.835847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:24.835847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:11:24.835847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:11:24.835847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:11:24.835847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:11:24.835847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:11:24.835847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:24.835847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:24.835847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:11:24.835847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:11:24.836357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:11:24.836357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:11:24.836357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:24.836357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:24.836357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:24.836357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:11:24.836357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:24.836357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:24.836357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:24.836357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:11:24.836357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:24.836357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:11:24.836357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:11:24.836357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:24.836357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:11:24.836357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:24.836357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:11:24.836357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:11:24.836357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:24.836745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:11:24.836745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:24.836745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:11:24.993868 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:11:24.994915 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:11:24.994915 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:11:24.995684 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:11:24.996022 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:11:24.996260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:24.999482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:11:24.999482 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:24.999482 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:11:24.999482 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:11:24.999482 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:24.999482 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:11:24.999482 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:24.999482 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:11:24.999482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:24.999482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:11:24.999482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:24.999482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:11:24.999482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:11:24.999482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:11:24.999482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:11:24.999951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:24.999951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:24.999951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:24.999951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:11:24.999951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:24.999951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:24.999951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:24.999951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:11:24.999951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:24.999951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:11:24.999951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:11:24.999951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:24.999951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:11:24.999951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:25.000325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:11:25.000325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:11:25.000325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:25.000325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:11:25.000325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:25.000325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:11:25.000325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:25.000325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:11:25.000325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:11:25.000325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:11:25.000325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:11:25.000325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:11:25.000325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:25.000325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:11:25.000705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:11:25.000705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:11:25.000705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:11:25.000705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:11:25.000705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:25.000705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:25.000705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:11:25.000705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:11:25.000705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:11:25.000705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:11:25.000705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:25.000705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:25.000705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:25.000705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:11:25.000705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:25.000705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:25.000705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:25.000705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:11:25.000705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:25.000705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:11:25.001249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:11:25.001249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:25.001249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:11:25.001249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:25.001249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:11:25.001249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:11:25.001249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:25.001249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:11:25.001249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:25.001249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:11:25.002715 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:11:25.003009 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:11:25.003226 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:11:25.003481 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:11:25.003909 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:11:25.004099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:25.007442 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:11:25.007442 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:25.007442 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:11:25.007442 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:11:25.007442 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:25.007442 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:11:25.007442 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:25.007442 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:11:25.007442 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:25.007442 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:11:25.007442 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:11:25.007442 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:11:25.007442 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:25.007442 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:11:25.007442 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:25.007442 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:11:25.007442 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:25.007442 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:11:25.008036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:25.008036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:11:25.008036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:25.008036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:11:25.008036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:11:25.008036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:25.008036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:11:25.008036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:25.008036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:11:25.008036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:11:25.008036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:25.008036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:11:25.008036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:25.008036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:11:25.008036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:25.008537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:11:25.008537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:25.008537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:11:25.008537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:11:25.008537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:11:25.008537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:11:25.008537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:11:25.008537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:11:25.008537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:11:25.008537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:25.008537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:25.008537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:25.008537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:11:25.008919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:25.008919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:25.008919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:25.008919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:11:25.008919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:25.008919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:11:25.008919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:11:25.008919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:25.008919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:11:25.008919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:25.008919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:11:25.008919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:11:25.008919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:25.008919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:11:25.009342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:25.009342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:11:25.009498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:25.009498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:11:25.009498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:11:25.009498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:11:25.009498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:11:25.009498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:11:25.009498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:25.009498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:11:25.009498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:11:25.009498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:11:25.009498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:11:25.009498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:11:25.009498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:25.009498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:25.009498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:11:25.009498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:11:25.009939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:11:25.009939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:11:25.009939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:11:25.009939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:11:25.009939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:11:25.009939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:25.009939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:25.009939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:25.009939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:11:25.009939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:25.009939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:25.009939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:25.009939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:11:25.009939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:25.009939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:11:25.009939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:11:25.009939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:25.009939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:11:25.010432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:25.010432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:11:25.010432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:11:25.010432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:25.010432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:11:25.010432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:25.010432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:11:25.010812 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:11:25.011446 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:11:25.011446 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:11:25.011446 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:11:25.011901 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:11:25.012265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:25.016295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:11:25.016295 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:25.016295 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:11:25.016295 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:11:25.016295 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:25.016295 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:11:25.016295 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:25.016295 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:11:25.016295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:25.016295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:11:25.016295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:11:25.016295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:11:25.016295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:25.016295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:11:25.016295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:25.016295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:11:25.016295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:25.016295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:11:25.016820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:25.016820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:11:25.016820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:25.016820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:11:25.016820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:11:25.016820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:25.016820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:11:25.016820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:25.016820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:11:25.016820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:11:25.016820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:25.016820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:11:25.016820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:25.016820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:11:25.016820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:25.017239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:11:25.017239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:25.017239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:11:25.017239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:11:25.017239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:11:25.017239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:11:25.017239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:11:25.017239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:11:25.017239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:25.017239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:25.017239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:25.017239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:11:25.017239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:25.017616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:25.017616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:25.017616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:11:25.017616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:25.017616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:11:25.017616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:11:25.017616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:25.017616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:11:25.017616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:25.017616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:11:25.017616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:11:25.017616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:25.017616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:11:25.018006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:25.018006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:11:25.018006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:25.018006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:11:25.018006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:11:25.018006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:11:25.018006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:11:25.018006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:11:25.018006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:25.018006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:11:25.018006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:11:25.018006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:11:25.018006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:11:25.018006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:11:25.018006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:25.018435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:25.018435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:11:25.018435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:11:25.018435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:11:25.018435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:11:25.018435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:11:25.018435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:11:25.018435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:25.018435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:25.018435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:25.018435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:11:25.018435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:25.018435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:25.018435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:25.018435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:11:25.018435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:25.018435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:11:25.018435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:11:25.018435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:25.018962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:11:25.018962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:25.018962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:11:25.018962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:11:25.018962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:25.018962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:11:25.018962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:25.018962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:11:25.018962 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:11:25.019473 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:11:25.019683 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:11:25.019928 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:11:25.020816 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:11:25.021023 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:11:25.021167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:25.023917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:11:25.023917 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:25.023917 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:11:25.023917 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:11:25.023917 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:25.023917 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:11:25.023917 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:25.023917 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:11:25.023917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:25.023917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:11:25.023917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:25.023917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:11:25.023917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:11:25.023917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:11:25.023917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:11:25.024359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:25.024359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:25.024359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:25.024359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:11:25.024359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:25.024359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:25.024359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:25.024359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:11:25.024359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:25.024359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:11:25.024359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:11:25.024359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:25.024359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:11:25.024359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:25.024695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:11:25.024695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:11:25.024695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:25.024695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:11:25.024695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:25.024695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:11:25.024695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:25.024695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:11:25.024695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:11:25.024695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:11:25.024695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:11:25.024695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:11:25.024695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:25.024695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:11:25.024967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:11:25.024967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:11:25.024967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:11:25.024967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:11:25.024967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:25.024967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:25.024967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:11:25.024967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:11:25.024967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:11:25.024967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:11:25.024967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:25.024967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:25.024967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:25.024967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:11:25.024967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:25.024967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:25.024967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:25.024967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:11:25.024967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:25.024967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:11:25.025350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:11:25.025350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:25.025350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:11:25.025350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:25.025350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:11:25.025350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:11:25.025350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:25.025350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:11:25.025350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:25.025350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:11:29.782932 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:11:29.783529 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:11:29.783707 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:11:29.785545 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:11:29.823202 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:11:29.823787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:29.824893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:11:29.824893 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:29.824893 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:11:29.824893 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:11:29.824893 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:29.824893 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:11:29.824893 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:29.824893 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:11:29.825985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:29.825985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:11:29.825985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:29.825985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:11:29.825985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:11:29.825985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:11:29.825985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:11:29.825985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:29.825985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:29.825985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:29.825985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:11:29.825985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:29.826700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:29.826700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:29.826700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:11:29.826700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:29.826700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:11:29.826700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:11:29.826700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:29.826700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:11:29.826700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:29.826700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:11:29.826700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:11:29.826700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:29.826700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:11:29.827188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:29.827188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:11:29.827188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:29.827188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:11:29.827188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:11:29.827188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:11:29.827188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:11:29.827188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:11:29.827188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:29.827188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:11:29.827188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:11:29.827188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:11:29.827188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:11:29.827188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:11:29.827188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:29.827635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:29.827635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:11:29.827635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:11:29.827635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:11:29.827635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:11:29.827635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:29.827635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:29.827635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:29.827635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:11:29.827635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:29.827635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:29.827635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:29.827635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:11:29.827635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:29.827635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:11:29.827635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:11:29.827635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:29.827635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:11:29.827635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:29.828154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:11:29.828154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:11:29.828154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:29.828154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:11:29.828154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:29.828154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:11:29.994559 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:11:29.995102 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:11:29.995292 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:11:29.996594 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:11:29.997030 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:11:29.997299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:29.998348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:11:29.998348 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:29.998348 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:11:29.998348 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:11:29.998348 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:29.998348 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:11:29.998348 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:29.998348 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:11:29.999903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:29.999903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:11:29.999903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:29.999903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:11:29.999903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:11:29.999903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:11:29.999903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:11:29.999903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:29.999903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:29.999903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:29.999903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:11:29.999903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:30.000547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:30.000547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:30.000547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:11:30.000547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:30.000547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:11:30.000547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:11:30.000547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:30.000547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:11:30.000547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:30.000547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:11:30.000547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:11:30.000547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:30.000547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:11:30.001171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:30.001171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:11:30.001382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:30.001382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:11:30.001382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:11:30.001382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:11:30.001382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:11:30.001382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:11:30.001382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:30.001382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:11:30.001382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:11:30.001382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:11:30.001382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:11:30.001382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:11:30.001382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:30.001382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:30.001382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:11:30.001382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:11:30.002105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:11:30.002105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:11:30.002105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:30.002105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:30.002105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:30.002105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:11:30.002105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:30.002105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:30.002105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:30.002105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:11:30.002105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:30.002105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:11:30.002105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:11:30.002105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:30.002105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:11:30.002105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:30.002105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:11:30.002105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:11:30.002105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:30.003065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:11:30.003065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:30.003065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:11:30.003219 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:11:30.003436 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:11:30.003539 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:11:30.003658 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:11:30.004247 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:11:30.004602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:30.005371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:11:30.005371 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:30.005371 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:11:30.005371 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:11:30.005371 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:30.005371 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:11:30.005371 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:30.005371 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:11:30.006112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:30.006112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:11:30.006112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:11:30.006112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:11:30.006112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:30.006112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:11:30.006112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:30.006112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:11:30.006112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:30.006112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:11:30.006112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:30.006112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:11:30.006112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:30.006112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:11:30.006112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:11:30.006623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:30.006623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:11:30.006623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:30.006623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:11:30.006623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:11:30.006623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:30.006623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:11:30.006623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:30.006623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:11:30.006907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:30.006907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:11:30.006907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:30.006907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:11:30.006907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:11:30.006907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:11:30.006907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:11:30.006907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:11:30.006907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:11:30.006907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:11:30.006907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:30.006907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:30.007267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:30.007267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:11:30.007267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:30.007267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:30.007267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:30.007267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:11:30.007267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:30.007267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:11:30.007267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:11:30.007267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:30.007267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:11:30.007267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:30.007267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:11:30.007267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:11:30.007695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:30.007695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:11:30.007695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:30.007695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:11:30.007880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:30.007880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:11:30.007880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:11:30.007880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:11:30.007880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:11:30.007880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:11:30.007880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:30.007880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:11:30.007880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:11:30.007880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:11:30.007880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:11:30.007880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:11:30.007880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:30.007880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:30.007880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:11:30.007880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:11:30.008360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:11:30.008360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:11:30.008360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:11:30.008360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:11:30.008360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:11:30.008360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:30.008360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:30.008360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:30.008360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:11:30.008360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:30.008360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:30.008360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:30.008360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:11:30.008360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:30.008360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:11:30.008360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:11:30.008360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:30.008360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:11:30.008912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:30.008912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:11:30.008912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:11:30.008912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:30.008912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:11:30.008912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:30.008912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:11:30.011293 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:11:30.011619 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:11:30.011697 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:11:30.011823 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:11:30.012249 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:11:30.012738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:30.013364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:11:30.013364 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:30.013364 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:11:30.013364 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:11:30.013364 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:30.013364 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:11:30.013364 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:30.013364 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:11:30.014086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:30.014086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:11:30.014086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:11:30.014086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:11:30.014086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:30.014086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:11:30.014086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:30.014086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:11:30.014086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:30.014086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:11:30.014086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:30.014086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:11:30.014086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:30.014086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:11:30.014086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:11:30.014537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:30.014537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:11:30.014537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:30.014537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:11:30.014537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:11:30.014537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:30.014537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:11:30.014537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:30.014537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:11:30.014918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:30.014918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:11:30.014918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:30.014918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:11:30.014918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:11:30.014918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:11:30.014918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:11:30.014918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:11:30.014918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:11:30.014918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:30.014918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:30.015256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:30.015256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:11:30.015256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:30.015256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:30.015256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:30.015256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:11:30.015256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:30.015256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:11:30.015256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:11:30.015256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:30.015256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:11:30.015256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:30.015256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:11:30.015256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:11:30.015670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:30.015670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:11:30.015670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:30.015670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:11:30.015948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:30.015948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:11:30.015948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:11:30.015948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:11:30.015948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:11:30.015948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:11:30.015948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:30.015948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:11:30.015948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:11:30.015948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:11:30.015948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:11:30.015948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:11:30.015948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:30.015948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:30.015948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:11:30.015948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:11:30.015948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:11:30.016474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:11:30.016474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:11:30.016474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:11:30.016474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:30.016474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:30.016474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:30.016474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:11:30.016474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:30.016474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:30.016474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:30.016474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:11:30.016474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:30.016474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:11:30.016474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:11:30.016474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:30.016474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:11:30.016474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:30.016474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:11:30.016474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:11:30.017126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:30.017126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:11:30.017126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:30.017126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:11:30.019602 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:11:30.019860 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:11:30.019967 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:11:30.020112 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:11:30.020559 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:11:30.021044 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:11:30.021275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:30.021931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:11:30.021931 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:30.021931 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:11:30.021931 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:11:30.021931 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:30.021931 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:11:30.021931 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:30.021931 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:11:30.022801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:30.022801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:11:30.022801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:30.022801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:11:30.022801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:11:30.022801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:11:30.022801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:11:30.022801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:30.022801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:30.022801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:30.022801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:11:30.022801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:30.023136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:30.023136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:30.023136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:11:30.023136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:30.023136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:11:30.023136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:11:30.023136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:30.023136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:11:30.023136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:30.023136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:11:30.023136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:11:30.023136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:30.023136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:11:30.023527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:30.023527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:11:30.023739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:30.023739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:11:30.023739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:11:30.023739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:11:30.023739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:11:30.023739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:11:30.023739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:30.023739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:11:30.023739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:11:30.023739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:11:30.023739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:11:30.023739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:11:30.023739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:30.023739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:30.023739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:11:30.023739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:11:30.024193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:11:30.024193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:11:30.024193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:30.024193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:30.024193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:30.024193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:11:30.024193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:30.024193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:30.024193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:30.024193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:11:30.024193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:30.024193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:11:30.024193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:11:30.024193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:30.024193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:11:30.024193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:30.024193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:11:30.024193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:11:30.024193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:30.024737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:11:30.024737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:30.024737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:11:34.783562 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:11:34.784644 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:11:34.784992 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:11:34.786475 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:11:34.821662 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:11:34.821995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:34.822819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:11:34.822819 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:34.822819 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:11:34.822819 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:11:34.822819 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:34.822819 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:11:34.822819 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:34.822819 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:11:34.823778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:34.823778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:11:34.823778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:34.823778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:11:34.823778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:11:34.823778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:11:34.823778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:11:34.823778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:34.823778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:34.823778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:34.823778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:11:34.823778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:34.824305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:34.824305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:34.824305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:11:34.824305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:34.824305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:11:34.824305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:11:34.824305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:34.824305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:11:34.824305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:34.824305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:11:34.824305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:11:34.824305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:34.824305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:11:34.825086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:34.825086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:11:34.825086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:34.825086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:11:34.825086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:11:34.825086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:11:34.825086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:11:34.825086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:11:34.825086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:34.825086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:11:34.825086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:11:34.825086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:11:34.825086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:11:34.825086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:11:34.825086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:34.826044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:34.826044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:11:34.826044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:11:34.826044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:11:34.826044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:11:34.826044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:34.826044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:34.826044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:34.826044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:11:34.826044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:34.826044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:34.826044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:34.826044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:11:34.826044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:34.826044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:11:34.826044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:11:34.826044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:34.826044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:11:34.826044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:34.827276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:11:34.827276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:11:34.827276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:34.827276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:11:34.827276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:34.827276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:11:34.995067 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:11:34.995724 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:11:34.995901 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:11:34.997058 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:11:34.998175 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:11:34.998175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:34.998779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:11:34.998779 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:34.998779 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:11:34.998779 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:11:34.998779 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:34.998779 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:11:34.998779 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:34.998779 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:11:35.000438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:35.000438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:11:35.000438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:35.000438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:11:35.000438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:11:35.000438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:11:35.000438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:11:35.000438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:35.000438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:35.000438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:35.000438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:11:35.000438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:35.000762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:35.000762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:35.000762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:11:35.000762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:35.000762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:11:35.000762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:11:35.000762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:35.000762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:11:35.000762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:35.000762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:11:35.000762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:11:35.000762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:35.000762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:11:35.001016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:35.001016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:11:35.002826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:35.002826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:11:35.002826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:11:35.002826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:11:35.002826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:11:35.002826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:11:35.002826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:35.002826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:11:35.002826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:11:35.002826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:11:35.002826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:11:35.002826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:11:35.002826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:35.002826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:35.002826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:11:35.002826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:11:35.003200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:11:35.003200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:11:35.003200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:35.003200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:35.003200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:35.003200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:11:35.003200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:35.003200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:35.003200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:35.003200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:11:35.003200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:35.003200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:11:35.003200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:11:35.003200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:35.003200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:11:35.003200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:35.003200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:11:35.003200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:11:35.003200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:35.003626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:11:35.003626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:35.003626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:11:35.003626 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:11:35.003860 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:11:35.004578 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:11:35.004578 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:11:35.004578 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:11:35.004922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:35.005500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:11:35.005500 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:35.005500 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:11:35.005500 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:11:35.005500 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:35.005500 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:11:35.005500 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:35.005500 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:11:35.006539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:35.006539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:11:35.006539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:11:35.006539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:11:35.006539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:35.006539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:11:35.006539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:35.006539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:11:35.006539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:35.006539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:11:35.006539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:35.006539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:11:35.006539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:35.006539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:11:35.006539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:11:35.007337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:35.007337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:11:35.007337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:35.007337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:11:35.007337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:11:35.007337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:35.007337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:11:35.007337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:35.007337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:11:35.007337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:35.007337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:11:35.007337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:35.007337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:11:35.007337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:11:35.008068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:11:35.008068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:11:35.008068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:11:35.008068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:11:35.008068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:11:35.008068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:35.008068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:35.008068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:35.008068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:11:35.008068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:35.008068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:35.008068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:35.008068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:11:35.008744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:35.008744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:11:35.008744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:11:35.008744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:35.008744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:11:35.008744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:35.008744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:11:35.008744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:11:35.008744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:35.008744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:11:35.008744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:35.008744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:11:35.009313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:35.009313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:11:35.009313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:11:35.009313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:11:35.009313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:11:35.009313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:11:35.009313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:35.009313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:11:35.009313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:11:35.009313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:11:35.009313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:11:35.009313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:11:35.009313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:35.009313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:35.009313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:11:35.009313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:11:35.010296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:11:35.010296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:11:35.010296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:11:35.010296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:11:35.010296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:11:35.010296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:35.010296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:35.010296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:35.010296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:11:35.010296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:35.010296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:35.010296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:35.010296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:11:35.010296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:35.010296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:11:35.010296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:11:35.010296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:35.010296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:11:35.010827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:35.010827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:11:35.010827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:11:35.010827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:35.010827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:11:35.010827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:35.010827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:11:35.011748 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:11:35.012035 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:11:35.012146 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:11:35.012272 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:11:35.012764 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:11:35.013160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:35.014042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:11:35.014042 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:35.014042 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:11:35.014042 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:11:35.014042 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:35.014042 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:11:35.014042 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:35.014042 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:11:35.015453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:35.015453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:11:35.015453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:11:35.015453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:11:35.015453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:35.015453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:11:35.015453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:35.015453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:11:35.015453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:35.015453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:11:35.015453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:35.015453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:11:35.015453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:35.015453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:11:35.015453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:11:35.015980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:35.015980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:11:35.015980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:35.015980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:11:35.015980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:11:35.015980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:35.015980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:11:35.015980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:35.015980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:11:35.015980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:35.015980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:11:35.015980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:35.015980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:11:35.015980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:11:35.016401 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:11:35.016401 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:11:35.016401 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:11:35.016401 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:11:35.016401 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:35.016401 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:35.016401 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:35.016401 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:11:35.016401 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:35.016401 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:35.016401 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:35.016401 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:11:35.016401 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:35.016811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:11:35.016811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:11:35.016811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:35.016811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:11:35.016811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:35.016811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:11:35.016811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:11:35.016811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:35.016811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:11:35.016811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:35.016811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:11:35.016811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:35.017157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:11:35.017157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:11:35.017157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:11:35.017157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:11:35.017157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:11:35.017157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:35.017157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:11:35.017157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:11:35.017157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:11:35.017157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:11:35.017157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:11:35.017157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:35.017157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:35.017157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:11:35.017157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:11:35.017157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:11:35.017157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:11:35.017157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:11:35.017157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:11:35.017723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:35.017723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:35.017723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:35.017723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:11:35.017723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:35.017723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:35.017723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:35.017723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:11:35.017723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:35.017723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:11:35.017723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:11:35.017723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:35.017723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:11:35.017723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:35.017723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:11:35.017723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:11:35.017723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:35.017723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:11:35.017723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:35.017723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:11:35.020082 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:11:35.020374 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:11:35.020509 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:11:35.020657 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:11:35.021123 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:11:35.021564 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:11:35.021808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:35.022573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:11:35.022573 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:35.022573 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:11:35.022573 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:11:35.022573 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:35.022573 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:11:35.022573 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:35.022573 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:11:35.023373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:35.023373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:11:35.023373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:35.023373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:11:35.023373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:11:35.023373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:11:35.023373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:11:35.023373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:35.023373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:35.023373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:35.023373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:11:35.023373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:35.023911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:35.023911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:35.023911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:11:35.023911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:35.023911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:11:35.023911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:11:35.023911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:35.023911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:11:35.023911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:35.023911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:11:35.023911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:11:35.023911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:35.023911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:11:35.024396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:35.024396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:11:35.024396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:35.024396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:11:35.024396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:11:35.024396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:11:35.024396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:11:35.024396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:11:35.024396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:35.024396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:11:35.024396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:11:35.024396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:11:35.024396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:11:35.024396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:11:35.024396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:35.025015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:35.025015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:11:35.025015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:11:35.025015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:11:35.025015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:11:35.025015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:35.025015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:35.025015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:35.025015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:11:35.025015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:35.025015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:35.025015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:35.025015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:11:35.025015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:35.025015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:11:35.025015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:11:35.025015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:35.025015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:11:35.025015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:35.025839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:11:35.025839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:11:35.025839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:35.025839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:11:35.025839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:35.025839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:11:39.784511 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:11:39.785401 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:11:39.785401 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:11:39.786054 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:11:39.823113 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:11:39.823385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:39.824255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:11:39.824255 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:39.824255 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:11:39.824255 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:11:39.824255 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:39.824255 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:11:39.824255 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:39.824255 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:11:39.825278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:39.825278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:11:39.825278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:39.825278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:11:39.825278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:11:39.825278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:11:39.825278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:11:39.825278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:39.825278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:39.825278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:39.825278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:11:39.825278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:39.825841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:39.825841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:39.825841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:11:39.825841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:39.825841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:11:39.825841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:11:39.825841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:39.825841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:11:39.825841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:39.825841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:11:39.825841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:11:39.825841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:39.825841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:11:39.826318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:39.826318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:11:39.826318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:39.826318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:11:39.826318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:11:39.826318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:11:39.826318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:11:39.826318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:11:39.826318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:39.826318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:11:39.826318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:11:39.826318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:11:39.826318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:11:39.826318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:11:39.826318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:39.826954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:39.826954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:11:39.826954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:11:39.826954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:11:39.826954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:11:39.826954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:39.826954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:39.826954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:39.826954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:11:39.826954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:39.826954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:39.826954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:39.826954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:11:39.826954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:39.826954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:11:39.826954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:11:39.826954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:39.826954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:11:39.826954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:39.827759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:11:39.827759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:11:39.827759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:39.827759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:11:39.827759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:39.827759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:11:39.995619 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:11:39.996235 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:11:39.996463 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:11:39.997964 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:11:39.998374 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:11:39.998741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:39.999816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:11:39.999816 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:39.999816 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:11:39.999816 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:11:39.999816 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:39.999816 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:11:39.999816 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:39.999816 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:11:40.001658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:40.001658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:11:40.001658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:40.001658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:11:40.001658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:11:40.001658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:11:40.001658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:11:40.001658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:40.001658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:40.001658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:40.001658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:11:40.001658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:40.002056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:40.002056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:40.002056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:11:40.002056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:40.002056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:11:40.002056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:11:40.002056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:40.002056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:11:40.002056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:40.002056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:11:40.002056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:11:40.002056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:40.002056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:11:40.002466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:40.002466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:11:40.003065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:40.003065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:11:40.003065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:11:40.003065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:11:40.003065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:11:40.003065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:11:40.003065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:40.003065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:11:40.003065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:11:40.003065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:11:40.003065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:11:40.003065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:11:40.003065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:40.003065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:40.003065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:11:40.003065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:11:40.003556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:11:40.003556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:11:40.003556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:40.003556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:40.003556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:40.003556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:11:40.003556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:40.003556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:40.003556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:40.003556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:11:40.003556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:40.003556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:11:40.003556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:11:40.003556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:40.003556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:11:40.003556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:40.003556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:11:40.003556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:11:40.003556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:40.004298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:11:40.004298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:40.004298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:11:40.004298 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:11:40.004298 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:11:40.004479 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:11:40.004521 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:11:40.005118 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:11:40.005596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:40.006681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:11:40.006681 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:40.006681 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:11:40.006681 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:11:40.006681 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:40.006681 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:11:40.006681 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:40.006681 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:11:40.007458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:40.007458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:11:40.007458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:11:40.007458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:11:40.007458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:40.007458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:11:40.007458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:40.007458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:11:40.007458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:40.007458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:11:40.007458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:40.007458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:11:40.007458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:40.007458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:11:40.007458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:11:40.007944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:40.007944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:11:40.007944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:40.007944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:11:40.007944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:11:40.007944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:40.007944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:11:40.007944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:40.007944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:11:40.008584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:40.008584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:11:40.008584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:40.008584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:11:40.008584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:11:40.008584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:11:40.008584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:11:40.008584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:11:40.008584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:11:40.008584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:11:40.008584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:40.008584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:40.008968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:40.008968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:11:40.008968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:40.008968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:40.008968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:40.008968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:11:40.008968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:40.008968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:11:40.008968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:11:40.008968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:40.008968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:11:40.008968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:40.008968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:11:40.008968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:11:40.009360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:40.009360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:11:40.009360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:40.009360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:11:40.009360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:40.009360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:11:40.009360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:11:40.009360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:11:40.009360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:11:40.009360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:11:40.009360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:40.009360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:11:40.009360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:11:40.009360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:11:40.009799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:11:40.009799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:11:40.009799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:40.009799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:40.009799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:11:40.009799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:11:40.009799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:11:40.009799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:11:40.009799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:11:40.009799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:11:40.009799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:11:40.009799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:40.009799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:40.009799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:40.009799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:11:40.009799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:40.009799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:40.009799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:40.009799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:11:40.010332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:40.010332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:11:40.010332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:11:40.010332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:40.010332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:11:40.010332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:40.010332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:11:40.010332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:11:40.010332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:40.010332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:11:40.010332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:40.010332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:11:40.012231 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:11:40.012551 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:11:40.012661 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:11:40.012786 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:11:40.013310 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:11:40.013711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:40.014462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:11:40.014462 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:40.014462 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:11:40.014462 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:11:40.014462 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:40.014462 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:11:40.014462 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:40.014462 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:11:40.015217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:40.015217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:11:40.015217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:11:40.015217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:11:40.015217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:40.015217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:11:40.015217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:40.015217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:11:40.015217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:40.015217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:11:40.015217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:40.015217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:11:40.015217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:40.015217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:11:40.015217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:11:40.015646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:40.015646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:11:40.015646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:40.015646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:11:40.015646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:11:40.015646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:40.015646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:11:40.015646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:40.015646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:11:40.016090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:40.016090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:11:40.016090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:40.016090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:11:40.016090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:11:40.016090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:11:40.016090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:11:40.016090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:11:40.016090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:11:40.016090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:40.016090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:40.016435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:40.016435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:11:40.016435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:40.016435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:40.016435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:40.016435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:11:40.016435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:40.016435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:11:40.016435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:11:40.016435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:40.016435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:11:40.016435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:40.016435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:11:40.016435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:11:40.016858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:40.016858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:11:40.016858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:40.016858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:11:40.017129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:40.017129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:11:40.017129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:11:40.017129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:11:40.017129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:11:40.017129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:11:40.017129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:40.017129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:11:40.017129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:11:40.017129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:11:40.017129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:11:40.017129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:11:40.017129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:40.017129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:40.017129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:11:40.017129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:11:40.017129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:11:40.017706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:11:40.017706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:11:40.017706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:11:40.017706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:40.017706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:40.017706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:40.017706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:11:40.017706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:40.017706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:40.017706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:40.017706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:11:40.017706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:40.017706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:11:40.017706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:11:40.017706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:40.017706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:11:40.017706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:40.017706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:11:40.017706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:11:40.018251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:40.018251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:11:40.018251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:40.018251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:11:40.020572 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:11:40.020863 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:11:40.020972 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:11:40.021121 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:11:40.021654 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:11:40.022015 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:11:40.022250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:40.022992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:11:40.022992 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:40.022992 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:11:40.022992 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:11:40.022992 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:40.022992 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:11:40.022992 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:40.022992 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:11:40.023872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:40.023872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:11:40.023872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:40.023872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:11:40.023872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:11:40.023872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:11:40.023872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:11:40.023872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:40.023872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:40.023872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:40.023872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:11:40.023872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:40.024233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:40.024233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:40.024233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:11:40.024233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:40.024233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:11:40.024233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:11:40.024233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:40.024233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:11:40.024233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:40.024233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:11:40.024233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:11:40.024233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:40.024233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:11:40.024631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:40.024631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:11:40.024825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:40.024825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:11:40.024825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:11:40.024825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:11:40.024825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:11:40.024825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:11:40.024825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:40.024825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:11:40.024825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:11:40.024825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:11:40.024825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:11:40.024825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:11:40.024825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:40.024825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:40.024825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:11:40.024825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:11:40.025280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:11:40.025280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:11:40.025280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:40.025280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:40.025280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:40.025280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:11:40.025280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:40.025280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:40.025280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:40.025280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:11:40.025280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:40.025280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:11:40.025280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:11:40.025280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:40.025280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:11:40.025280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:40.025280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:11:40.025280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:11:40.025280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:40.025831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:11:40.025831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:40.025831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:11:44.788790 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:11:44.789313 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:11:44.789496 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:11:44.790371 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:11:44.825631 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:11:44.825803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:44.826417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:11:44.826417 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:44.826417 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:11:44.826417 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:11:44.826417 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:44.826417 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:11:44.826417 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:44.826417 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:11:44.827168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:44.827168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:11:44.827168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:44.827168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:11:44.827168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:11:44.827168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:11:44.827168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:11:44.827168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:44.827168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:44.827168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:44.827168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:11:44.827168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:44.827545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:44.827545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:44.827545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:11:44.827545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:44.827545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:11:44.827545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:11:44.827545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:44.827545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:11:44.827545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:44.827545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:11:44.827545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:11:44.827545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:44.827545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:11:44.827860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:44.827860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:11:44.827927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:44.827927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:11:44.827927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:11:44.827927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:11:44.827927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:11:44.827927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:11:44.827927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:44.827927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:11:44.827927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:11:44.827927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:11:44.827927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:11:44.827927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:11:44.827927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:44.827927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:44.827927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:11:44.827927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:11:44.828324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:11:44.828324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:11:44.828324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:44.828324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:44.828324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:44.828324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:11:44.828324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:44.828324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:44.828324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:44.828324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:11:44.828324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:44.828324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:11:44.828324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:11:44.828324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:44.828324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:11:44.828324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:44.828324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:11:44.828324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:11:44.828324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:44.828823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:11:44.828823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:44.828823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:11:44.996327 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:11:44.996811 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:11:44.996893 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:11:44.997733 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:11:44.997973 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:11:44.998098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:44.998773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:11:44.998773 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:44.998773 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:11:44.998773 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:11:44.998773 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:44.998773 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:11:44.998773 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:44.998773 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:11:45.000903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:45.000903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:11:45.000903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:45.000903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:11:45.000903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:11:45.000903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:11:45.000903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:11:45.000903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:45.000903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:45.000903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:45.000903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:11:45.000903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:45.001438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:45.001438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:45.001438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:11:45.001438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:45.001438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:11:45.001438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:11:45.001438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:45.001438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:11:45.001438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:45.001438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:11:45.001438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:11:45.001438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:45.001438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:11:45.001951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:45.001951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:11:45.001951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:45.001951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:11:45.001951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:11:45.001951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:11:45.001951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:11:45.001951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:11:45.001951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:45.001951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:11:45.001951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:11:45.001951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:11:45.001951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:11:45.001951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:11:45.001951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:45.003842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:45.003842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:11:45.003842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:11:45.003842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:11:45.003842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:11:45.003842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:45.003842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:45.003842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:45.003842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:11:45.003842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:45.003842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:45.003842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:45.003842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:11:45.003842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:45.003842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:11:45.003842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:11:45.003842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:45.003842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:11:45.003842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:45.006229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:11:45.006229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:11:45.006229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:45.006229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:11:45.006229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:45.006229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:11:45.006229 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:11:45.006229 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:11:45.006229 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:11:45.006229 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:11:45.007675 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:11:45.007675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:45.007675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:11:45.007675 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:45.007675 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:11:45.007675 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:11:45.007675 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:45.007675 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:11:45.007675 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:45.007675 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:11:45.008579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:45.008579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:11:45.008579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:11:45.008579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:11:45.008579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:45.008579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:11:45.008579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:45.008579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:11:45.008579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:45.008579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:11:45.008579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:45.008579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:11:45.008579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:45.008579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:11:45.008579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:11:45.009285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:45.009285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:11:45.009285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:45.009285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:11:45.009285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:11:45.009285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:45.009285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:11:45.009285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:45.009285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:11:45.009285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:45.009285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:11:45.009285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:45.009285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:11:45.009285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:11:45.009660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:11:45.009660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:11:45.009660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:11:45.009660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:11:45.009660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:11:45.009660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:45.009660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:45.009660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:45.009660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:11:45.009660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:45.009660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:45.009660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:45.009660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:11:45.009991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:45.009991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:11:45.009991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:11:45.009991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:45.009991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:11:45.009991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:45.009991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:11:45.009991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:11:45.009991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:45.009991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:11:45.009991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:45.009991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:11:45.010294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:45.010294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:11:45.010294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:11:45.010294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:11:45.010294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:11:45.010294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:11:45.010294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:45.010294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:11:45.010294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:11:45.010294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:11:45.010294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:11:45.010294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:11:45.010294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:45.010294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:45.010294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:11:45.010294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:11:45.011624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:11:45.011624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:11:45.011624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:11:45.011624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:11:45.011624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:11:45.011624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:45.011624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:45.011624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:45.011624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:11:45.011624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:45.011624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:45.011624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:45.011624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:11:45.011624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:45.011624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:11:45.011624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:11:45.011624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:45.011624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:11:45.012653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:45.012653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:11:45.012653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:11:45.012653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:45.012653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:11:45.012653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:45.012653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:11:45.012653 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:11:45.012984 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:11:45.013111 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:11:45.013242 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:11:45.013776 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:11:45.014176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:45.015889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:11:45.015889 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:45.015889 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:11:45.015889 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:11:45.015889 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:45.015889 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:11:45.015889 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:45.015889 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:11:45.019669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:45.019669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:11:45.019669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:11:45.019669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:11:45.019669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:45.019669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:11:45.019669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:45.019669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:11:45.019669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:45.019669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:11:45.019669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:45.019669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:11:45.019669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:45.019669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:11:45.019669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:11:45.020083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:45.020083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:11:45.020083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:45.020083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:11:45.020083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:11:45.020083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:45.020083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:11:45.020083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:45.020083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:11:45.020645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:45.020645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:11:45.020645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:45.020645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:11:45.020645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:11:45.020645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:11:45.020645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:11:45.020645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:11:45.020645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:11:45.020645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:45.020645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:45.020999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:45.020999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:11:45.020999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:45.020999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:45.020999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:45.020999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:11:45.020999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:45.020999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:11:45.020999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:11:45.020999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:45.020999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:11:45.020999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:45.020999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:11:45.020999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:11:45.021355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:45.021355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:11:45.021355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:45.021355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:11:45.021947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:45.021947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:11:45.021947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:11:45.021947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:11:45.021947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:11:45.021947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:11:45.021947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:45.021947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:11:45.021947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:11:45.021947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:11:45.021947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:11:45.021947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:11:45.021947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:45.021947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:45.021947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:11:45.021947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:11:45.021947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:11:45.023428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:11:45.023428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:11:45.023428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:11:45.023428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:45.023428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:45.023428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:45.023428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:11:45.023428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:45.023428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:45.023428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:45.023428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:11:45.023428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:45.023428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:11:45.023428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:11:45.023428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:45.023428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:11:45.023428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:45.023428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:11:45.023428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:11:45.024017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:45.024017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:11:45.024017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:45.024017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:11:45.024017 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:11:45.025730 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:11:45.025837 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:11:45.026015 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:11:45.026586 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:11:45.026944 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:11:45.027233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:45.027991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:11:45.027991 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:45.027991 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:11:45.027991 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:11:45.027991 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:45.027991 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:11:45.027991 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:45.027991 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:11:45.028702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:45.028702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:11:45.028702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:45.028702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:11:45.028702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:11:45.028702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:11:45.028702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:11:45.028702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:45.028702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:45.028702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:45.028702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:11:45.028702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:45.029032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:45.029032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:45.029032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:11:45.029032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:45.029032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:11:45.029032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:11:45.029032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:45.029032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:11:45.029032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:45.029032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:11:45.029032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:11:45.029032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:45.029032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:11:45.029338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:45.029338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:11:45.029422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:45.029422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:11:45.029422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:11:45.029422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:11:45.029422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:11:45.029422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:11:45.029422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:45.029422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:11:45.029422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:11:45.029422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:11:45.029422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:11:45.029422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:11:45.029422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:45.029422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:45.029422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:11:45.029422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:11:45.029816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:11:45.029816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:11:45.029816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:45.029816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:45.029816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:45.029816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:11:45.029816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:45.029816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:45.029816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:45.029816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:11:45.029816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:45.029816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:11:45.029816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:11:45.029816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:45.029816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:11:45.029816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:45.029816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:11:45.029816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:11:45.029816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:45.030268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:11:45.030268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:45.030268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:11:49.789399 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:11:49.790069 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:11:49.790352 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:11:49.792230 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:11:49.834480 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:11:49.835048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:49.836118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:11:49.836118 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:49.836118 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:11:49.836118 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:11:49.836118 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:49.836118 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:11:49.836118 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:49.836118 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:11:49.837074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.046 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:49.837074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:11:49.837074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:49.837074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:11:49.837074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:11:49.837074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:11:49.837074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:11:49.837074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:49.837074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:49.837074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:49.837074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:11:49.837074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:49.837725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:49.837725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:49.837725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:11:49.837725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:49.837725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:11:49.837725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:11:49.837725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:49.837725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:11:49.837725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:49.837725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:11:49.837725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:11:49.837725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:49.837725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:11:49.838270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:49.838270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:11:49.838270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:49.838270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:11:49.838270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:11:49.838270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:11:49.838270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:11:49.838270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:11:49.838270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:49.838270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:11:49.838270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:11:49.838270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:11:49.838270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:11:49.838270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:11:49.838270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:49.838942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:49.838942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:11:49.838942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:11:49.838942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:11:49.838942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:11:49.838942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:49.838942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:49.838942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:49.838942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:11:49.838942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:49.838942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:49.838942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:49.838942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:11:49.838942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:49.838942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:11:49.838942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:11:49.838942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:49.838942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:11:49.838942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:49.839683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:11:49.839683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:11:49.839683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:49.839683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:11:49.839683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:49.839683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:11:49.996911 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:11:49.997602 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:11:49.997810 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:11:49.998808 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:11:49.999029 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:11:49.999181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:49.999814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:11:49.999814 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:49.999814 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:11:49.999814 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:11:49.999814 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:49.999814 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:11:49.999814 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:49.999814 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:11:50.001540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:50.001540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:11:50.001540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:50.001540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:11:50.001540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:11:50.001540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:11:50.001540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:11:50.001540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:50.001540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:50.001540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:50.001540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:11:50.001540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:50.002219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:50.002219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:50.002219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:11:50.002219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:50.002219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:11:50.002219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:11:50.002219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:50.002219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:11:50.002219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:50.002219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:11:50.002219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:11:50.002219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:50.002219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:11:50.002834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:50.002834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:11:50.002834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:50.002834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:11:50.002834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:11:50.002834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:11:50.002834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:11:50.002834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:11:50.002834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:50.002834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:11:50.002834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:11:50.002834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:11:50.002834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:11:50.002834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:11:50.002834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:50.003560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:50.003560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:11:50.003560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:11:50.003560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:11:50.003560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:11:50.003560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:50.003560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:50.003560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:50.003560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:11:50.003560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:50.003560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:50.003560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:50.003560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:11:50.003560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:50.003560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:11:50.003560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:11:50.003560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:50.003560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:11:50.003560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:50.004419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:11:50.004419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:11:50.004419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:50.004419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:11:50.004419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:50.004419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:11:50.004840 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:11:50.005144 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:11:50.005260 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:11:50.005498 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:11:50.005820 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:11:50.006010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:50.006499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:11:50.006499 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:50.006499 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:11:50.006499 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:11:50.006499 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:50.006499 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:11:50.006499 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:50.006499 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:11:50.007367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:50.007367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:11:50.007367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:11:50.007367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:11:50.007367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:50.007367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:11:50.007367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:50.007367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:11:50.007367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:50.007367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:11:50.007367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:50.007367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:11:50.007367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:50.007367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:11:50.007367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:11:50.008174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:50.008174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:11:50.008174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:50.008174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:11:50.008174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:11:50.008174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:50.008174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:11:50.008174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:50.008174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:11:50.008645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:50.008645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:11:50.008645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:50.008645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:11:50.008645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:11:50.008645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:11:50.008645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:11:50.008645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:11:50.008645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:11:50.008645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:11:50.008645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:50.008645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:50.009287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:50.009287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:11:50.009287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:50.009287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:50.009287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:50.009287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:11:50.009287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:50.009287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:11:50.009287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:11:50.009287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:50.009287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:11:50.009287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:50.009287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:11:50.009287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:11:50.009738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:50.009738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:11:50.009738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:50.009738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:11:50.009738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:50.009738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:11:50.009738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:11:50.009738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:11:50.009738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:11:50.009738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:11:50.009738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:50.009738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:11:50.009738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:11:50.009738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:11:50.010028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:11:50.010028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:11:50.010028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:50.010028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:50.010028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:11:50.010028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:11:50.010028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:11:50.010028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:11:50.010028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:11:50.010028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:11:50.010028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:11:50.010028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:50.010028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:50.010028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:50.010028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:11:50.010028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:50.010028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:50.010028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:50.010028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:11:50.010589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:50.010589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:11:50.010589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:11:50.010589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:50.010589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:11:50.010589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:50.010589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:11:50.010589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:11:50.010589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:50.010589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:11:50.010589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:50.010589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:11:50.013057 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:11:50.013366 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:11:50.013505 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:11:50.013662 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:11:50.014006 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:11:50.014203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:50.014774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:11:50.014774 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:50.014774 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:11:50.014774 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:11:50.014774 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:50.014774 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:11:50.014774 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:50.014774 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:11:50.015712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:50.015712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:11:50.015712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:11:50.015712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:11:50.015712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:50.015712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:11:50.015712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:50.015712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:11:50.015712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:50.015712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:11:50.015712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:50.015712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:11:50.015712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:50.015712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:11:50.015712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:11:50.016035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:50.016035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:11:50.016035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:50.016035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:11:50.016035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:11:50.016035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:50.016035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:11:50.016035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:50.016035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:11:50.016708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:50.016708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:11:50.016708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:50.016708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:11:50.016708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:11:50.016708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:11:50.016708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:11:50.016708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:11:50.016708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:11:50.016708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:50.016708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:50.016984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:50.016984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:11:50.016984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:50.016984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:50.016984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:50.016984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:11:50.016984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:50.016984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:11:50.016984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:11:50.016984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:50.016984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:11:50.016984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:50.016984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:11:50.016984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:11:50.017316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:50.017316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:11:50.017316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:50.017316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:11:50.018092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:50.018092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:11:50.018092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:11:50.018092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:11:50.018092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:11:50.018092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:11:50.018092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:50.018092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:11:50.018092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:11:50.018092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:11:50.018092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:11:50.018092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:11:50.018092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:50.018092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:50.018092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:11:50.018092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:11:50.018092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:11:50.018638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:11:50.018638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:11:50.018638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:11:50.018638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:50.018638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:50.018638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:50.018638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:11:50.018638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:50.018638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:50.018638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:50.018638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:11:50.018638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:50.018638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:11:50.018638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:11:50.018638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:50.018638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:11:50.018638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:50.018638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:11:50.018638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:11:50.019174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:50.019174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:11:50.019174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:50.019174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:11:50.025869 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:11:50.026193 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:11:50.026337 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:11:50.026558 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:11:50.028426 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:11:50.028882 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:11:50.029017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:50.029888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:11:50.029888 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:50.029888 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:11:50.029888 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:11:50.029888 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:50.029888 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:11:50.029888 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:50.029888 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:11:50.031398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:50.031398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:11:50.031398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:50.031398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:11:50.031398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:11:50.031398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:11:50.031398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:11:50.031398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:50.031398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:50.031398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:50.031398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:11:50.031398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:50.032041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:50.032041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:50.032041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:11:50.032041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:50.032041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:11:50.032041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:11:50.032041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:50.032041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:11:50.032041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:50.032041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:11:50.032041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:11:50.032041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:50.032041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:11:50.032641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:50.032641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:11:50.033197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:50.033197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:11:50.033197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:11:50.033197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:11:50.033197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:11:50.033197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:11:50.033197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:50.033197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:11:50.033197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:11:50.033197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:11:50.033197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:11:50.033197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:11:50.033197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:50.033197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:50.033197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:11:50.033197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:11:50.033832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:11:50.033832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:11:50.033832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:50.033832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:50.033832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:50.033832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:11:50.033832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:50.033832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:50.033832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:50.033832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:11:50.033832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:50.033832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:11:50.033832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:11:50.033832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:50.033832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:11:50.033832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:50.033832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:11:50.033832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:11:50.033832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:50.034395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:11:50.034395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:50.034395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:11:54.790130 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:11:54.790768 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:11:54.790970 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:11:54.792227 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:11:54.821682 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:11:54.822067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:54.822805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:11:54.822805 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:54.822805 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:11:54.822805 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:11:54.822805 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:54.822805 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:11:54.822805 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:54.822805 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:11:54.823647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:54.823647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:11:54.823647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:54.823647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:11:54.823647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:11:54.823647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:11:54.823647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:11:54.823647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:54.823647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:54.823647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:54.823647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:11:54.823647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:54.824008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:54.824008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:54.824008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:11:54.824008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:54.824008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:11:54.824008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:11:54.824008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:54.824008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:11:54.824008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:54.824008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:11:54.824008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:11:54.824008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:54.824008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:11:54.824475 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:54.824475 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:11:54.824543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:54.824543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:11:54.824543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:11:54.824543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:11:54.824543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:11:54.824543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:11:54.824543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:54.824543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:11:54.824543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:11:54.824543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:11:54.824543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:11:54.824543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:11:54.824543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:54.824543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:54.824543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:11:54.824543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:11:54.824942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:11:54.824942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:11:54.824942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:54.824942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:54.824942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:54.824942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:11:54.824942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:54.824942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:54.824942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:54.824942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:11:54.824942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:54.824942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:11:54.824942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:11:54.824942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:54.824942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:11:54.824942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:54.824942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:11:54.824942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:11:54.824942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:54.825511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:11:54.825511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:54.825511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:11:54.997590 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:11:54.998002 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:11:54.998179 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:11:54.999261 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:11:54.999536 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:11:54.999719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:55.000424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:11:55.000424 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:55.000424 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:11:55.000424 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:11:55.000424 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:55.000424 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:11:55.000424 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:55.000424 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:11:55.002019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:55.002019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:11:55.002019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:55.002019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:11:55.002019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:11:55.002019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:11:55.002019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:11:55.002019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:55.002019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:55.002019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:55.002019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:11:55.002019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:55.002622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:55.002622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:55.002622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:11:55.002622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:55.002622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:11:55.002622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:11:55.002622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:55.002622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:11:55.002622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:55.002622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:11:55.002622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:11:55.002622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:55.002622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:11:55.003228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:55.003228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:11:55.004168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:55.004168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:11:55.004168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:11:55.004168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:11:55.004168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:11:55.004168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:11:55.004168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:55.004168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:11:55.004168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:11:55.004168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:11:55.004168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:11:55.004168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:11:55.004168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:55.004168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:55.004168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:11:55.004168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:11:55.004885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:11:55.004885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:11:55.004885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:55.004885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:55.004885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:55.004885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:11:55.004885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:55.004885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:55.004885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:55.004885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:11:55.004885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:55.004885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:11:55.004885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:11:55.004885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:55.004885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:11:55.004885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:55.004885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:11:55.004885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:11:55.004885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:55.005696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:11:55.005696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:55.005696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:11:55.005696 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:11:55.005696 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:11:55.005696 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:11:55.005974 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:11:55.006669 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:11:55.006669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:55.006942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:11:55.006942 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:55.006942 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:11:55.006942 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:11:55.006942 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:55.006942 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:11:55.006942 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:55.006942 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:11:55.008378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:55.008378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:11:55.008378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:11:55.008378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:11:55.008378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:55.008378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:11:55.008378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:55.008378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:11:55.008378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:55.008378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:11:55.008378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:55.008378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:11:55.008378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:55.008378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:11:55.008378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:11:55.009404 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:55.009404 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:11:55.009404 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:55.009404 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:11:55.009404 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:11:55.009404 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:55.009404 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:11:55.009404 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:55.009404 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:11:55.010058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:55.010058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:11:55.010058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:55.010058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:11:55.010058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:11:55.010058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:11:55.010058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:11:55.010058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:11:55.010058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:11:55.010058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:11:55.010058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:55.010058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:55.010654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:55.010654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:11:55.010654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:55.010654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:55.010654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:55.010654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:11:55.010654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:55.010654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:11:55.010654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:11:55.010654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:55.010654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:11:55.010654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:55.010654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:11:55.010654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:11:55.010993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:55.010993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:11:55.010993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:55.010993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:11:55.011698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:55.011698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:11:55.011698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:11:55.011698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:11:55.011698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:11:55.011698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:11:55.011698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:55.011698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:11:55.011698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:11:55.011698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:11:55.011698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:11:55.011698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:11:55.011698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:55.011698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:55.011698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:11:55.011698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:11:55.012106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:11:55.012106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:11:55.012106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:11:55.012106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:11:55.012106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:11:55.012106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:55.012106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:55.012106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:55.012106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:11:55.012106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:55.012106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:55.012106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:55.012106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:11:55.012106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:55.012106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:11:55.012106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:11:55.012106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:55.012106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:11:55.012570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:55.012570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:11:55.012570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:11:55.012570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:55.012570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:11:55.012570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:55.012570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:11:55.013522 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:11:55.013758 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:11:55.013869 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:11:55.013987 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:11:55.014477 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:11:55.014873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:55.015708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:11:55.015708 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:55.015708 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:11:55.015708 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:11:55.015708 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:55.015708 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:11:55.015708 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:55.015708 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:11:55.016448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:55.016448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:11:55.016448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:11:55.016448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:11:55.016448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:55.016448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:11:55.016448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:55.016448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:11:55.016448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:55.016448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:11:55.016448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:55.016448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:11:55.016448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:55.016448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:11:55.016448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:11:55.016819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:55.016819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:11:55.016819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:55.016819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:11:55.016819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:11:55.016819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:55.016819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:11:55.016819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:55.016819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:11:55.017281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:55.017281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:11:55.017281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:55.017281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:11:55.017281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:11:55.017281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:11:55.017281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:11:55.017281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:11:55.017281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:11:55.017281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:55.017281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:55.017587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:55.017587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:11:55.017587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:55.017587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:55.017587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:55.017587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:11:55.017587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:55.017587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:11:55.017587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:11:55.017587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:55.017587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:11:55.017587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:55.017587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:11:55.017587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:11:55.018082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:55.018082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:11:55.018082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:55.018082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:11:55.018519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:55.018519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:11:55.018519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:11:55.018519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:11:55.018519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:11:55.018519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:11:55.018519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:55.018519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:11:55.018519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:11:55.018519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:11:55.018519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:11:55.018519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:11:55.018519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:55.018519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:55.018519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:11:55.018519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:11:55.018519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:11:55.019642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:11:55.019642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:11:55.019642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:11:55.019642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:55.019642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:55.019642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:55.019642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:11:55.019642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:55.019642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:55.019642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:55.019642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:11:55.019642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:55.019642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:11:55.019642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:11:55.019642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:55.019642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:11:55.019642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:55.019642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:11:55.019642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:11:55.020267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:55.020267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:11:55.020267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:55.020267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:11:55.026668 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:11:55.026911 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:11:55.027014 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:11:55.027160 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:11:55.027627 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:11:55.027885 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:11:55.028115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:55.029100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:11:55.029100 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:55.029100 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:11:55.029100 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:11:55.029100 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:55.029100 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:11:55.029100 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:55.029100 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:11:55.030489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:55.030489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:11:55.030489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:55.030489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:11:55.030489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:11:55.030489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:11:55.030489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:11:55.030489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:55.030489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:55.030489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:55.030489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:11:55.030489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:55.030995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:55.030995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:55.030995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:11:55.030995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:55.030995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:11:55.030995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:11:55.030995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:55.030995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:11:55.030995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:55.030995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:11:55.030995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:11:55.030995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:55.030995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:11:55.031570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:55.031570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:11:55.032051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:55.032051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:11:55.032051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:11:55.032051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:11:55.032051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:11:55.032051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:11:55.032051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:55.032051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:11:55.032051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:11:55.032051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:11:55.032051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:11:55.032051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:11:55.032051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:55.032051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:55.032051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:11:55.032051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:11:55.032705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:11:55.032705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:11:55.032705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:55.032705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:55.032705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:55.032705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:11:55.032705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:55.032705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:55.032705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:55.032705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:11:55.032705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:55.032705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:11:55.032705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:11:55.032705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:55.032705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:11:55.032705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:55.032705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:11:55.032705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:11:55.032705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:55.033451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:11:55.033451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:55.033451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:11:59.790864 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:11:59.791574 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:11:59.791574 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:11:59.792829 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:11:59.826915 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:11:59.827082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:59.827551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:11:59.827551 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:59.827551 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:11:59.827551 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:11:59.827551 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:59.827551 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:11:59.827551 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:59.827551 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:11:59.828204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:59.828204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:11:59.828204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:59.828204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:11:59.828204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:11:59.828204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:11:59.828204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:11:59.828204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:59.828204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:59.828204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:59.828204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:11:59.828204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:59.828849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:11:59.828849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:59.828849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:11:59.828849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:59.828849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:11:59.828849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:11:59.828849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:59.828849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:11:59.828849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:59.828849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:11:59.828849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:11:59.828849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:59.828849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:11:59.829624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:59.829624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:11:59.829624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:59.829624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:11:59.829624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:11:59.829624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:11:59.829624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:11:59.829624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:11:59.829624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:59.829624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:11:59.829624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:11:59.829624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:11:59.829624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:11:59.829624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:11:59.829624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:11:59.830408 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:11:59.830408 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:11:59.830408 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:11:59.830408 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:11:59.830408 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:11:59.830408 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:59.830408 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:59.830408 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:11:59.830408 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:11:59.830408 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:11:59.830408 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:11:59.830408 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:11:59.830408 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:11:59.830408 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:11:59.830408 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:11:59.830408 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:11:59.830408 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:11:59.830408 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:11:59.830408 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:11:59.831263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:11:59.831263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:11:59.831263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:11:59.831263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:11:59.831263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:11:59.831263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:11:59.998378 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:11:59.998854 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:11:59.999077 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:12:00.000164 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:12:00.000509 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:12:00.000736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:00.002098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:12:00.002098 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:00.002098 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:12:00.002098 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:12:00.002098 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:00.002098 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:12:00.002098 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:00.002098 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:12:00.003641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:00.003641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:12:00.003641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:00.003641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:12:00.003641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:12:00.003641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:12:00.003641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:12:00.003641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:00.003641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:00.003641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:00.003641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:12:00.003641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:00.004463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:00.004463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:00.004463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:12:00.004463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:00.004463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:12:00.004463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:12:00.004463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:00.004463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:12:00.004463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:00.004463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:12:00.004463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:12:00.004463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:00.004463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:12:00.005082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:00.005082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:12:00.005861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:00.005861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:12:00.005861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:12:00.005861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:12:00.005861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:12:00.005861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:12:00.005861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:00.005861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:12:00.005861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:12:00.005861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:12:00.005861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:12:00.005861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:12:00.005861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:00.005861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:00.005861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:12:00.005861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:12:00.008056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:12:00.008056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:12:00.008056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:00.008056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:00.008056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:00.008056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:12:00.008056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:00.008056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:00.008056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:00.008056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:12:00.008056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:00.008056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:12:00.008056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:12:00.008056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:00.008056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:12:00.008056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:00.008056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:12:00.008056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:12:00.008056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:00.008785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:12:00.008785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:00.008785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:12:00.008785 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:12:00.008785 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:12:00.008785 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:12:00.008785 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:12:00.008785 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:12:00.009058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:00.009058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:12:00.009058 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:00.009058 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:12:00.009058 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:12:00.009058 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:00.009058 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:12:00.009058 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:00.009058 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:12:00.009058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:00.009058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:12:00.009058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:12:00.009058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:12:00.009058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:00.009058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:12:00.009058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:00.009058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:12:00.009704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:00.009704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:12:00.009704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:00.009704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:12:00.009704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:00.009704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:12:00.009704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:12:00.009704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:00.009704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:12:00.009704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:00.009704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:12:00.009704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:12:00.009704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:00.009704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:12:00.009704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:00.009704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:12:00.010357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:00.010357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:12:00.010357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:00.010357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:12:00.010357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:12:00.010357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:12:00.010357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:12:00.010357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:12:00.010357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:12:00.010357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:12:00.010357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:00.010357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:00.010786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:00.010786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:12:00.010786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:00.010786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:00.010786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:00.010786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:12:00.010786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:00.010786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:12:00.010786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:12:00.010786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:00.010786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:12:00.010786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:00.010786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:12:00.010786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:12:00.011157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:00.011157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:12:00.011157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:00.011157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:12:00.011157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:00.011157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:12:00.011157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:12:00.011157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:12:00.011157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:12:00.011157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:12:00.011157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:00.011157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:12:00.011157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:12:00.011157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:12:00.011801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:12:00.011801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:12:00.011801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:00.011801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:00.011801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:12:00.011801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:12:00.011801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:12:00.011801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:12:00.011801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:12:00.011801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:12:00.011801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:12:00.011801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:00.011801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:00.011801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:00.011801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:12:00.011801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:00.011801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:00.011801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:00.011801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:12:00.012420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:00.012420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:12:00.012420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:12:00.012420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:00.012420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:12:00.012420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:00.012420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:12:00.012420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:12:00.012420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:00.012420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:12:00.012420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:00.012420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:12:00.013902 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:12:00.014076 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:12:00.014152 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:12:00.014246 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:12:00.014515 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:12:00.014776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:00.015072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:12:00.015072 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:00.015072 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:12:00.015072 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:12:00.015072 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:00.015072 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:12:00.015072 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:00.015072 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:12:00.015643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:00.015643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:12:00.015643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:12:00.015643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:12:00.015643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:00.015643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:12:00.015643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:00.015643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:12:00.015643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:00.015643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:12:00.015643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:00.015643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:12:00.015643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:00.015643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:12:00.015643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:12:00.016002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:00.016002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:12:00.016002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:00.016002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:12:00.016002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:12:00.016002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:00.016002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:12:00.016002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:00.016002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:12:00.016351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:00.016351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:12:00.016351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:00.016351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:12:00.016351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:12:00.016351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:12:00.016351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:12:00.016351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:12:00.016351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:12:00.016351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:00.016351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:00.016694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:00.016694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:12:00.016694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:00.016694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:00.016694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:00.016694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:12:00.016694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:00.016694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:12:00.016694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:12:00.016694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:00.016694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:12:00.016694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:00.016694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:12:00.016694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:12:00.016950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:00.016950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:12:00.016950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:00.016950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:12:00.017384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:00.017384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:12:00.017384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:12:00.017384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:12:00.017384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:12:00.017384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:12:00.017384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:00.017384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:12:00.017384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:12:00.017384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:12:00.017384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:12:00.017384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:12:00.017384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:00.017384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:00.017384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:12:00.017384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:12:00.017384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:12:00.017739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:12:00.017739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:12:00.017739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:12:00.017739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:00.017739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:00.017739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:00.017739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:12:00.017739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:00.017739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:00.017739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:00.017739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:12:00.017739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:00.017739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:12:00.017739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:12:00.017739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:00.017739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:12:00.017739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:00.017739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:12:00.017739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:12:00.018109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:00.018109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:12:00.018109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:00.018109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:12:00.027297 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:12:00.027534 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:12:00.027768 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:12:00.028030 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:12:00.028399 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:12:00.028834 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:12:00.029066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:00.029905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:12:00.029905 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:00.029905 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:12:00.029905 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:12:00.029905 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:00.029905 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:12:00.029905 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:00.029905 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:12:00.031270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:00.031270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:12:00.031270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:00.031270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:12:00.031270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:12:00.031270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:12:00.031270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:12:00.031270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:00.031270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:00.031270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:00.031270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:12:00.031270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:00.031976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:00.031976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:00.031976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:12:00.031976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:00.031976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:12:00.031976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:12:00.031976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:00.031976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:12:00.031976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:00.031976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:12:00.031976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:12:00.031976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:00.031976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:12:00.032724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:00.032724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:12:00.032724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:00.032724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:12:00.032724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:12:00.032724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:12:00.032724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:12:00.032724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:12:00.032724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:00.032724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:12:00.032724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:12:00.032724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:12:00.032724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:12:00.032724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:12:00.032724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:00.033483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:00.033483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:12:00.033483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:12:00.033483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:12:00.033483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:12:00.033483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:00.033483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:00.033483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:00.033483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:12:00.033483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:00.033483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:00.033483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:00.033483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:12:00.033483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:00.033483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:12:00.033483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:12:00.033483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:00.033483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:12:00.033483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:00.034341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:12:00.034341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:12:00.034341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:00.034341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:12:00.034341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:00.034341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:12:04.791456 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:12:04.792324 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:12:04.792324 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:12:04.793490 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:12:04.825656 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:12:04.825866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:04.826231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:12:04.826231 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:04.826231 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:12:04.826231 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:12:04.826231 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:04.826231 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:12:04.826231 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:04.826231 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:12:04.826924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:04.826924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:12:04.826924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:04.826924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:12:04.826924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:12:04.826924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:12:04.826924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:12:04.826924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:04.826924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:04.826924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:04.826924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:12:04.826924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:04.827524 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:04.827524 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:04.827524 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:12:04.827524 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:04.827524 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:12:04.827524 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:12:04.827524 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:04.827524 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:12:04.827524 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:04.827524 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:12:04.827524 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:12:04.827524 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:04.827524 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:12:04.828082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:04.828082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:12:04.828082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:04.828082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:12:04.828082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:12:04.828082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:12:04.828082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:12:04.828082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:12:04.828082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:04.828082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:12:04.828082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:12:04.828082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:12:04.828082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:12:04.828082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:12:04.828082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:04.828766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:04.828766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:12:04.828766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:12:04.828766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:12:04.828766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:12:04.828766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:04.828766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:04.828766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:04.828766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:12:04.828766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:04.828766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:04.828766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:04.828766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:12:04.828766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:04.828766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:12:04.828766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:12:04.828766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:04.828766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:12:04.828766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:04.829558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:12:04.829558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:12:04.829558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:04.829558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:12:04.829558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:04.829558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:12:04.998853 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:12:04.999432 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:12:04.999647 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:12:05.000669 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:12:05.000984 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:12:05.001226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:05.002226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:12:05.002226 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:05.002226 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:12:05.002226 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:12:05.002226 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:05.002226 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:12:05.002226 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:05.002226 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:12:05.003724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:05.003724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:12:05.003724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:05.003724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:12:05.003724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:12:05.003724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:12:05.003724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:12:05.003724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:05.003724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:05.003724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:05.003724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:12:05.003724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:05.004262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:05.004262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:05.004262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:12:05.004262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:05.004262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:12:05.004262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:12:05.004262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:05.004262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:12:05.004262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:05.004262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:12:05.004262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:12:05.004262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:05.004262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:12:05.005123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:05.005123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:12:05.005565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:05.005565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:12:05.005565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:12:05.005565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:12:05.005565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:12:05.005565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:12:05.005565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:05.005565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:12:05.005565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:12:05.005565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:12:05.005565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:12:05.005565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:12:05.005565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:05.005565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:05.005565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:12:05.005565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:12:05.006185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:12:05.006185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:12:05.006185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:05.006185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:05.006185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:05.006185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:12:05.006185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:05.006185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:05.006185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:05.006185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:12:05.006185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:05.006185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:12:05.006185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:12:05.006185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:05.006185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:12:05.006185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:05.006185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:12:05.006185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:12:05.006185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:05.006742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:12:05.006742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:05.006742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:12:05.006981 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:12:05.007238 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:12:05.007345 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:12:05.007468 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:12:05.007877 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:12:05.008086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:05.008898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:12:05.008898 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:05.008898 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:12:05.008898 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:12:05.008898 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:05.008898 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:12:05.008898 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:05.008898 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:12:05.009608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:05.009608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:12:05.009608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:12:05.009608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:12:05.009608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:05.009608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:12:05.009608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:05.009608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:12:05.009608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:05.009608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:12:05.009608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:05.009608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:12:05.009608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:05.009608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:12:05.009608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:12:05.010014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:05.010014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:12:05.010014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:05.010014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:12:05.010014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:12:05.010014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:05.010014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:12:05.010014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:05.010014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:12:05.010450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:05.010450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:12:05.010450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:05.010450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:12:05.010450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:12:05.010450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:12:05.010450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:12:05.010450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:12:05.010450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:12:05.010450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:12:05.010450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:05.010450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:05.010829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:05.010829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:12:05.010829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:05.010829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:05.010829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:05.010829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:12:05.010829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:05.010829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:12:05.010829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:12:05.010829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:05.010829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:12:05.010829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:05.010829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:12:05.010829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:12:05.011254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:05.011254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:12:05.011254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:05.011254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:12:05.012203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:05.012203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:12:05.012203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:12:05.012203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:12:05.012203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:12:05.012203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:12:05.012203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:05.012203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:12:05.012203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:12:05.012203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:12:05.012203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:12:05.012203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:12:05.012203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:05.012203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:05.012203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:12:05.012203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:12:05.012817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:12:05.012817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:12:05.012817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:12:05.012817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:12:05.012817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:12:05.012817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:05.012817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:05.012817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:05.012817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:12:05.012817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:05.012817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:05.012817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:05.012817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:12:05.012817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:05.012817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:12:05.012817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:12:05.012817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:05.012817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:12:05.013326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:05.013326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:12:05.013326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:12:05.013326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:05.013326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:12:05.013326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:05.013326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:12:05.014982 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:12:05.015224 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:12:05.015296 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:12:05.015352 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:12:05.015559 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:12:05.015697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:05.016185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:12:05.016185 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:05.016185 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:12:05.016185 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:12:05.016185 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:05.016185 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:12:05.016185 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:05.016185 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:12:05.016673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:05.016673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:12:05.016673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:12:05.016673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:12:05.016673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:05.016673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:12:05.016673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:05.016673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:12:05.016673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:05.016673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:12:05.016673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:05.016673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:12:05.016673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:05.016673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:12:05.016673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:12:05.017077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:05.017077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:12:05.017077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:05.017077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:12:05.017077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:12:05.017077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:05.017077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:12:05.017077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:05.017077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:12:05.017310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:05.017310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:12:05.017310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:05.017310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:12:05.017310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:12:05.017310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:12:05.017310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:12:05.017310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:12:05.017310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:12:05.017310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:05.017310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:05.018251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:05.018251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:12:05.018251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:05.018251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:05.018251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:05.018251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:12:05.018251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:05.018251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:12:05.018251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:12:05.018251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:05.018251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:12:05.018251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:05.018251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:12:05.018251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:12:05.018936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:05.018936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:12:05.018936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:05.018936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:12:05.018936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:05.018936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:12:05.018936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:12:05.018936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:12:05.018936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:12:05.018936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:12:05.018936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:05.018936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:12:05.018936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:12:05.018936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:12:05.019449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:12:05.019449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:12:05.019449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:05.019449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:05.019449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:12:05.019449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:12:05.019449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:12:05.019449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:12:05.019449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:12:05.019449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:12:05.019449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:05.019449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:05.019449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:05.019449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:12:05.019449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:05.019449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:05.019449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:05.019449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:12:05.019449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:05.019983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:12:05.019983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:12:05.019983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:05.019983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:12:05.019983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:05.019983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:12:05.019983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:12:05.019983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:05.019983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:12:05.019983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:05.019983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:12:05.027510 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:12:05.027740 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:12:05.027925 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:12:05.028126 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:12:05.028522 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:12:05.028755 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:12:05.028902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:05.030083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:12:05.030083 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:05.030083 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:12:05.030083 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:12:05.030083 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:05.030083 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:12:05.030083 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:05.030083 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:12:05.031669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:05.031669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:12:05.031669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:05.031669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:12:05.031669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:12:05.031669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:12:05.031669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:12:05.031669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:05.031669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:05.031669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:05.031669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:12:05.031669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:05.032094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:05.032094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:05.032094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:12:05.032094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:05.032094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:12:05.032094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:12:05.032094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:05.032094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:12:05.032094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:05.032094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:12:05.032094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:12:05.032094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:05.032094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:12:05.032495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:05.032495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:12:05.033373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:05.033373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:12:05.033373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:12:05.033373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:12:05.033373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:12:05.033373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:12:05.033373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:05.033373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:12:05.033373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:12:05.033373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:12:05.033373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:12:05.033373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:12:05.033373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:05.033373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:05.033373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:12:05.033373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:12:05.033868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:12:05.033868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:12:05.033868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:05.033868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:05.033868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:05.033868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:12:05.033868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:05.033868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:05.033868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:05.033868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:12:05.033868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:05.033868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:12:05.033868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:12:05.033868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:05.033868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:12:05.033868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:05.033868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:12:05.033868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:12:05.033868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:05.034396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:12:05.034396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:05.034396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:12:09.792451 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:12:09.794471 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:12:09.794584 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:12:09.795461 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:12:09.834127 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:12:09.834334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:09.835067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:12:09.835067 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:09.835067 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:12:09.835067 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:12:09.835067 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:09.835067 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:12:09.835067 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:09.835067 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:12:09.836276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:09.836276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:12:09.836276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:09.836276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:12:09.836276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:12:09.836276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:12:09.836276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:12:09.836276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:09.836276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:09.836276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:09.836276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:12:09.836276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:09.837250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:09.837250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:09.837250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:12:09.837250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:09.837250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:12:09.837250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:12:09.837250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:09.837250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:12:09.837250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:09.837250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:12:09.837250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:12:09.837250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:09.837250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:12:09.837525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:09.837525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:12:09.837525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:09.837525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:12:09.837525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:12:09.837525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:12:09.837525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:12:09.837525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:12:09.837525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:09.837525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:12:09.837525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:12:09.837525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:12:09.837525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:12:09.837525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:12:09.837525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:09.837832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:09.837832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:12:09.837832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:12:09.837832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:12:09.837832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:12:09.837832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:09.837832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:09.837832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:09.837832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:12:09.837832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:09.837832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:09.837832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:09.837832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:12:09.837832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:09.837832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:12:09.837832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:12:09.837832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:09.837832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:12:09.837832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:09.838284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:12:09.838284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:12:09.838284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:09.838284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:12:09.838284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:09.838284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:12:09.999458 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:12:10.000332 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:12:10.000607 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:12:10.001796 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:12:10.002023 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:12:10.002155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:10.002860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:12:10.002860 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:10.002860 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:12:10.002860 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:12:10.002860 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:10.002860 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:12:10.002860 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:10.002860 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:12:10.004457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:10.004457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:12:10.004457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:10.004457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:12:10.004457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:12:10.004457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:12:10.004457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:12:10.004457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:10.004457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:10.004457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:10.004457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:12:10.004457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:10.004767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:10.004767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:10.004767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:12:10.004767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:10.004767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:12:10.004767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:12:10.004767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:10.004767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:12:10.004767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:10.004767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:12:10.004767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:12:10.004767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:10.004767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:12:10.005020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:10.005020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:12:10.006250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:10.006250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:12:10.006250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:12:10.006250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:12:10.006250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:12:10.006250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:12:10.006250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:10.006250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:12:10.006250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:12:10.006250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:12:10.006250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:12:10.006250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:12:10.006250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:10.006250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:10.006250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:12:10.006250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:12:10.006599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:12:10.006599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:12:10.006599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:10.006599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:10.006599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:10.006599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:12:10.006599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:10.006599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:10.006599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:10.006599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:12:10.006599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:10.006599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:12:10.006599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:12:10.006599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:10.006599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:12:10.006599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:10.006599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:12:10.006599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:12:10.006599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:10.006957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:12:10.006957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:10.006957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:12:10.007365 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:12:10.007687 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:12:10.007794 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:12:10.007915 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:12:10.008273 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:12:10.008453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:10.008848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:12:10.008848 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:10.008848 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:12:10.008848 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:12:10.008848 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:10.008848 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:12:10.008848 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:10.008848 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:12:10.009588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:10.009588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:12:10.009588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:12:10.009588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:12:10.009588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:10.009588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:12:10.009588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:10.009588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:12:10.009588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:10.009588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:12:10.009588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:10.009588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:12:10.009588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:10.009588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:12:10.009588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:12:10.009935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:10.009935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:12:10.009935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:10.009935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:12:10.009935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:12:10.009935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:10.009935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:12:10.009935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:10.009935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:12:10.010429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:10.010429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:12:10.010429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:10.010429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:12:10.010429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:12:10.010429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:12:10.010429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:12:10.010429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:12:10.010429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:12:10.010429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:12:10.010429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:10.010429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:10.010757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:10.010757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:12:10.010757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:10.010757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:10.010757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:10.010757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:12:10.010757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:10.010757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:12:10.010757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:12:10.010757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:10.010757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:12:10.010757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:10.010757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:12:10.010757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:12:10.011139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:10.011139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:12:10.011139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:10.011139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:12:10.011483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:10.011483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:12:10.011483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:12:10.011483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:12:10.011483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:12:10.011483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:12:10.011483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:10.011483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:12:10.011483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:12:10.011483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:12:10.011483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:12:10.011483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:12:10.011483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:10.011483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:10.011483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:12:10.011483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:12:10.011926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:12:10.011926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:12:10.011926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:12:10.011926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:12:10.011926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:12:10.011926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:10.011926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:10.011926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:10.011926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:12:10.011926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:10.011926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:10.011926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:10.011926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:12:10.011926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:10.011926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:12:10.011926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:12:10.011926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:10.011926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:12:10.012360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:10.012360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:12:10.012360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:12:10.012360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:10.012360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:12:10.012360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:10.012360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:12:10.015427 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:12:10.015841 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:12:10.015948 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:12:10.016081 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:12:10.016356 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:12:10.016537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:10.016922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:12:10.016922 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:10.016922 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:12:10.016922 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:12:10.016922 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:10.016922 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:12:10.016922 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:10.016922 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:12:10.017806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:10.017806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:12:10.017806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:12:10.017806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:12:10.017806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:10.017806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:12:10.017806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:10.017806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:12:10.017806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:10.017806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:12:10.017806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:10.017806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:12:10.017806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:10.017806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:12:10.017806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:12:10.018241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:10.018241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:12:10.018241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:10.018241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:12:10.018241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:12:10.018241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:10.018241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:12:10.018241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:10.018241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:12:10.019106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:10.019106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:12:10.019106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:10.019106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:12:10.019106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:12:10.019106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:12:10.019106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:12:10.019106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:12:10.019106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:12:10.019106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:10.019106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:10.019467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:10.019467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:12:10.019467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:10.019467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:10.019467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:10.019467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:12:10.019467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:10.019467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:12:10.019467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:12:10.019467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:10.019467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:12:10.019467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:10.019467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:12:10.019467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:12:10.019974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:10.019974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:12:10.019974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:10.019974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:12:10.019974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:10.019974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:12:10.019974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:12:10.019974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:12:10.019974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:12:10.019974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:12:10.019974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:10.019974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:12:10.019974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:12:10.019974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:12:10.020515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:12:10.020515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:12:10.020515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:10.020515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:10.020515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:12:10.020515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:12:10.020515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:12:10.020515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:12:10.020515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:12:10.020515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:12:10.020515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:10.020515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:10.020515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:10.020515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:12:10.020515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:10.020515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:10.020515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:10.020515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:12:10.020515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:10.021063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:12:10.021063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:12:10.021063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:10.021063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:12:10.021063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:10.021063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:12:10.021063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:12:10.021063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:10.021063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:12:10.021063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:10.021063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:12:10.027938 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:12:10.028541 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:12:10.028745 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:12:10.029047 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:12:10.029278 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:12:10.029450 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:12:10.029550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:10.031245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:12:10.031245 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:10.031245 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:12:10.031245 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:12:10.031245 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:10.031245 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:12:10.031245 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:10.031245 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:12:10.031245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:10.031245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:12:10.031245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:10.031245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:12:10.031245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:12:10.031245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:12:10.031245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:12:10.031585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:10.031585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:10.031585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:10.031585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:12:10.031585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:10.031585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:10.031585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:10.031585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:12:10.031585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:10.031585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:12:10.031585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:12:10.031585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:10.031585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:12:10.031585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:10.031848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:12:10.031848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:12:10.031848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:10.031848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:12:10.031848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:10.031848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:12:10.032121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:10.032121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:12:10.032121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:12:10.032121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:12:10.032121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:12:10.032121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:12:10.032121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:10.032121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:12:10.032121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:12:10.032121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:12:10.032121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:12:10.032121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:12:10.032121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:10.032121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:10.032121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:12:10.032121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:12:10.032458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:12:10.032458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:12:10.032458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:10.032458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:10.032458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:10.032458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:12:10.032458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:10.032458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:10.032458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:10.032458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:12:10.032458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:10.032458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:12:10.032458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:12:10.032458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:10.032458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:12:10.032458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:10.032458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:12:10.032458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:12:10.032458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:10.032792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:12:10.032792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:10.032792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:12:14.794439 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:12:14.795689 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:12:14.795689 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:12:14.797318 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:12:14.841896 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:12:14.842317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:14.843282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:12:14.843282 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:14.843282 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:12:14.843282 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:12:14.843282 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:14.843282 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:12:14.843282 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:14.843282 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:12:14.844147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.048 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:14.844147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:12:14.844147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:14.844147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:12:14.844147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:12:14.844147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:12:14.844147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:12:14.844147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:14.844147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:14.844147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:14.844147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:12:14.844147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:14.844865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:14.844865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:14.844865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:12:14.844865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:14.844865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:12:14.844865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:12:14.844865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:14.844865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:12:14.844865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:14.844865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:12:14.844865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:12:14.844865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:14.844865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:12:14.845518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:14.845518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:12:14.845518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:14.845518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:12:14.845518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:12:14.845518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:12:14.845518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:12:14.845518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:12:14.845518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:14.845518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:12:14.845518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:12:14.845518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:12:14.845518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:12:14.845518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:12:14.845518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:14.846246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:14.846246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:12:14.846246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:12:14.846246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:12:14.846246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:12:14.846246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:14.846246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:14.846246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:14.846246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:12:14.846246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:14.846246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:14.846246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:14.846246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:12:14.846246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:14.846246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:12:14.846246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:12:14.846246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:14.846246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:12:14.846246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:14.847097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:12:14.847097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:12:14.847097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:14.847097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:12:14.847097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:14.847097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:12:15.000154 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:12:15.033791 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:12:15.033929 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:12:15.034203 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:12:15.037740 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:12:15.037919 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:12:15.038230 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:12:15.038325 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:12:15.038537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:15.039365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:12:15.039365 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:15.039365 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:12:15.039365 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:12:15.039365 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:15.039365 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:12:15.039365 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:15.039365 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:12:15.041469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:15.041469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:12:15.041469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:15.041469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:12:15.041469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:12:15.041469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:12:15.041469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:12:15.041469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:15.041469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:15.041469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:15.041469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:12:15.041469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:15.041846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:15.041846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:15.041846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:12:15.041846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:15.041846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:12:15.041846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:12:15.041846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:15.041846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:12:15.041846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:15.041846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:12:15.041846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:12:15.041846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:15.041846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:12:15.042356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:15.042356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:12:15.043348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:15.043348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:12:15.043348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:12:15.043348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:12:15.043348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:12:15.043348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:12:15.043348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:15.043348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:12:15.043348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:12:15.043348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:12:15.043348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:12:15.043348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:12:15.043348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:15.043348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:15.043348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:12:15.043348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:12:15.044035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:12:15.044035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:12:15.044035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:15.044035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:15.044035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:15.044035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:12:15.044035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:15.044035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:15.044035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:15.044035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:12:15.044035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:15.044035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:12:15.044035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:12:15.044035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:15.044035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:12:15.044035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:15.044035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:12:15.044035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:12:15.044035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:15.044850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:12:15.044850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:15.044850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:12:15.044850 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:12:15.044850 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:12:15.044850 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:12:15.045202 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:12:15.045420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:15.045975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:12:15.045975 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:15.045975 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:12:15.045975 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:12:15.045975 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:15.045975 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:12:15.045975 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:15.045975 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:12:15.046838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:15.046838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:12:15.046838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:12:15.046838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:12:15.046838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:15.046838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:12:15.046838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:15.046838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:12:15.046838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:15.046838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:12:15.046838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:15.046838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:12:15.046838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:15.046838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:12:15.046838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:12:15.047627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:15.047627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:12:15.047627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:15.047627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:12:15.047627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:12:15.047627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:15.047627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:12:15.047627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:15.047627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:12:15.048021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:15.048021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:12:15.048021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:15.048021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:12:15.048021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:12:15.048021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:12:15.048021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:12:15.048021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:12:15.048021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:12:15.048021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:12:15.048021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:15.048021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:15.048679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:15.048679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:12:15.048679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:15.048679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:15.048679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:15.048679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:12:15.048679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:15.048679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:12:15.048679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:12:15.048679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:15.048679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:12:15.048679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:15.048679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:12:15.048679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:12:15.049220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:15.049220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:12:15.049220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:15.049220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:12:15.049426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:15.049426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:12:15.049426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:12:15.049426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:12:15.049426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:12:15.049426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:12:15.049426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:15.049426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:12:15.049426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:12:15.049426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:12:15.049426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:12:15.049426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:12:15.049426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:15.049426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:15.049426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:12:15.049426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:12:15.050296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:12:15.050296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:12:15.050296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:12:15.050296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:12:15.050296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:12:15.050296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:15.050296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:15.050296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:15.050296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:12:15.050296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:15.050296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:15.050296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:15.050296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:12:15.050296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:15.050296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:12:15.050296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:12:15.050296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:15.050296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:12:15.051888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:15.051888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:12:15.051888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:12:15.051888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:15.051888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:12:15.051888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:15.051888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:12:15.051888 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:12:15.051888 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:12:15.051888 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:12:15.052354 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:12:15.052354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:15.052354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:12:15.052354 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:15.052354 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:12:15.052354 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:12:15.052354 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:15.052354 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:12:15.052354 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:15.052354 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:12:15.052354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:15.052354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:12:15.052354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:12:15.052354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:12:15.052882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:15.052882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:12:15.052882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:15.052882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:12:15.052882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:15.052882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:12:15.052882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:15.052882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:12:15.052882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:15.052882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:12:15.052882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:12:15.052882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:15.052882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:12:15.052882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:15.052882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:12:15.052882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:12:15.052882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:15.052882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:12:15.053374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:15.053374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:12:15.053374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:15.053374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:12:15.053374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:15.053374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:12:15.053374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:12:15.053374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:12:15.053374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:12:15.053374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:12:15.053374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:12:15.053374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:15.053766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:15.053766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:15.053766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:12:15.053766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:15.053766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:15.053766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:15.053766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:12:15.053766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:15.053766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:12:15.053766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:12:15.053766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:15.053766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:12:15.053766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:15.053766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:12:15.054152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:12:15.054152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:15.054152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:12:15.054152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:15.054152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:12:15.054152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:15.054152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:12:15.054152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:12:15.054152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:12:15.054152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:12:15.054152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:12:15.054152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:15.054152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:12:15.054152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:12:15.054152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:12:15.054582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:12:15.054582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:12:15.054582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:15.054582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:15.054582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:12:15.054582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:12:15.054582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:12:15.054582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:12:15.054582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:12:15.054582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:12:15.054582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:15.054582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:15.054582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:15.054582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:12:15.054582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:15.054582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:15.054582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:15.054582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:12:15.054582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:15.055111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:12:15.055111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:12:15.055111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:15.055111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:12:15.055111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:15.055111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:12:15.055111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:12:15.055111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:15.055111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:12:15.055111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:15.055111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:12:15.055111 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:12:15.055111 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:12:15.055545 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:12:15.055545 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:12:15.055545 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:12:15.055545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:15.055545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:12:15.055545 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:15.055545 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:12:15.055545 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:12:15.055545 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:15.055545 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:12:15.055545 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:15.055545 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:12:15.056083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:15.056083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:12:15.056083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:15.056083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:12:15.056083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:12:15.056083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:12:15.056083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:12:15.056083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:15.056083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:15.056083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:15.056083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:12:15.056083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:15.056553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:15.056553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:15.056553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:12:15.056553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:15.056553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:12:15.056553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:12:15.056553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:15.056553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:12:15.056553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:15.056553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:12:15.056553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:12:15.056553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:15.056553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:12:15.056894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:15.056894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:12:15.057057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:15.057057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:12:15.057057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:12:15.057057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:12:15.057057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:12:15.057057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:12:15.057057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:15.057057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:12:15.057057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:12:15.057057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:12:15.057057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:12:15.057057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:12:15.057057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:15.057057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:15.057057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:12:15.057057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:12:15.057573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:12:15.057573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:12:15.057573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:15.057573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:15.057573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:15.057573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:12:15.057573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:15.057573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:15.057573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:15.057573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:12:15.057573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:15.057573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:12:15.057573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:12:15.057573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:15.057573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:12:15.057573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:15.057573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:12:15.057573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:12:15.057573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:15.058297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:12:15.058297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:15.058297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:12:19.795326 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:12:19.796145 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:12:19.796145 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:12:19.796861 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:12:19.832597 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:12:19.833015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:19.833882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:12:19.833882 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:19.833882 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:12:19.833882 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:12:19.833882 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:19.833882 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:12:19.833882 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:19.833882 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:12:19.834920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:19.834920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:12:19.834920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:19.834920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:12:19.834920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:12:19.834920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:12:19.834920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:12:19.834920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:19.834920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:19.834920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:19.834920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:12:19.834920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:19.835480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:19.835480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:19.835480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:12:19.835480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:19.835480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:12:19.835480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:12:19.835480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:19.835480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:12:19.835480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:19.835480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:12:19.835480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:12:19.835480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:19.835480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:12:19.835977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:19.835977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:12:19.835977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:19.835977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:12:19.835977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:12:19.835977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:12:19.835977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:12:19.835977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:12:19.835977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:19.835977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:12:19.835977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:12:19.835977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:12:19.835977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:12:19.835977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:12:19.835977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:19.836643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:19.836643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:12:19.836643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:12:19.836643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:12:19.836643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:12:19.836643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:19.836643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:19.836643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:19.836643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:12:19.836643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:19.836643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:19.836643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:19.836643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:12:19.836643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:19.836643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:12:19.836643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:12:19.836643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:19.836643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:12:19.836643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:19.837450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:12:19.837450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:12:19.837450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:19.837450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:12:19.837450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:19.837450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:12:20.033653 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:12:20.034188 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:12:20.034305 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:12:20.035155 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:12:20.035391 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:12:20.035571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:20.036186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:12:20.036186 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:20.036186 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:12:20.036186 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:12:20.036186 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:20.036186 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:12:20.036186 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:20.036186 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:12:20.037190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:20.037190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:12:20.037190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:20.037190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:12:20.037190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:12:20.037190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:12:20.037190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:12:20.037190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:20.037190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:20.037190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:20.037190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:12:20.037190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:20.037784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:20.037784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:20.037784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:12:20.037784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:20.037784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:12:20.037784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:12:20.037784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:20.037784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:12:20.037784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:20.037784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:12:20.037784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:12:20.037784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:20.037784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:12:20.038303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:20.038303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:12:20.038303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:20.038303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:12:20.038303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:12:20.038303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:12:20.038303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:12:20.038303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:12:20.038303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:20.038303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:12:20.038303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:12:20.038303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:12:20.038303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:12:20.038303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:12:20.038303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:20.038932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:20.038932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:12:20.038932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:12:20.038932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:12:20.038932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:12:20.038932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:20.038932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:20.038932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:20.038932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:12:20.038932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:20.038932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:20.038932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:20.038932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:12:20.038932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:20.038932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:12:20.038932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:12:20.038932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:20.038932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:12:20.038932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:20.039748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:12:20.039748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:12:20.039748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:20.039748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:12:20.039748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:20.039748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:12:20.043749 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:12:20.044011 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:12:20.044111 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:12:20.044238 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:12:20.044576 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:12:20.044768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:20.045222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:12:20.045222 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:20.045222 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:12:20.045222 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:12:20.045222 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:20.045222 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:12:20.045222 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:20.045222 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:12:20.046021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:20.046021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:12:20.046021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:12:20.046021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:12:20.046021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:20.046021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:12:20.046021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:20.046021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:12:20.046021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:20.046021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:12:20.046021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:20.046021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:12:20.046021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:20.046021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:12:20.046021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:12:20.046431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:20.046431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:12:20.046431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:20.046431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:12:20.046431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:12:20.046431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:20.046431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:12:20.046431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:20.046431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:12:20.046809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:20.046809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:12:20.046809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:20.046809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:12:20.046809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:12:20.046809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:12:20.046809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:12:20.046809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:12:20.046809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:12:20.046809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:12:20.046809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:20.046809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:20.047121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:20.047121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:12:20.047121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:20.047121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:20.047121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:20.047121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:12:20.047121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:20.047121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:12:20.047121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:12:20.047121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:20.047121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:12:20.047121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:20.047121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:12:20.047121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:12:20.047507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:20.047507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:12:20.047507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:20.047507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:12:20.048085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:20.048085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:12:20.048085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:12:20.048085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:12:20.048085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:12:20.048085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:12:20.048085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:20.048085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:12:20.048085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:12:20.048085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:12:20.048085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:12:20.048085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:12:20.048085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:20.048085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:20.048085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:12:20.048085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:12:20.048515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:12:20.048515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:12:20.048515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:12:20.048515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:12:20.048515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:12:20.048515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:20.048515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:20.048515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:20.048515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:12:20.048515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:20.048515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:20.048515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:20.048515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:12:20.048515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:20.048515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:12:20.048515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:12:20.048515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:20.048515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:12:20.048970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:20.048970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:12:20.048970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:12:20.048970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:20.048970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:12:20.048970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:20.048970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:12:20.049616 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:12:20.049976 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:12:20.050151 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:12:20.050353 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:12:20.050877 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:12:20.051223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:20.051986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:12:20.051986 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:20.051986 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:12:20.051986 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:12:20.051986 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:20.051986 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:12:20.051986 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:20.051986 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:12:20.053198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:20.053198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:12:20.053198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:12:20.053198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:12:20.053198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:20.053198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:12:20.053198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:20.053198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:12:20.053198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:20.053198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:12:20.053198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:20.053198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:12:20.053198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:20.053198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:12:20.053198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:12:20.053672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:20.053672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:12:20.053672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:20.053672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:12:20.053672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:12:20.053672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:20.053672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:12:20.053672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:20.053672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:12:20.054885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:20.054885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:12:20.054885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:20.054885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:12:20.054885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:12:20.054885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:12:20.054885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:12:20.054885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:12:20.054885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:12:20.054885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:20.054885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:20.055231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:20.055231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:12:20.055231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:20.055231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:20.055231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:20.055231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:12:20.055231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:20.055231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:12:20.055231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:12:20.055231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:20.055231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:12:20.055231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:20.055231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:12:20.055231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:12:20.055594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:20.055594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:12:20.055594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:20.055594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:12:20.055594 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:12:20.056345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:20.056345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:12:20.056345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:12:20.056345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:12:20.056345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:12:20.056345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:12:20.056345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:20.056345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:12:20.056345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:12:20.056345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:12:20.056345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:12:20.056345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:12:20.056345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:20.056345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:20.056345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:12:20.056345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:12:20.056345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:12:20.057020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:12:20.057020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:12:20.057020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:12:20.057020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:20.057020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:20.057020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:20.057020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:12:20.057020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:20.057020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:20.057020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:20.057020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:12:20.057020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:20.057020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:12:20.057020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:12:20.057020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:20.057020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:12:20.057020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:20.057020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:12:20.057020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:12:20.057537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:20.057537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:12:20.057537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:20.057537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:12:20.057537 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:12:20.057537 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:12:20.057537 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:12:20.057728 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:12:20.058025 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:12:20.058232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:20.058942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:12:20.058942 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:20.058942 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:12:20.058942 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:12:20.058942 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:20.058942 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:12:20.058942 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:20.058942 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:12:20.059801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:20.059801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:12:20.059801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:20.059801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:12:20.059801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:12:20.059801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:12:20.059801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:12:20.059801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:20.059801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:20.059801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:20.059801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:12:20.059801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:20.060188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:20.060188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:20.060188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:12:20.060188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:20.060188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:12:20.060188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:12:20.060188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:20.060188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:12:20.060188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:20.060188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:12:20.060188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:12:20.060188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:20.060188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:12:20.060555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:20.060555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:12:20.060734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:20.060734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:12:20.060734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:12:20.060734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:12:20.060734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:12:20.060734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:12:20.060734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:20.060734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:12:20.060734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:12:20.060734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:12:20.060734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:12:20.060734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:12:20.060734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:20.060734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:20.060734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:12:20.060734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:12:20.061148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:12:20.061148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:12:20.061148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:20.061148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:20.061148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:20.061148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:12:20.061148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:20.061148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:20.061148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:20.061148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:12:20.061148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:20.061148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:12:20.061148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:12:20.061148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:20.061148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:12:20.061148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:20.061148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:12:20.061148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:12:20.061148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:20.061624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:12:20.061624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:20.061624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:12:24.796069 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:12:24.796714 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:12:24.796833 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:12:24.797519 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:12:24.800052 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG dbcounter [-] [105581] Writing DB stats neutron:SELECT=240 {{(pid=105581) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 14:12:24.837355 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:12:24.837538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:24.837985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:12:24.837985 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:24.837985 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:12:24.837985 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:12:24.837985 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:24.837985 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:12:24.837985 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:24.837985 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:12:24.838634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:24.838634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:12:24.838634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:24.838634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:12:24.838634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:12:24.838634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:12:24.838634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:12:24.838634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:24.838634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:24.838634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:24.838634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:12:24.838634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:24.839001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:24.839001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:24.839001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:12:24.839001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:24.839001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:12:24.839001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:12:24.839001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:24.839001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:12:24.839001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:24.839001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:12:24.839001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:12:24.839001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:24.839001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:12:24.839423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:24.839423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:12:24.839423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:24.839423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:12:24.839423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:12:24.839423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:12:24.839423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:12:24.839423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:12:24.839423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:24.839423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:12:24.839423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:12:24.839423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:12:24.839423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:12:24.839423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:12:24.839423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:24.839837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:24.839837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:12:24.839837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:12:24.839837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:12:24.839837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:12:24.839837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:24.839837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:24.839837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:24.839837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:12:24.839837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:24.839837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:24.839837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:24.839837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:12:24.839837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:24.839837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:12:24.839837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:12:24.839837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:24.839837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:12:24.839837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:24.840331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:12:24.840331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:12:24.840331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:24.840331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:12:24.840331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:24.840331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:12:25.034207 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:12:25.034722 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:12:25.034722 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:12:25.035317 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:12:25.035484 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:12:25.035594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:25.036015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:12:25.036015 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:25.036015 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:12:25.036015 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:12:25.036015 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:25.036015 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:12:25.036015 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:25.036015 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:12:25.036801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:25.036801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:12:25.036801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:25.036801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:12:25.036801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:12:25.036801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:12:25.036801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:12:25.036801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:25.036801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:25.036801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:25.036801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:12:25.036801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:25.037140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:25.037140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:25.037140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:12:25.037140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:25.037140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:12:25.037140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:12:25.037140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:25.037140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:12:25.037140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:25.037140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:12:25.037140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:12:25.037140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:25.037140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:12:25.037521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:25.037521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:12:25.038109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:25.038109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:12:25.038109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:12:25.038109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:12:25.038109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:12:25.038109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:12:25.038109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:25.038109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:12:25.038109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:12:25.038109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:12:25.038109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:12:25.038109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:12:25.038109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:25.038109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:25.038109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:12:25.038109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:12:25.038853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:12:25.038853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:12:25.038853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:25.038853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:25.038853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:25.038853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:12:25.038853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:25.038853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:25.038853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:25.038853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:12:25.038853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:25.038853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:12:25.038853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:12:25.038853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:25.038853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:12:25.038853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:25.038853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:12:25.038853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:12:25.038853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:25.039733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:12:25.039733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:25.039733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:12:25.044088 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:12:25.044357 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:12:25.044499 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:12:25.044618 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:12:25.044864 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:12:25.045031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:25.045361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:12:25.045361 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:25.045361 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:12:25.045361 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:12:25.045361 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:25.045361 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:12:25.045361 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:25.045361 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:12:25.046138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:25.046138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:12:25.046138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:12:25.046138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:12:25.046138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:25.046138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:12:25.046138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:25.046138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:12:25.046138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:25.046138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:12:25.046138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:25.046138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:12:25.046138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:25.046138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:12:25.046138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:12:25.046596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:25.046596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:12:25.046596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:25.046596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:12:25.046596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:12:25.046596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:25.046596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:12:25.046596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:25.046596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:12:25.047423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:25.047423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:12:25.047423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:25.047423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:12:25.047423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:12:25.047423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:12:25.047423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:12:25.047423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:12:25.047423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:12:25.047423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:12:25.047423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:25.047423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:25.048097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:25.048097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:12:25.048097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:25.048097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:25.048097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:25.048097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:12:25.048097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:25.048097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:12:25.048097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:12:25.048097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:25.048097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:12:25.048097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:25.048097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:12:25.048097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:12:25.048504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:25.048504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:12:25.048504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:25.048504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:12:25.048504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:25.048504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:12:25.048504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:12:25.048504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:12:25.048504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:12:25.048504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:12:25.048504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:25.048504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:12:25.048504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:12:25.048504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:12:25.048873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:12:25.048873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:12:25.048873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:25.048873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:25.048873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:12:25.048873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:12:25.048873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:12:25.048873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:12:25.048873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:12:25.048873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:12:25.048873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:12:25.048873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:25.048873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:25.048873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:25.048873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:12:25.048873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:25.048873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:25.048873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:25.048873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:12:25.049376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:25.049376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:12:25.049376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:12:25.049376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:25.049376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:12:25.049376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:25.049376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:12:25.049376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:12:25.049376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:25.049376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:12:25.049376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:25.049376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:12:25.049989 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:12:25.050279 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:12:25.050417 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:12:25.050553 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:12:25.050769 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:12:25.050921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:25.051254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:12:25.051254 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:25.051254 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:12:25.051254 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:12:25.051254 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:25.051254 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:12:25.051254 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:25.051254 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:12:25.052162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:25.052162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:12:25.052162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:12:25.052162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:12:25.052162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:25.052162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:12:25.052162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:25.052162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:12:25.052162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:25.052162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:12:25.052162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:25.052162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:12:25.052162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:25.052162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:12:25.052162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:12:25.052596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:25.052596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:12:25.052596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:25.052596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:12:25.052596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:12:25.052596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:25.052596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:12:25.052596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:25.052596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:12:25.053108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:25.053108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:12:25.053108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:25.053108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:12:25.053108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:12:25.053108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:12:25.053108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:12:25.053108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:12:25.053108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:12:25.053108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:25.053108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:25.053417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:25.053417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:12:25.053417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:25.053417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:25.053417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:25.053417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:12:25.053417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:25.053417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:12:25.053417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:12:25.053417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:25.053417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:12:25.053417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:25.053417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:12:25.053417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:12:25.053820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:25.053820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:12:25.053820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:25.053820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:12:25.054069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:25.054069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:12:25.054069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:12:25.054069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:12:25.054069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:12:25.054069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:12:25.054069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:25.054069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:12:25.054069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:12:25.054069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:12:25.054069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:12:25.054069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:12:25.054069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:25.054069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:25.054069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:12:25.054069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:12:25.054069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:12:25.054564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:12:25.054564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:12:25.054564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:12:25.054564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:25.054564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:25.054564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:25.054564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:12:25.054564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:25.054564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:25.054564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:25.054564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:12:25.054564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:25.054564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:12:25.054564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:12:25.054564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:25.054564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:12:25.054564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:25.054564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:12:25.054564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:12:25.055064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:25.055064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:12:25.055064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:25.055064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:12:25.056625 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:12:25.056858 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:12:25.056968 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:12:25.057112 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:12:25.057632 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:12:25.057632 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:12:25.057729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:25.058182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:12:25.058182 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:25.058182 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:12:25.058182 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:12:25.058182 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:25.058182 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:12:25.058182 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:25.058182 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:12:25.059025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:25.059025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:12:25.059025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:25.059025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:12:25.059025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:12:25.059025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:12:25.059025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:12:25.059025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:25.059025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:25.059025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:25.059025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:12:25.059025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:25.059374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:25.059374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:25.059374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:12:25.059374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:25.059374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:12:25.059374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:12:25.059374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:25.059374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:12:25.059374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:25.059374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:12:25.059374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:12:25.059374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:25.059374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:12:25.059790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:25.059790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:12:25.060037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:25.060037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:12:25.060037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:12:25.060037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:12:25.060037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:12:25.060037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:12:25.060037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:25.060037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:12:25.060037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:12:25.060037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:12:25.060037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:12:25.060037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:12:25.060037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:25.060037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:25.060037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:12:25.060037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:12:25.060500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:12:25.060500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:12:25.060500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:25.060500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:25.060500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:25.060500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:12:25.060500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:25.060500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:25.060500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:25.060500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:12:25.060500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:25.060500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:12:25.060500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:12:25.060500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:25.060500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:12:25.060500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:25.060500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:12:25.060500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:12:25.060500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:25.061168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:12:25.061168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:25.061168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:12:29.796875 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:12:29.797365 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:12:29.797589 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:12:29.798894 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:12:29.840816 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:12:29.841221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:29.842311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:12:29.842311 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:29.842311 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:12:29.842311 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:12:29.842311 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:29.842311 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:12:29.842311 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:29.842311 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:12:29.843656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.045 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:29.843656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:12:29.843656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:29.843656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:12:29.843656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:12:29.843656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:12:29.843656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:12:29.843656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:29.843656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:29.843656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:29.843656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:12:29.843656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:29.844084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:29.844084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:29.844084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:12:29.844084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:29.844084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:12:29.844084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:12:29.844084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:29.844084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:12:29.844084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:29.844084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:12:29.844084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:12:29.844084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:29.844084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:12:29.844681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:29.844681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:12:29.844681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:29.844681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:12:29.844681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:12:29.844681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:12:29.844681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:12:29.844681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:12:29.844681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:29.844681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:12:29.844681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:12:29.844681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:12:29.844681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:12:29.844681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:12:29.844681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:29.845374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:29.845374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:12:29.845374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:12:29.845374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:12:29.845374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:12:29.845374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:29.845374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:29.845374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:29.845374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:12:29.845374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:29.845374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:29.845374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:29.845374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:12:29.845374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:29.845374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:12:29.845374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:12:29.845374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:29.845374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:12:29.845374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:29.845950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:12:29.845950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:12:29.845950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:29.845950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:12:29.845950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:29.845950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:12:30.034821 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:12:30.035803 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:12:30.035916 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:12:30.037185 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:12:30.037638 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:12:30.037862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:30.038873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:12:30.038873 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:30.038873 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:12:30.038873 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:12:30.038873 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:30.038873 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:12:30.038873 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:30.038873 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:12:30.040563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:30.040563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:12:30.040563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:30.040563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:12:30.040563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:12:30.040563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:12:30.040563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:12:30.040563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:30.040563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:30.040563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:30.040563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:12:30.040563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:30.041045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:30.041045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:30.041045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:12:30.041045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:30.041045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:12:30.041045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:12:30.041045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:30.041045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:12:30.041045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:30.041045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:12:30.041045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:12:30.041045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:30.041045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:12:30.041477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:30.041477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:12:30.042173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:30.042173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:12:30.042173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:12:30.042173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:12:30.042173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:12:30.042173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:12:30.042173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:30.042173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:12:30.042173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:12:30.042173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:12:30.042173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:12:30.042173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:12:30.042173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:30.042173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:30.042173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:12:30.042173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:12:30.042733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:12:30.042733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:12:30.042733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:30.042733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:30.042733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:30.042733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:12:30.042733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:30.042733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:30.042733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:30.042733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:12:30.042733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:30.042733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:12:30.042733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:12:30.042733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:30.042733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:12:30.042733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:30.042733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:12:30.042733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:12:30.042733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:30.043275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:12:30.043275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:30.043275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:12:30.044683 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:12:30.045034 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:12:30.045128 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:12:30.045256 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:12:30.045797 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:12:30.046110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:30.046980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:12:30.046980 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:30.046980 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:12:30.046980 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:12:30.046980 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:30.046980 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:12:30.046980 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:30.046980 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:12:30.047768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:30.047768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:12:30.047768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:12:30.047768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:12:30.047768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:30.047768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:12:30.047768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:30.047768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:12:30.047768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:30.047768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:12:30.047768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:30.047768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:12:30.047768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:30.047768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:12:30.047768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:12:30.048281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:30.048281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:12:30.048281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:30.048281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:12:30.048281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:12:30.048281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:30.048281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:12:30.048281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:30.048281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:12:30.048580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:30.048580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:12:30.048580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:30.048580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:12:30.048580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:12:30.048580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:12:30.048580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:12:30.048580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:12:30.048580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:12:30.048580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:12:30.048580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:30.048580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:30.048936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:30.048936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:12:30.048936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:30.048936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:30.048936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:30.048936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:12:30.048936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:30.048936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:12:30.048936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:12:30.048936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:30.048936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:12:30.048936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:30.048936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:12:30.048936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:12:30.049340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:30.049340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:12:30.049340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:30.049340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:12:30.049973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:30.049973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:12:30.049973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:12:30.049973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:12:30.049973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:12:30.049973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:12:30.049973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:30.049973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:12:30.049973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:12:30.049973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:12:30.049973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:12:30.049973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:12:30.049973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:30.049973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:30.049973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:12:30.049973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:12:30.050485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:12:30.050485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:12:30.050485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:12:30.050485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:12:30.050485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:12:30.050485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:30.050485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:30.050485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:30.050485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:12:30.050485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:30.050485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:30.050485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:30.050485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:12:30.050485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:30.050485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:12:30.050485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:12:30.050485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:30.050485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:12:30.051082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:30.051082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:12:30.051082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:12:30.051082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:30.051082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:12:30.051082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:30.051082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:12:30.051082 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:12:30.051082 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:12:30.051082 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:12:30.051402 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:12:30.051745 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:12:30.051898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:30.052620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:12:30.052620 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:30.052620 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:12:30.052620 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:12:30.052620 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:30.052620 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:12:30.052620 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:30.052620 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:12:30.053428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:30.053428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:12:30.053428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:12:30.053428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:12:30.053428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:30.053428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:12:30.053428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:30.053428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:12:30.053428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:30.053428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:12:30.053428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:30.053428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:12:30.053428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:30.053428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:12:30.053428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:12:30.054014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:30.054014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:12:30.054014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:30.054014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:12:30.054014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:12:30.054014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:30.054014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:12:30.054014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:30.054014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:12:30.054296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:30.054296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:12:30.054296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:30.054296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:12:30.054296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:12:30.054296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:12:30.054296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:12:30.054296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:12:30.054296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:12:30.054296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:30.054296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:30.054685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:30.054685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:12:30.054685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:30.054685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:30.054685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:30.054685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:12:30.054685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:30.054685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:12:30.054685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:12:30.054685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:30.054685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:12:30.054685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:30.054685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:12:30.054685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:12:30.055113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:30.055113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:12:30.055113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:30.055113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:12:30.055565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:30.055565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:12:30.055565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:12:30.055565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:12:30.055565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:12:30.055565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:12:30.055565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:30.055565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:12:30.055565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:12:30.055565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:12:30.055565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:12:30.055565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:12:30.055565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:30.055565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:30.055565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:12:30.055565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:12:30.055565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:12:30.055954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:12:30.055954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:12:30.055954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:12:30.055954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:30.055954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:30.055954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:30.055954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:12:30.055954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:30.055954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:30.055954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:30.055954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:12:30.055954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:30.055954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:12:30.055954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:12:30.055954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:30.055954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:12:30.055954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:30.055954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:12:30.055954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:12:30.056601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:30.056601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:12:30.056601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:30.056601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:12:30.057139 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:12:30.057462 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:12:30.057555 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:12:30.057709 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:12:30.058249 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:12:30.058808 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:12:30.059052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:30.059893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:12:30.059893 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:30.059893 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:12:30.059893 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:12:30.059893 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:30.059893 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:12:30.059893 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:30.059893 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:12:30.060731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:30.060731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:12:30.060731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:30.060731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:12:30.060731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:12:30.060731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:12:30.060731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:12:30.060731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:30.060731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:30.060731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:30.060731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:12:30.060731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:30.061002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:30.061002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:30.061002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:12:30.061002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:30.061002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:12:30.061002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:12:30.061002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:30.061002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:12:30.061002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:30.061002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:12:30.061002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:12:30.061002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:30.061002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:12:30.061244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:30.061244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:12:30.061703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:30.061703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:12:30.061703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:12:30.061703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:12:30.061703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:12:30.061703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:12:30.061703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:30.061703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:12:30.061703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:12:30.061703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:12:30.061703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:12:30.061703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:12:30.061703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:30.061703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:30.061703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:12:30.061703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:12:30.062040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:12:30.062040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:12:30.062040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:30.062040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:30.062040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:30.062040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:12:30.062040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:30.062040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:30.062040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:30.062040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:12:30.062040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:30.062040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:12:30.062040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:12:30.062040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:30.062040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:12:30.062040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:30.062040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:12:30.062040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:12:30.062040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:30.062392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:12:30.062392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:30.062392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:12:34.797597 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:12:34.798492 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:12:34.798492 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:12:34.799872 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:12:34.830673 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:12:34.831292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:34.832550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:12:34.832550 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:34.832550 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:12:34.832550 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:12:34.832550 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:34.832550 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:12:34.832550 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:34.832550 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:12:34.833551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:34.833551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:12:34.833551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:34.833551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:12:34.833551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:12:34.833551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:12:34.833551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:12:34.833551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:34.833551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:34.833551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:34.833551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:12:34.833551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:34.833926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:34.833926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:34.833926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:12:34.833926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:34.833926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:12:34.833926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:12:34.833926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:34.833926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:12:34.833926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:34.833926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:12:34.833926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:12:34.833926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:34.833926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:12:34.834316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:34.834316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:12:34.835567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:34.835567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:12:34.835567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:12:34.835567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:12:34.835567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:12:34.835567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:12:34.835567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:34.835567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:12:34.835567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:12:34.835567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:12:34.835567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:12:34.835567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:12:34.835567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:34.835567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:34.835567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:12:34.835567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:12:34.836224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:12:34.836224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:12:34.836224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:34.836224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:34.836224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:34.836224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:12:34.836224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:34.836224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:34.836224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:34.836224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:12:34.836224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:34.836224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:12:34.836224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:12:34.836224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:34.836224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:12:34.836224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:34.836224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:12:34.836224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:12:34.836224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:34.836776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:12:34.836776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:34.836776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:12:35.035952 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:12:35.036254 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:12:35.036485 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:12:35.038595 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:12:35.038595 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:12:35.038595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:35.039158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:12:35.039158 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:35.039158 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:12:35.039158 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:12:35.039158 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:35.039158 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:12:35.039158 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:35.039158 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:12:35.040862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:35.040862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:12:35.040862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:35.040862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:12:35.040862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:12:35.040862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:12:35.040862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:12:35.040862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:35.040862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:35.040862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:35.040862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:12:35.040862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:35.041239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:35.041239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:35.041239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:12:35.041239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:35.041239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:12:35.041239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:12:35.041239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:35.041239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:12:35.041239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:35.041239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:12:35.041239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:12:35.041239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:35.041239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:12:35.041629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:35.041629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:12:35.042547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:35.042547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:12:35.042547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:12:35.042547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:12:35.042547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:12:35.042547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:12:35.042547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:35.042547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:12:35.042547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:12:35.042547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:12:35.042547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:12:35.042547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:12:35.042547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:35.042547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:35.042547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:12:35.042547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:12:35.043024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:12:35.043024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:12:35.043024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:35.043024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:35.043024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:35.043024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:12:35.043024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:35.043024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:35.043024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:35.043024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:12:35.043024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:35.043024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:12:35.043024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:12:35.043024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:35.043024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:12:35.043024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:35.043024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:12:35.043024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:12:35.043024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:35.043604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:12:35.043604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:35.043604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:12:35.045198 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:12:35.045597 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:12:35.045772 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:12:35.046088 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:12:35.046416 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:12:35.046627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:35.047176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:12:35.047176 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:35.047176 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:12:35.047176 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:12:35.047176 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:35.047176 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:12:35.047176 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:35.047176 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:12:35.047986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:35.047986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:12:35.047986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:12:35.047986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:12:35.047986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:35.047986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:12:35.047986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:35.047986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:12:35.047986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:35.047986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:12:35.047986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:35.047986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:12:35.047986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:35.047986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:12:35.047986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:12:35.048580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:35.048580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:12:35.048580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:35.048580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:12:35.048580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:12:35.048580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:35.048580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:12:35.048580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:35.048580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:12:35.048875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:35.048875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:12:35.048875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:35.048875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:12:35.048875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:12:35.048875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:12:35.048875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:12:35.048875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:12:35.048875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:12:35.048875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:12:35.048875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:35.048875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:35.049245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:35.049245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:12:35.049245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:35.049245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:35.049245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:35.049245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:12:35.049245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:35.049245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:12:35.049245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:12:35.049245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:35.049245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:12:35.049245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:35.049245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:12:35.049245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:12:35.049668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:35.049668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:12:35.049668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:35.049668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:12:35.049791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:35.049791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:12:35.049791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:12:35.049791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:12:35.049791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:12:35.049791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:12:35.049791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:35.049791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:12:35.049791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:12:35.049791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:12:35.049791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:12:35.049791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:12:35.049791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:35.049791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:35.049791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:12:35.049791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:12:35.050246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:12:35.050246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:12:35.050246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:12:35.050246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:12:35.050246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:12:35.050246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:35.050246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:35.050246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:35.050246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:12:35.050246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:35.050246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:35.050246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:35.050246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:12:35.050246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:35.050246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:12:35.050246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:12:35.050246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:35.050246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:12:35.050796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:35.050796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:12:35.050796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:12:35.050796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:35.050796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:12:35.050796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:35.050796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:12:35.051900 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:12:35.052117 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:12:35.052208 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:12:35.052343 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:12:35.055462 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:12:35.055462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:35.055462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:12:35.055462 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:35.055462 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:12:35.055462 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:12:35.055462 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:35.055462 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:12:35.055462 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:35.055462 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:12:35.055775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:35.055775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:12:35.055775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:12:35.055775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:12:35.055775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:35.055775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:12:35.055775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:35.055775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:12:35.055775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:35.055775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:12:35.055775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:35.055775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:12:35.055775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:35.055775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:12:35.055775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:12:35.056089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:35.056089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:12:35.056089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:35.056089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:12:35.056089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:12:35.056089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:35.056089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:12:35.056089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:35.056089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:12:35.056376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:35.056376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:12:35.056376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:35.056376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:12:35.056376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:12:35.056376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:12:35.056376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:12:35.056376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:12:35.056376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:12:35.056376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:35.056376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:35.056637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:35.056637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:12:35.056637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:35.056637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:35.056637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:35.056637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:12:35.056637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:35.056637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:12:35.056637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:12:35.056637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:35.056637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:12:35.056637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:35.056637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:12:35.056637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:12:35.056892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:35.056892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:12:35.056892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:35.056892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:12:35.057419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:35.057419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:12:35.057419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:12:35.057419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:12:35.057419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:12:35.057419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:12:35.057419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:35.057419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:12:35.057419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:12:35.057419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:12:35.057419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:12:35.057419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:12:35.057419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:35.057419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:35.057419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:12:35.057419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:12:35.057419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:12:35.057746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:12:35.057746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:12:35.057746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:12:35.057746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:35.057746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:35.057746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:35.057746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:12:35.057746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:35.057746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:35.057746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:35.057746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:12:35.057746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:35.057746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:12:35.057746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:12:35.057746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:35.057746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:12:35.057746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:35.057746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:12:35.057746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:12:35.058079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:35.058079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:12:35.058079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:35.058079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:12:35.058079 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:12:35.058079 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:12:35.058206 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:12:35.058261 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:12:35.059269 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:12:35.059881 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:12:35.060053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:35.060580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:12:35.060580 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:35.060580 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:12:35.060580 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:12:35.060580 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:35.060580 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:12:35.060580 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:35.060580 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:12:35.061344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:35.061344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:12:35.061344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:35.061344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:12:35.061344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:12:35.061344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:12:35.061344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:12:35.061344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:35.061344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:35.061344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:35.061344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:12:35.061344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:35.061697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:35.061697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:35.061697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:12:35.061697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:35.061697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:12:35.061697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:12:35.061697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:35.061697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:12:35.061697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:35.061697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:12:35.061697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:12:35.061697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:35.061697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:12:35.061940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:35.061940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:12:35.062268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:35.062268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:12:35.062268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:12:35.062268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:12:35.062268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:12:35.062268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:12:35.062268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:35.062268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:12:35.062268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:12:35.062268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:12:35.062268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:12:35.062268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:12:35.062268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:35.062268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:35.062268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:12:35.062268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:12:35.062629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:12:35.062629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:12:35.062629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:35.062629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:35.062629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:35.062629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:12:35.062629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:35.062629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:35.062629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:35.062629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:12:35.062629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:35.062629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:12:35.062629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:12:35.062629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:35.062629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:12:35.062629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:35.062629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:12:35.062629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:12:35.062629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:35.062980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:12:35.062980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:35.062980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:12:39.798192 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:12:39.801247 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:12:39.801247 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:12:39.801247 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:12:39.837612 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:12:39.838334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:39.839399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:12:39.839399 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:39.839399 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:12:39.839399 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:12:39.839399 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:39.839399 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:12:39.839399 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:39.839399 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:12:39.840523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:39.840523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:12:39.840523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:39.840523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:12:39.840523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:12:39.840523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:12:39.840523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:12:39.840523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:39.840523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:39.840523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:39.840523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:12:39.840523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:39.840810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:39.840810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:39.840810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:12:39.840810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:39.840810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:12:39.840810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:12:39.840810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:39.840810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:12:39.840810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:39.840810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:12:39.840810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:12:39.840810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:39.840810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:12:39.841066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:39.841066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:12:39.841908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:39.841908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:12:39.841908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:12:39.841908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:12:39.841908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:12:39.841908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:12:39.841908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:39.841908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:12:39.841908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:12:39.841908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:12:39.841908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:12:39.841908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:12:39.841908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:39.841908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:39.841908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:12:39.841908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:12:39.842263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:12:39.842263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:12:39.842263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:39.842263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:39.842263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:39.842263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:12:39.842263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:39.842263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:39.842263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:39.842263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:12:39.842263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:39.842263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:12:39.842263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:12:39.842263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:39.842263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:12:39.842263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:39.842263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:12:39.842263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:12:39.842263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:39.842656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:12:39.842656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:39.842656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:12:40.036257 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:12:40.036889 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:12:40.037082 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:12:40.038105 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:12:40.038562 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:12:40.038755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:40.039694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:12:40.039694 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:40.039694 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:12:40.039694 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:12:40.039694 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:40.039694 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:12:40.039694 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:40.039694 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:12:40.041431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:40.041431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:12:40.041431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:40.041431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:12:40.041431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:12:40.041431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:12:40.041431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:12:40.041431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:40.041431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:40.041431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:40.041431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:12:40.041431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:40.041929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:40.041929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:40.041929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:12:40.041929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:40.041929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:12:40.041929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:12:40.041929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:40.041929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:12:40.041929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:40.041929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:12:40.041929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:12:40.041929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:40.041929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:12:40.042513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:40.042513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:12:40.042513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:40.042513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:12:40.042513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:12:40.042513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:12:40.042513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:12:40.042513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:12:40.042513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:40.042513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:12:40.042513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:12:40.042513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:12:40.042513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:12:40.042513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:12:40.042513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:40.043228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:40.043228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:12:40.043228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:12:40.043228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:12:40.043228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:12:40.043228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:40.043228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:40.043228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:40.043228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:12:40.043228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:40.043228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:40.043228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:40.043228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:12:40.043228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:40.043228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:12:40.043228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:12:40.043228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:40.043228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:12:40.043228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:40.043998 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:12:40.043998 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:12:40.043998 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:40.043998 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:12:40.043998 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:40.043998 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:12:40.045806 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:12:40.046286 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:12:40.046370 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:12:40.046537 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:12:40.046875 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:12:40.047070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:40.047592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:12:40.047592 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:40.047592 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:12:40.047592 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:12:40.047592 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:40.047592 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:12:40.047592 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:40.047592 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:12:40.048791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:40.048791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:12:40.048791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:12:40.048791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:12:40.048791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:40.048791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:12:40.048791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:40.048791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:12:40.048791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:40.048791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:12:40.048791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:40.048791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:12:40.048791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:40.048791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:12:40.048791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:12:40.049454 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:40.049454 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:12:40.049454 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:40.049454 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:12:40.049454 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:12:40.049454 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:40.049454 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:12:40.049454 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:40.049454 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:12:40.049835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:40.049835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:12:40.049835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:40.049835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:12:40.049835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:12:40.049835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:12:40.049835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:12:40.049835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:12:40.049835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:12:40.049835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:12:40.049835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:40.049835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:40.050673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:40.050673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:12:40.050673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:40.050673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:40.050673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:40.050673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:12:40.050673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:40.050673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:12:40.050673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:12:40.050673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:40.050673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:12:40.050673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:40.050673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:12:40.050673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:12:40.051078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:40.051078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:12:40.051078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:40.051078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:12:40.051078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:40.051078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:12:40.051078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:12:40.051078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:12:40.051078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:12:40.051078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:12:40.051078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:40.051078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:12:40.051078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:12:40.051078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:12:40.051379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:12:40.051379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:12:40.051379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:40.051379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:40.051379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:12:40.051379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:12:40.051379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:12:40.051379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:12:40.051379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:12:40.051379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:12:40.051379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:12:40.051379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:40.051379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:40.051379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:40.051379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:12:40.051379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:40.051379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:40.051379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:40.051379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:12:40.051798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:40.051798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:12:40.051798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:12:40.051798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:40.051798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:12:40.051798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:40.051798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:12:40.051798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:12:40.051798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:40.051798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:12:40.051798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:40.051798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:12:40.052254 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:12:40.052582 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:12:40.052694 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:12:40.052817 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:12:40.053277 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:12:40.053678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:40.054388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:12:40.054388 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:40.054388 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:12:40.054388 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:12:40.054388 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:40.054388 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:12:40.054388 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:40.054388 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:12:40.055221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:40.055221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:12:40.055221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:12:40.055221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:12:40.055221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:40.055221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:12:40.055221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:40.055221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:12:40.055221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:40.055221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:12:40.055221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:40.055221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:12:40.055221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:40.055221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:12:40.055221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:12:40.055560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:40.055560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:12:40.055560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:40.055560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:12:40.055560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:12:40.055560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:40.055560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:12:40.055560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:40.055560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:12:40.056549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:40.056549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:12:40.056549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:40.056549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:12:40.056549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:12:40.056549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:12:40.056549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:12:40.056549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:12:40.056549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:12:40.056549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:40.056549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:40.056793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:40.056793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:12:40.056793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:40.056793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:40.056793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:40.056793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:12:40.056793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:40.056793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:12:40.056793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:12:40.056793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:40.056793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:12:40.056793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:40.056793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:12:40.056793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:12:40.057058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:40.057058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:12:40.057058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:40.057058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:12:40.057799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:40.057799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:12:40.057799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:12:40.057799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:12:40.057799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:12:40.057799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:12:40.057799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:40.057799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:12:40.057799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:12:40.057799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:12:40.057799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:12:40.057799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:12:40.057799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:40.057799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:40.057799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:12:40.057799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:12:40.057799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:12:40.058255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:12:40.058255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:12:40.058255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:12:40.058255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:40.058255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:40.058255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:40.058255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:12:40.058255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:40.058255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:40.058255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:40.058255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:12:40.058255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:40.058255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:12:40.058255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:12:40.058255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:40.058255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:12:40.058255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:40.058255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:12:40.058255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:12:40.058672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:40.058672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:12:40.058672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:40.058672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:12:40.058672 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:12:40.058672 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:12:40.058672 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:12:40.058822 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:12:40.059056 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:12:40.059279 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:12:40.059427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:40.059937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:12:40.059937 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:40.059937 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:12:40.059937 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:12:40.059937 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:40.059937 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:12:40.059937 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:40.059937 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:12:40.060796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:40.060796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:12:40.060796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:40.060796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:12:40.060796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:12:40.060796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:12:40.060796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:12:40.060796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:40.060796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:40.060796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:40.060796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:12:40.060796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:40.061056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:40.061056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:40.061056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:12:40.061056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:40.061056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:12:40.061056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:12:40.061056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:40.061056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:12:40.061056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:40.061056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:12:40.061056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:12:40.061056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:40.061056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:12:40.061299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:40.061299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:12:40.061738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:40.061738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:12:40.061738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:12:40.061738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:12:40.061738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:12:40.061738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:12:40.061738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:40.061738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:12:40.061738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:12:40.061738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:12:40.061738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:12:40.061738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:12:40.061738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:40.061738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:40.061738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:12:40.061738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:12:40.062116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:12:40.062116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:12:40.062116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:40.062116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:40.062116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:40.062116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:12:40.062116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:40.062116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:40.062116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:40.062116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:12:40.062116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:40.062116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:12:40.062116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:12:40.062116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:40.062116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:12:40.062116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:40.062116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:12:40.062116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:12:40.062116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:40.062483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:12:40.062483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:40.062483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:12:44.798866 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:12:44.799804 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:12:44.799804 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:12:44.800286 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:12:44.832625 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:12:44.832983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:44.833940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:12:44.833940 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:44.833940 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:12:44.833940 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:12:44.833940 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:44.833940 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:12:44.833940 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:44.833940 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:12:44.836312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:44.836312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:12:44.836312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:44.836312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:12:44.836312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:12:44.836312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:12:44.836312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:12:44.836312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:44.836312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:44.836312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:44.836312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:12:44.836312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:44.837158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:44.837158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:44.837158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:12:44.837158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:44.837158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:12:44.837158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:12:44.837158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:44.837158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:12:44.837158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:44.837158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:12:44.837158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:12:44.837158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:44.837158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:12:44.838223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:44.838223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:12:44.838223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:44.838223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:12:44.838223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:12:44.838223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:12:44.838223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:12:44.838223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:12:44.838223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:44.838223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:12:44.838223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:12:44.838223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:12:44.838223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:12:44.838223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:12:44.838223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:44.839081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:44.839081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:12:44.839081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:12:44.839081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:12:44.839081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:12:44.839081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:44.839081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:44.839081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:44.839081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:12:44.839081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:44.839081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:44.839081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:44.839081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:12:44.839081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:44.839081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:12:44.839081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:12:44.839081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:44.839081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:12:44.839081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:44.842110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:12:44.842110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:12:44.842110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:44.842110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:12:44.842110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:44.842110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:12:45.037024 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:12:45.037514 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:12:45.037690 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:12:45.038801 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:12:45.039045 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:12:45.039211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:45.039952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:12:45.039952 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:45.039952 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:12:45.039952 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:12:45.039952 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:45.039952 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:12:45.039952 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:45.039952 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:12:45.041486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:45.041486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:12:45.041486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:45.041486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:12:45.041486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:12:45.041486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:12:45.041486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:12:45.041486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:45.041486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:45.041486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:45.041486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:12:45.041486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:45.042007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:45.042007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:45.042007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:12:45.042007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:45.042007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:12:45.042007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:12:45.042007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:45.042007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:12:45.042007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:45.042007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:12:45.042007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:12:45.042007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:45.042007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:12:45.042475 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:45.042475 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:12:45.043483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:45.043483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:12:45.043483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:12:45.043483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:12:45.043483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:12:45.043483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:12:45.043483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:45.043483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:12:45.043483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:12:45.043483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:12:45.043483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:12:45.043483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:12:45.043483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:45.043483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:45.043483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:12:45.043483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:12:45.044020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:12:45.044020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:12:45.044020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:45.044020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:45.044020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:45.044020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:12:45.044020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:45.044020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:45.044020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:45.044020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:12:45.044020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:45.044020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:12:45.044020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:12:45.044020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:45.044020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:12:45.044020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:45.044020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:12:45.044020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:12:45.044020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:45.044655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:12:45.044655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:45.044655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:12:45.046375 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:12:45.046695 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:12:45.046902 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:12:45.047088 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:12:45.047471 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:12:45.047695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:45.048215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:12:45.048215 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:45.048215 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:12:45.048215 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:12:45.048215 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:45.048215 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:12:45.048215 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:45.048215 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:12:45.049004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:45.049004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:12:45.049004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:12:45.049004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:12:45.049004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:45.049004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:12:45.049004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:45.049004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:12:45.049004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:45.049004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:12:45.049004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:45.049004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:12:45.049004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:45.049004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:12:45.049004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:12:45.049450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:45.049450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:12:45.049450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:45.049450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:12:45.049450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:12:45.049450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:45.049450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:12:45.049450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:45.049450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:12:45.049919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:45.049919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:12:45.049919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:45.049919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:12:45.049919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:12:45.049919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:12:45.049919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:12:45.049919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:12:45.049919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:12:45.049919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:12:45.049919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:45.049919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:45.050984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:45.050984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:12:45.050984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:45.050984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:45.050984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:45.050984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:12:45.050984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:45.050984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:12:45.050984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:12:45.050984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:45.050984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:12:45.050984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:45.050984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:12:45.050984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:12:45.051396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:45.051396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:12:45.051396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:45.051396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:12:45.051396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:45.051396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:12:45.051396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:12:45.051396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:12:45.051396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:12:45.051396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:12:45.051396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:45.051396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:12:45.051396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:12:45.051396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:12:45.051806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:12:45.051806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:12:45.051806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:45.051806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:45.051806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:12:45.051806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:12:45.051806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:12:45.051806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:12:45.051806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:12:45.051806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:12:45.051806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:12:45.051806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:45.051806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:45.051806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:45.051806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:12:45.051806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:45.051806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:45.051806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:45.051806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:12:45.052285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:45.052285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:12:45.052285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:12:45.052285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:45.052285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:12:45.052285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:45.052285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:12:45.052285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:12:45.052285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:45.052285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:12:45.052285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:45.052285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:12:45.053145 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:12:45.053145 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:12:45.053145 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:12:45.053145 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:12:45.053491 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:12:45.053739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:45.054220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:12:45.054220 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:45.054220 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:12:45.054220 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:12:45.054220 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:45.054220 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:12:45.054220 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:45.054220 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:12:45.054938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:45.054938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:12:45.054938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:12:45.054938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:12:45.054938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:45.054938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:12:45.054938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:45.054938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:12:45.054938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:45.054938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:12:45.054938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:45.054938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:12:45.054938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:45.054938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:12:45.054938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:12:45.055622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:45.055622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:12:45.055622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:45.055622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:12:45.055622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:12:45.055622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:45.055622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:12:45.055622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:45.055622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:12:45.055854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:45.055854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:12:45.055854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:45.055854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:12:45.055854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:12:45.055854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:12:45.055854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:12:45.055854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:12:45.055854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:12:45.055854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:45.055854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:45.056237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:45.056237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:12:45.056237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:45.056237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:45.056237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:45.056237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:12:45.056237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:45.056237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:12:45.056237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:12:45.056237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:45.056237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:12:45.056237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:45.056237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:12:45.056237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:12:45.056665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:45.056665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:12:45.056665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:45.056665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:12:45.057192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:45.057192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:12:45.057192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:12:45.057192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:12:45.057192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:12:45.057192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:12:45.057192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:45.057192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:12:45.057192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:12:45.057192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:12:45.057192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:12:45.057192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:12:45.057192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:45.057192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:45.057192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:12:45.057192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:12:45.057192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:12:45.058425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:12:45.058425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:12:45.058425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:12:45.058425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:45.058425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:45.058425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:45.058425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:12:45.058425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:45.058425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:45.058425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:45.058425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:12:45.058425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:45.058425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:12:45.058425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:12:45.058425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:45.058425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:12:45.058425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:45.058425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:12:45.058425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:12:45.059856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:45.059856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:12:45.059856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:45.059856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:12:45.059856 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:12:45.059856 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:12:45.059856 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:12:45.059856 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:12:45.060166 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:12:45.060166 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:12:45.060166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:45.060534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:12:45.060534 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:45.060534 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:12:45.060534 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:12:45.060534 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:45.060534 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:12:45.060534 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:45.060534 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:12:45.061895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:45.061895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:12:45.061895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:45.061895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:12:45.061895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:12:45.061895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:12:45.061895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:12:45.061895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:45.061895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:45.061895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:45.061895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:12:45.061895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:45.062883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:45.062883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:45.062883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:12:45.062883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:45.062883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:12:45.062883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:12:45.062883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:45.062883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:12:45.062883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:45.062883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:12:45.062883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:12:45.062883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:45.062883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:12:45.063201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:45.063201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:12:45.063696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:45.063696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:12:45.063696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:12:45.063696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:12:45.063696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:12:45.063696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:12:45.063696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:45.063696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:12:45.063696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:12:45.063696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:12:45.063696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:12:45.063696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:12:45.063696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:45.063696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:45.063696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:12:45.063696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:12:45.064100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:12:45.064100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:12:45.064100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:45.064100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:45.064100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:45.064100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:12:45.064100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:45.064100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:45.064100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:45.064100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:12:45.064100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:45.064100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:12:45.064100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:12:45.064100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:45.064100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:12:45.064100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:45.064100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:12:45.064100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:12:45.064100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:45.064692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:12:45.064692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:45.064692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:12:49.799566 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:12:49.800198 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:12:49.800286 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:12:49.801147 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:12:49.834955 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:12:49.834955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:49.835177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:12:49.835177 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:49.835177 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:12:49.835177 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:12:49.835177 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:49.835177 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:12:49.835177 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:49.835177 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:12:49.835928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:49.835928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:12:49.835928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:49.835928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:12:49.835928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:12:49.835928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:12:49.835928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:12:49.835928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:49.835928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:49.835928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:49.835928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:12:49.835928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:49.836182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:49.836182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:49.836182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:12:49.836182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:49.836182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:12:49.836182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:12:49.836182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:49.836182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:12:49.836182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:49.836182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:12:49.836182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:12:49.836182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:49.836182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:12:49.836483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:49.836483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:12:49.836689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:49.836689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:12:49.836689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:12:49.836689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:12:49.836689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:12:49.836689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:12:49.836689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:49.836689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:12:49.836689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:12:49.836689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:12:49.836689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:12:49.836689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:12:49.836689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:49.836689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:49.836689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:12:49.836689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:12:49.837014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:12:49.837014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:12:49.837014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:49.837014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:49.837014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:49.837014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:12:49.837014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:49.837014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:49.837014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:49.837014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:12:49.837014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:49.837014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:12:49.837014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:12:49.837014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:49.837014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:12:49.837014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:49.837014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:12:49.837014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:12:49.837014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:49.837415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:12:49.837415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:49.837415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:12:50.037835 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:12:50.038601 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:12:50.038802 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:12:50.040035 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:12:50.040421 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:12:50.040709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:50.041673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:12:50.041673 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:50.041673 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:12:50.041673 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:12:50.041673 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:50.041673 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:12:50.041673 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:50.041673 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:12:50.043225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:50.043225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:12:50.043225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:50.043225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:12:50.043225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:12:50.043225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:12:50.043225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:12:50.043225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:50.043225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:50.043225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:50.043225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:12:50.043225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:50.043641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:50.043641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:50.043641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:12:50.043641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:50.043641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:12:50.043641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:12:50.043641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:50.043641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:12:50.043641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:50.043641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:12:50.043641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:12:50.043641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:50.043641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:12:50.044002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:50.044002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:12:50.045904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:50.045904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:12:50.045904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:12:50.045904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:12:50.045904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:12:50.045904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:12:50.045904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:50.045904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:12:50.045904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:12:50.045904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:12:50.045904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:12:50.045904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:12:50.045904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:50.045904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:50.045904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:12:50.045904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:12:50.046383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:12:50.046383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:12:50.046383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:50.046383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:50.046383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:50.046383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:12:50.046383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:50.046383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:50.046383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:50.046383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:12:50.046383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:50.046383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:12:50.046383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:12:50.046383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:50.046383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:12:50.046383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:50.046383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:12:50.046383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:12:50.046383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:50.046905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:12:50.046905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:50.046905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:12:50.046905 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:12:50.047643 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:12:50.047740 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:12:50.047861 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:12:50.048197 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:12:50.048396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:50.048934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:12:50.048934 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:50.048934 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:12:50.048934 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:12:50.048934 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:50.048934 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:12:50.048934 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:50.048934 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:12:50.049761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:50.049761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:12:50.049761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:12:50.049761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:12:50.049761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:50.049761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:12:50.049761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:50.049761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:12:50.049761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:50.049761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:12:50.049761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:50.049761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:12:50.049761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:50.049761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:12:50.049761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:12:50.050189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:50.050189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:12:50.050189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:50.050189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:12:50.050189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:12:50.050189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:50.050189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:12:50.050189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:50.050189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:12:50.050601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:50.050601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:12:50.050601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:50.050601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:12:50.050601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:12:50.050601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:12:50.050601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:12:50.050601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:12:50.050601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:12:50.050601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:12:50.050601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:50.050601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:50.050989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:50.050989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:12:50.050989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:50.050989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:50.050989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:50.050989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:12:50.050989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:50.050989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:12:50.050989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:12:50.050989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:50.050989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:12:50.050989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:50.050989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:12:50.050989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:12:50.051395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:50.051395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:12:50.051395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:50.051395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:12:50.051533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:50.051533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:12:50.051533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:12:50.051533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:12:50.051533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:12:50.051533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:12:50.051533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:50.051533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:12:50.051533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:12:50.051533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:12:50.051533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:12:50.051533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:12:50.051533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:50.051533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:50.051533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:12:50.051533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:12:50.051961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:12:50.051961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:12:50.051961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:12:50.051961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:12:50.051961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:12:50.051961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:50.051961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:50.051961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:50.051961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:12:50.051961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:50.051961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:50.051961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:50.051961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:12:50.051961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:50.051961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:12:50.051961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:12:50.051961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:50.051961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:12:50.052954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:50.052954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:12:50.052954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:12:50.052954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:50.052954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:12:50.052954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:50.052954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:12:50.053147 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:12:50.053337 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:12:50.053516 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:12:50.053645 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:12:50.054654 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:12:50.055019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:50.055749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:12:50.055749 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:50.055749 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:12:50.055749 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:12:50.055749 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:50.055749 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:12:50.055749 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:50.055749 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:12:50.057001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:50.057001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:12:50.057001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:12:50.057001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:12:50.057001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:50.057001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:12:50.057001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:50.057001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:12:50.057001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:50.057001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:12:50.057001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:50.057001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:12:50.057001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:50.057001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:12:50.057001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:12:50.057452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:50.057452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:12:50.057452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:50.057452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:12:50.057452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:12:50.057452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:50.057452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:12:50.057452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:50.057452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:12:50.057922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:50.057922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:12:50.057922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:50.057922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:12:50.057922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:12:50.057922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:12:50.057922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:12:50.057922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:12:50.057922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:12:50.057922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:50.057922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:50.058256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:50.058256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:12:50.058256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:50.058256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:50.058256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:50.058256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:12:50.058256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:50.058256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:12:50.058256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:12:50.058256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:50.058256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:12:50.058256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:50.058256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:12:50.058256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:12:50.058688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:50.058688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:12:50.058688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:50.058688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:12:50.058943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:50.058943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:12:50.058943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:12:50.058943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:12:50.058943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:12:50.058943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:12:50.058943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:50.058943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:12:50.058943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:12:50.058943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:12:50.058943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:12:50.058943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:12:50.058943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:50.058943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:50.058943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:12:50.058943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:12:50.058943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:12:50.059453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:12:50.059453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:12:50.059453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:12:50.059453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:50.059453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:50.059453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:50.059453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:12:50.059453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:50.059453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:50.059453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:50.059453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:12:50.059453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:50.059453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:12:50.059453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:12:50.059453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:50.059453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:12:50.059453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:50.059453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:12:50.059453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:12:50.060067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:50.060067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:12:50.060067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:50.060067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:12:50.060067 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:12:50.060067 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:12:50.060067 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:12:50.060988 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:12:50.061496 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:12:50.061869 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:12:50.062142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:50.063130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:12:50.063130 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:50.063130 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:12:50.063130 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:12:50.063130 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:50.063130 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:12:50.063130 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:50.063130 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:12:50.063688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:50.063688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:12:50.063688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:50.063688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:12:50.063688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:12:50.063688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:12:50.063688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:12:50.063688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:50.063688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:50.063688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:50.063688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:12:50.063688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:50.064147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:50.064147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:50.064147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:12:50.064147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:50.064147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:12:50.064147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:12:50.064147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:50.064147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:12:50.064147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:50.064147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:12:50.064147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:12:50.064147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:50.064147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:12:50.064534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:50.064534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:12:50.064600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:50.064600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:12:50.064600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:12:50.064600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:12:50.064600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:12:50.064600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:12:50.064600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:50.064600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:12:50.064600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:12:50.064600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:12:50.064600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:12:50.064600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:12:50.064600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:50.064600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:50.064600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:12:50.064600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:12:50.065028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:12:50.065028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:12:50.065028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:50.065028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:50.065028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:50.065028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:12:50.065028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:50.065028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:50.065028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:50.065028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:12:50.065028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:50.065028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:12:50.065028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:12:50.065028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:50.065028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:12:50.065028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:50.065028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:12:50.065028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:12:50.065028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:50.065570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:12:50.065570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:50.065570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:12:54.800172 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:12:54.801813 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:12:54.801813 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:12:54.802135 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:12:54.838152 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:12:54.838609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:54.839430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:12:54.839430 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:54.839430 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:12:54.839430 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:12:54.839430 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:54.839430 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:12:54.839430 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:54.839430 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:12:54.840128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:54.840128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:12:54.840128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:54.840128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:12:54.840128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:12:54.840128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:12:54.840128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:12:54.840128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:54.840128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:54.840128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:54.840128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:12:54.840128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:54.840749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:54.840749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:54.840749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:12:54.840749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:54.840749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:12:54.840749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:12:54.840749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:54.840749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:12:54.840749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:54.840749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:12:54.840749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:12:54.840749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:54.840749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:12:54.841316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:54.841316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:12:54.841316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:54.841316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:12:54.841316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:12:54.841316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:12:54.841316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:12:54.841316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:12:54.841316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:54.841316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:12:54.841316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:12:54.841316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:12:54.841316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:12:54.841316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:12:54.841316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:54.842044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:54.842044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:12:54.842044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:12:54.842044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:12:54.842044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:12:54.842044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:54.842044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:54.842044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:54.842044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:12:54.842044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:54.842044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:54.842044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:54.842044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:12:54.842044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:54.842044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:12:54.842044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:12:54.842044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:54.842044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:12:54.842044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:54.842937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:12:54.842937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:12:54.842937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:54.842937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:12:54.842937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:54.842937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:12:55.038691 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:12:55.039334 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:12:55.039561 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:12:55.040565 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:12:55.040814 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:12:55.040965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:55.041717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:12:55.041717 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:55.041717 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:12:55.041717 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:12:55.041717 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:55.041717 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:12:55.041717 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:55.041717 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:12:55.043213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:55.043213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:12:55.043213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:55.043213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:12:55.043213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:12:55.043213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:12:55.043213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:12:55.043213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:55.043213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:55.043213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:55.043213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:12:55.043213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:55.043861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:55.043861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:55.043861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:12:55.043861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:55.043861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:12:55.043861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:12:55.043861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:55.043861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:12:55.043861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:55.043861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:12:55.043861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:12:55.043861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:55.043861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:12:55.044463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:55.044463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:12:55.045327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:55.045327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:12:55.045327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:12:55.045327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:12:55.045327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:12:55.045327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:12:55.045327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:55.045327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:12:55.045327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:12:55.045327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:12:55.045327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:12:55.045327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:12:55.045327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:55.045327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:55.045327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:12:55.045327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:12:55.046121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:12:55.046121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:12:55.046121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:55.046121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:55.046121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:55.046121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:12:55.046121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:55.046121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:55.046121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:55.046121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:12:55.046121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:55.046121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:12:55.046121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:12:55.046121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:55.046121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:12:55.046121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:55.046121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:12:55.046121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:12:55.046121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:55.047036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:12:55.047036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:55.047036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:12:55.047185 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:12:55.047637 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:12:55.047757 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:12:55.047939 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:12:55.048280 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:12:55.048495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:55.049075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:12:55.049075 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:55.049075 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:12:55.049075 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:12:55.049075 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:55.049075 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:12:55.049075 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:55.049075 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:12:55.050003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:55.050003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:12:55.050003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:12:55.050003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:12:55.050003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:55.050003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:12:55.050003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:55.050003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:12:55.050003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:55.050003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:12:55.050003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:55.050003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:12:55.050003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:55.050003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:12:55.050003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:12:55.050687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:55.050687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:12:55.050687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:55.050687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:12:55.050687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:12:55.050687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:55.050687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:12:55.050687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:55.050687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:12:55.050951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:55.050951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:12:55.050951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:55.050951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:12:55.050951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:12:55.050951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:12:55.050951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:12:55.050951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:12:55.050951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:12:55.050951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:12:55.050951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:55.050951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:55.051296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:55.051296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:12:55.051296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:55.051296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:55.051296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:55.051296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:12:55.051296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:55.051296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:12:55.051296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:12:55.051296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:55.051296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:12:55.051296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:55.051296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:12:55.051296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:12:55.051667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:55.051667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:12:55.051667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:55.051667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:12:55.051853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:55.051853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:12:55.051853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:12:55.051853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:12:55.051853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:12:55.051853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:12:55.051853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:55.051853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:12:55.051853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:12:55.051853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:12:55.051853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:12:55.051853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:12:55.051853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:55.051853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:55.051853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:12:55.051853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:12:55.052271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:12:55.052271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:12:55.052271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:12:55.052271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:12:55.052271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:12:55.052271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:55.052271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:55.052271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:55.052271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:12:55.052271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:55.052271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:55.052271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:55.052271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:12:55.052271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:55.052271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:12:55.052271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:12:55.052271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:55.052271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:12:55.052760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:55.052760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:12:55.052760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:12:55.052760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:55.052760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:12:55.052760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:55.052760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:12:55.053579 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:12:55.053885 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:12:55.054042 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:12:55.054204 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:12:55.054663 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:12:55.055018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:55.055731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:12:55.055731 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:55.055731 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:12:55.055731 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:12:55.055731 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:55.055731 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:12:55.055731 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:55.055731 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:12:55.056550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:55.056550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:12:55.056550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:12:55.056550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:12:55.056550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:55.056550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:12:55.056550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:55.056550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:12:55.056550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:55.056550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:12:55.056550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:55.056550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:12:55.056550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:55.056550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:12:55.056550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:12:55.056941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:55.056941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:12:55.056941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:55.056941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:12:55.056941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:12:55.056941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:55.056941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:12:55.056941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:55.056941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:12:55.057688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:55.057688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:12:55.057688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:55.057688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:12:55.057688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:12:55.057688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:12:55.057688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:12:55.057688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:12:55.057688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:12:55.057688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:55.057688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:55.058021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:55.058021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:12:55.058021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:55.058021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:55.058021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:55.058021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:12:55.058021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:55.058021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:12:55.058021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:12:55.058021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:55.058021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:12:55.058021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:55.058021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:12:55.058021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:12:55.058359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:55.058359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:12:55.058359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:55.058359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:12:55.059094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:55.059094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:12:55.059094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:12:55.059094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:12:55.059094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:12:55.059094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:12:55.059094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:55.059094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:12:55.059094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:12:55.059094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:12:55.059094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:12:55.059094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:12:55.059094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:55.059094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:55.059094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:12:55.059094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:12:55.059094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:12:55.059604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:12:55.059604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:12:55.059604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:12:55.059604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:55.059604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:55.059604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:55.059604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:12:55.059604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:55.059604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:55.059604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:55.059604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:12:55.059604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:55.059604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:12:55.059604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:12:55.059604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:55.059604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:12:55.059604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:55.059604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:12:55.059604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:12:55.060138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:55.060138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:12:55.060138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:55.060138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:12:55.060138 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:12:55.060138 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:12:55.060138 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:12:55.060138 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:12:55.060648 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:12:55.060995 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:12:55.061218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:55.061972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:12:55.061972 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:55.061972 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:12:55.061972 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:12:55.061972 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:55.061972 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:12:55.061972 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:55.061972 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:12:55.062852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:55.062852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:12:55.062852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:55.062852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:12:55.062852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:12:55.062852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:12:55.062852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:12:55.062852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:55.062852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:55.062852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:55.062852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:12:55.062852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:55.063193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:55.063193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:55.063193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:12:55.063193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:55.063193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:12:55.063193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:12:55.063193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:55.063193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:12:55.063193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:55.063193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:12:55.063193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:12:55.063193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:55.063193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:12:55.063539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:55.063539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:12:55.064023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:55.064023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:12:55.064023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:12:55.064023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:12:55.064023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:12:55.064023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:12:55.064023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:55.064023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:12:55.064023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:12:55.064023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:12:55.064023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:12:55.064023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:12:55.064023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:55.064023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:55.064023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:12:55.064023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:12:55.064618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:12:55.064618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:12:55.064618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:55.064618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:55.064618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:55.064618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:12:55.064618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:55.064618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:55.064618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:55.064618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:12:55.064618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:55.064618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:12:55.064618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:12:55.064618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:55.064618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:12:55.064618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:55.064618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:12:55.064618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:12:55.064618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:55.065206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:12:55.065206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:55.065206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:12:59.800734 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:12:59.801394 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:12:59.801394 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:12:59.802204 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:12:59.844315 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:12:59.844918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:59.845971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:12:59.845971 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:59.845971 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:12:59.845971 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:12:59.845971 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:59.845971 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:12:59.845971 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:59.845971 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:12:59.847452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.045 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:59.847452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:12:59.847452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:59.847452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:12:59.847452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:12:59.847452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:12:59.847452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:12:59.847452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:59.847452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:59.847452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:59.847452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:12:59.847452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:59.847839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:12:59.847839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:59.847839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:12:59.847839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:59.847839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:12:59.847839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:12:59.847839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:59.847839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:12:59.847839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:59.847839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:12:59.847839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:12:59.847839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:59.847839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:12:59.848230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:59.848230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:12:59.849011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:59.849011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:12:59.849011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:12:59.849011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:12:59.849011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:12:59.849011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:12:59.849011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:59.849011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:12:59.849011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:12:59.849011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:12:59.849011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:12:59.849011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:12:59.849011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:12:59.849011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:12:59.849011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:12:59.849011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:12:59.851136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:12:59.851136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:12:59.851136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:59.851136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:59.851136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:12:59.851136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:12:59.851136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:12:59.851136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:12:59.851136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:12:59.851136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:12:59.851136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:12:59.851136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:12:59.851136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:12:59.851136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:12:59.851136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:12:59.851136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:12:59.851136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:12:59.851136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:12:59.851136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:12:59.851715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:12:59.851715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:12:59.851715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:13:00.039762 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:13:00.040952 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:13:00.040952 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:13:00.042712 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:13:00.042913 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:13:00.043030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:00.043644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:13:00.043644 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:00.043644 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:13:00.043644 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:13:00.043644 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:00.043644 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:13:00.043644 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:00.043644 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:13:00.045204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:00.045204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:13:00.045204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:00.045204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:13:00.045204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:13:00.045204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:13:00.045204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:13:00.045204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:00.045204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:00.045204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:00.045204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:13:00.045204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:00.045652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:00.045652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:00.045652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:13:00.045652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:00.045652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:13:00.045652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:13:00.045652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:00.045652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:13:00.045652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:00.045652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:13:00.045652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:13:00.045652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:00.045652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:13:00.046016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:00.046016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:13:00.047516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:00.047516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:13:00.047516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:13:00.047516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:13:00.047516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:13:00.047516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:13:00.047516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:00.047516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:13:00.047516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:13:00.047516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:13:00.047516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:13:00.047516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:13:00.047516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:00.047516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:00.047516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:13:00.047516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:13:00.047954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:13:00.047954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:13:00.047954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:00.047954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:00.047954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:00.047954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:13:00.047954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:00.047954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:00.047954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:00.047954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:13:00.047954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:00.047954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:13:00.047954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:13:00.047954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:00.047954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:13:00.047954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:00.047954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:13:00.047954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:13:00.047954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:00.048473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:13:00.048473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:00.048473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:13:00.049539 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:13:00.050009 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:13:00.050009 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:13:00.050159 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:13:00.050456 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:13:00.050616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:00.051942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:13:00.051942 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:00.051942 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:13:00.051942 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:13:00.051942 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:00.051942 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:13:00.051942 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:00.051942 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:13:00.052269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:00.052269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:13:00.052269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:13:00.052269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:13:00.052269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:00.052269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:13:00.052269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:00.052269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:13:00.052269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:00.052269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:13:00.052269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:00.052269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:13:00.052269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:00.052269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:13:00.052269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:13:00.052746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:00.052746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:13:00.052746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:00.052746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:13:00.052746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:13:00.052746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:00.052746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:13:00.052746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:00.052746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:13:00.053666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:00.053666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:13:00.053666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:00.053666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:13:00.053666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:13:00.053666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:13:00.053666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:13:00.053666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:13:00.053666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:13:00.053666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:13:00.053666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:00.053666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:00.054030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:00.054030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:13:00.054030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:00.054030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:00.054030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:00.054030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:13:00.054030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:00.054030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:13:00.054030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:13:00.054030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:00.054030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:13:00.054030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:00.054030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:13:00.054030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:13:00.054374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:00.054374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:13:00.054374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:00.054374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:13:00.055587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:00.055587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:13:00.055587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:13:00.055587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:13:00.055587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:13:00.055587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:13:00.055587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:00.055587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:13:00.055587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:13:00.055587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:13:00.055587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:13:00.055587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:13:00.055587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:00.055587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:00.055587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:13:00.055587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:13:00.056024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:13:00.056024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:13:00.056024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:13:00.056024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:13:00.056024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:13:00.056024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:00.056024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:00.056024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:00.056024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:13:00.056024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:00.056024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:00.056024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:00.056024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:13:00.056024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:00.056024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:13:00.056024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:13:00.056024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:00.056024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:13:00.056521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:00.056521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:13:00.056521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:13:00.056521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:00.056521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:13:00.056521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:00.056521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:13:00.056521 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:13:00.056521 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:13:00.056521 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:13:00.056803 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:13:00.056803 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:13:00.056803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:00.057143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:13:00.057143 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:00.057143 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:13:00.057143 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:13:00.057143 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:00.057143 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:13:00.057143 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:00.057143 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:13:00.058631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:00.058631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:13:00.058631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:13:00.058631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:13:00.058631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:00.058631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:13:00.058631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:00.058631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:13:00.058631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:00.058631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:13:00.058631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:00.058631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:13:00.058631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:00.058631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:13:00.058631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:13:00.059050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:00.059050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:13:00.059050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:00.059050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:13:00.059050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:13:00.059050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:00.059050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:13:00.059050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:00.059050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:13:00.059050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:00.059050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:13:00.059050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:00.059050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:13:00.059050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:13:00.059480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:13:00.059480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:13:00.059480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:13:00.059480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:13:00.059480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:00.059480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:00.059480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:00.059480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:13:00.059480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:00.059480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:00.059480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:00.059480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:13:00.059480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:00.059844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:13:00.059844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:13:00.059844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:00.059844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:13:00.059844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:00.059844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:13:00.059844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:13:00.059844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:00.059844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:13:00.059844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:00.059844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:13:00.060158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:00.060158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:13:00.060158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:13:00.060158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:13:00.060158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:13:00.060158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:13:00.060158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:00.060158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:13:00.060158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:13:00.060158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:13:00.060158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:13:00.060158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:13:00.060158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:00.060158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:00.060158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:13:00.060158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:13:00.060158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:13:00.060631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:13:00.060631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:13:00.060631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:13:00.060631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:00.060631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:00.060631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:00.060631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:13:00.060631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:00.060631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:00.060631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:00.060631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:13:00.060631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:00.060631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:13:00.060631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:13:00.060631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:00.060631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:13:00.060631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:00.060631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:13:00.060631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:13:00.061137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:00.061137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:13:00.061137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:00.061137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:13:00.061137 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:13:00.061137 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:13:00.061137 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:13:00.061137 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:13:00.061386 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:13:00.061386 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:13:00.061689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:00.062112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:13:00.062112 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:00.062112 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:13:00.062112 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:13:00.062112 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:00.062112 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:13:00.062112 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:00.062112 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:13:00.062871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:00.062871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:13:00.062871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:00.062871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:13:00.062871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:13:00.062871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:13:00.062871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:13:00.062871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:00.062871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:00.062871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:00.062871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:13:00.062871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:00.063192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:00.063192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:00.063192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:13:00.063192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:00.063192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:13:00.063192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:13:00.063192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:00.063192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:13:00.063192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:00.063192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:13:00.063192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:13:00.063192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:00.063192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:13:00.063557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:00.063557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:13:00.063557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:00.063557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:13:00.063557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:13:00.063557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:13:00.063557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:13:00.063557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:13:00.063557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:00.063557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:13:00.063557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:13:00.063557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:13:00.063557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:13:00.063557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:13:00.063557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:00.063955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:00.063955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:13:00.063955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:13:00.063955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:13:00.063955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:13:00.063955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:00.063955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:00.063955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:00.063955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:13:00.063955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:00.063955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:00.063955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:00.063955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:13:00.063955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:00.063955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:13:00.063955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:13:00.063955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:00.063955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:13:00.063955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:00.064482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:13:00.064482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:13:00.064482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:00.064482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:13:00.064482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:00.064482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:13:04.802199 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:13:04.803051 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:13:04.803051 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:13:04.804507 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:13:04.840346 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:13:04.840346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:04.840346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:13:04.840346 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:04.840346 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:13:04.840346 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:13:04.840346 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:04.840346 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:13:04.840346 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:04.840346 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:13:04.841424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:04.841424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:13:04.841424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:04.841424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:13:04.841424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:13:04.841424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:13:04.841424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:13:04.841424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:04.841424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:04.841424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:04.841424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:13:04.841424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:04.842065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:04.842065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:04.842065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:13:04.842065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:04.842065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:13:04.842065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:13:04.842065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:04.842065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:13:04.842065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:04.842065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:13:04.842065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:13:04.842065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:04.842065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:13:04.842659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:04.842659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:13:04.842659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:04.842659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:13:04.842659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:13:04.842659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:13:04.842659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:13:04.842659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:13:04.842659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:04.842659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:13:04.842659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:13:04.842659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:13:04.842659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:13:04.842659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:13:04.842659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:04.843354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:04.843354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:13:04.843354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:13:04.843354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:13:04.843354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:13:04.843354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:04.843354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:04.843354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:04.843354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:13:04.843354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:04.843354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:04.843354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:04.843354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:13:04.843354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:04.843354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:13:04.843354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:13:04.843354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:04.843354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:13:04.843354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:04.844262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:13:04.844262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:13:04.844262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:04.844262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:13:04.844262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:04.844262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:13:05.040356 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:13:05.041233 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:13:05.041372 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:13:05.042309 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:13:05.042717 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:13:05.042940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:05.043831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:13:05.043831 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:05.043831 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:13:05.043831 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:13:05.043831 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:05.043831 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:13:05.043831 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:05.043831 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:13:05.045637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:05.045637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:13:05.045637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:05.045637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:13:05.045637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:13:05.045637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:13:05.045637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:13:05.045637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:05.045637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:05.045637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:05.045637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:13:05.045637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:05.046305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:05.046305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:05.046305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:13:05.046305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:05.046305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:13:05.046305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:13:05.046305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:05.046305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:13:05.046305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:05.046305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:13:05.046305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:13:05.046305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:05.046305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:13:05.046924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:05.046924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:13:05.046924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:05.046924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:13:05.046924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:13:05.046924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:13:05.046924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:13:05.046924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:13:05.046924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:05.046924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:13:05.046924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:13:05.046924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:13:05.046924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:13:05.046924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:13:05.046924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:05.047706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:05.047706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:13:05.047706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:13:05.047706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:13:05.047706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:13:05.047706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:05.047706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:05.047706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:05.047706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:13:05.047706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:05.047706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:05.047706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:05.047706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:13:05.047706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:05.047706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:13:05.047706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:13:05.047706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:05.047706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:13:05.047706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:05.048624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:13:05.048624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:13:05.048624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:05.048624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:13:05.048624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:05.048624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:13:05.049953 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:13:05.050291 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:13:05.050398 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:13:05.050562 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:13:05.050985 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:13:05.051274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:05.052001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:13:05.052001 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:05.052001 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:13:05.052001 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:13:05.052001 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:05.052001 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:13:05.052001 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:05.052001 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:13:05.052800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:05.052800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:13:05.052800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:13:05.052800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:13:05.052800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:05.052800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:13:05.052800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:05.052800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:13:05.052800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:05.052800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:13:05.052800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:05.052800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:13:05.052800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:05.052800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:13:05.052800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:13:05.053284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:05.053284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:13:05.053284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:05.053284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:13:05.053284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:13:05.053284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:05.053284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:13:05.053284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:05.053284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:13:05.053577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:05.053577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:13:05.053577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:05.053577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:13:05.053577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:13:05.053577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:13:05.053577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:13:05.053577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:13:05.053577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:13:05.053577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:13:05.053577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:05.053577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:05.053918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:05.053918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:13:05.053918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:05.053918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:05.053918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:05.053918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:13:05.053918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:05.053918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:13:05.053918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:13:05.053918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:05.053918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:13:05.053918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:05.053918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:13:05.053918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:13:05.054258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:05.054258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:13:05.054258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:05.054258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:13:05.054556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:05.054556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:13:05.054556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:13:05.054556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:13:05.054556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:13:05.054556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:13:05.054556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:05.054556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:13:05.054556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:13:05.054556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:13:05.054556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:13:05.054556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:13:05.054556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:05.054556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:05.054556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:13:05.054556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:13:05.054969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:13:05.054969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:13:05.054969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:13:05.054969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:13:05.054969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:13:05.054969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:05.054969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:05.054969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:05.054969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:13:05.054969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:05.054969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:05.054969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:05.054969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:13:05.054969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:05.054969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:13:05.054969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:13:05.054969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:05.054969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:13:05.055594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:05.055594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:13:05.055594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:13:05.055594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:05.055594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:13:05.055594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:05.055594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:13:05.056084 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:13:05.057076 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:13:05.057076 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:13:05.057076 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:13:05.057225 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:13:05.057536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:05.058557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:13:05.058557 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:05.058557 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:13:05.058557 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:13:05.058557 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:05.058557 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:13:05.058557 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:05.058557 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:13:05.058896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:05.058896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:13:05.058896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:13:05.058896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:13:05.058896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:05.058896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:13:05.058896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:05.058896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:13:05.058896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:05.058896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:13:05.058896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:05.058896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:13:05.058896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:05.058896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:13:05.058896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:13:05.059297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:05.059297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:13:05.059297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:05.059297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:13:05.059297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:13:05.059297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:05.059297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:13:05.059297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:05.059297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:13:05.061642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:05.061642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:13:05.061642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:05.061642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:13:05.061642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:13:05.061642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:13:05.061642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:13:05.061642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:13:05.061642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:13:05.061642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:05.061642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:05.061967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:05.061967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:13:05.061967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:05.061967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:05.061967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:05.061967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:13:05.061967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:05.061967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:13:05.061967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:13:05.061967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:05.061967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:13:05.061967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:05.061967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:13:05.061967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:13:05.062293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:05.062293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:13:05.062293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:05.062293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:13:05.062293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:05.062293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:13:05.062293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:13:05.062293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:13:05.062293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:13:05.062293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:13:05.062293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:05.062293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:13:05.062293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:13:05.062293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:13:05.062683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:13:05.062683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:13:05.062683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:05.062683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:05.062683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:13:05.062683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:13:05.062683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:13:05.062683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:13:05.062683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:13:05.062683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:13:05.062683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:05.062683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:05.062683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:05.062683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:13:05.062683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:05.062683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:05.062683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:05.062683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:13:05.062683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:05.063147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:13:05.063147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:13:05.063147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:05.063147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:13:05.063147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:05.063147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:13:05.063147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:13:05.063147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:05.063147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:13:05.063147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:05.063147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:13:05.063147 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:13:05.063147 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:13:05.063535 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:13:05.063535 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:13:05.063535 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:13:05.063535 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:13:05.063535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:05.063535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:13:05.063535 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:05.063535 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:13:05.063535 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:13:05.063805 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:05.063805 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:13:05.063805 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:05.063805 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:13:05.064099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:05.064099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:13:05.064099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:05.064099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:13:05.064099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:13:05.064099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:13:05.064099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:13:05.064099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:05.064099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:05.064099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:05.064099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:13:05.064099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:05.064481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:05.064481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:05.064481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:13:05.064481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:05.064481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:13:05.064481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:13:05.064481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:05.064481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:13:05.064481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:05.064481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:13:05.064481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:13:05.064481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:05.064481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:13:05.064810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:05.064810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:13:05.065075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:05.065075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:13:05.065075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:13:05.065075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:13:05.065075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:13:05.065075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:13:05.065075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:05.065075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:13:05.065075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:13:05.065075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:13:05.065075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:13:05.065075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:13:05.065075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:05.065075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:05.065075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:13:05.065075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:13:05.065525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:13:05.065525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:13:05.065525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:05.065525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:05.065525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:05.065525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:13:05.065525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:05.065525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:05.065525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:05.065525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:13:05.065525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:05.065525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:13:05.065525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:13:05.065525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:05.065525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:13:05.065525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:05.065525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:13:05.065525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:13:05.065525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:05.066025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:13:05.066025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:05.066025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:13:09.802886 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:13:09.803431 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:13:09.803431 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:13:09.804107 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:13:09.837302 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:13:09.837806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:09.838640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:13:09.838640 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:09.838640 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:13:09.838640 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:13:09.838640 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:09.838640 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:13:09.838640 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:09.838640 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:13:09.839728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:09.839728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:13:09.839728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:09.839728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:13:09.839728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:13:09.839728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:13:09.839728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:13:09.839728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:09.839728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:09.839728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:09.839728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:13:09.839728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:09.840154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:09.840154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:09.840154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:13:09.840154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:09.840154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:13:09.840154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:13:09.840154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:09.840154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:13:09.840154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:09.840154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:13:09.840154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:13:09.840154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:09.840154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:13:09.840539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:09.840539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:13:09.840795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:09.840795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:13:09.840795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:13:09.840795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:13:09.840795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:13:09.840795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:13:09.840795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:09.840795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:13:09.840795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:13:09.840795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:13:09.840795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:13:09.840795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:13:09.840795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:09.840795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:09.840795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:13:09.840795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:13:09.841273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:13:09.841273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:13:09.841273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:09.841273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:09.841273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:09.841273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:13:09.841273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:09.841273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:09.841273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:09.841273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:13:09.841273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:09.841273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:13:09.841273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:13:09.841273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:09.841273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:13:09.841273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:09.841273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:13:09.841273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:13:09.841273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:09.842326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:13:09.842326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:09.842326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:13:10.042984 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:13:10.042984 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:13:10.042984 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:13:10.042984 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:13:10.042984 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:13:10.042984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:10.044879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:13:10.044879 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:10.044879 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:13:10.044879 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:13:10.044879 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:10.044879 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:13:10.044879 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:10.044879 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:13:10.045137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:10.045137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:13:10.045137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:10.045137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:13:10.045137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:13:10.045137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:13:10.045137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:13:10.045137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:10.045137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:10.045137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:10.045137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:13:10.045137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:10.046091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:10.046091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:10.046091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:13:10.046091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:10.046091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:13:10.046091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:13:10.046091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:10.046091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:13:10.046091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:10.046091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:13:10.046091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:13:10.046091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:10.046091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:13:10.046510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:10.046510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:13:10.046844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:10.046844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:13:10.046844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:13:10.046844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:13:10.046844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:13:10.046844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:13:10.046844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:10.046844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:13:10.046844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:13:10.046844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:13:10.046844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:13:10.046844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:13:10.046844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:10.046844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:10.046844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:13:10.046844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:13:10.047452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:13:10.047452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:13:10.047452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:10.047452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:10.047452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:10.047452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:13:10.047452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:10.047452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:10.047452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:10.047452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:13:10.047452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:10.047452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:13:10.047452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:13:10.047452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:10.047452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:13:10.047452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:10.047452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:13:10.047452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:13:10.047452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:10.048038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:13:10.048038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:10.048038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:13:10.050374 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:13:10.050724 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:13:10.050913 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:13:10.051022 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:13:10.051503 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:13:10.051790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:10.052508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:13:10.052508 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:10.052508 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:13:10.052508 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:13:10.052508 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:10.052508 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:13:10.052508 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:10.052508 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:13:10.053229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:10.053229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:13:10.053229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:13:10.053229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:13:10.053229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:10.053229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:13:10.053229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:10.053229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:13:10.053229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:10.053229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:13:10.053229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:10.053229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:13:10.053229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:10.053229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:13:10.053229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:13:10.053571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:10.053571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:13:10.053571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:10.053571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:13:10.053571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:13:10.053571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:10.053571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:13:10.053571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:10.053571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:13:10.053983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:10.053983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:13:10.053983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:10.053983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:13:10.053983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:13:10.053983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:13:10.053983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:13:10.053983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:13:10.053983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:13:10.053983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:13:10.053983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:10.053983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:10.054265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:10.054265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:13:10.054265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:10.054265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:10.054265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:10.054265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:13:10.054265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:10.054265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:13:10.054265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:13:10.054265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:10.054265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:13:10.054265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:10.054265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:13:10.054265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:13:10.055006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:10.055006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:13:10.055006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:10.055006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:13:10.055006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:10.055006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:13:10.055006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:13:10.055006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:13:10.055006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:13:10.055006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:13:10.055006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:10.055006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:13:10.055006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:13:10.055006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:13:10.055317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:13:10.055317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:13:10.055317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:10.055317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:10.055317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:13:10.055317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:13:10.055317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:13:10.055317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:13:10.055317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:13:10.055317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:13:10.055317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:13:10.055317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:10.055317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:10.055317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:10.055317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:13:10.055317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:10.055317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:10.055317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:10.055317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:13:10.055717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:10.055717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:13:10.055717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:13:10.055717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:10.055717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:13:10.055717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:10.055717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:13:10.055717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:13:10.055717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:10.055717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:13:10.055717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:10.055717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:13:10.056529 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:13:10.056763 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:13:10.056871 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:13:10.057001 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:13:10.057431 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:13:10.058155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:10.058841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:13:10.058841 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:10.058841 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:13:10.058841 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:13:10.058841 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:10.058841 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:13:10.058841 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:10.058841 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:13:10.059538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:10.059538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:13:10.059538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:13:10.059538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:13:10.059538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:10.059538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:13:10.059538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:10.059538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:13:10.059538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:10.059538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:13:10.059538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:10.059538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:13:10.059538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:10.059538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:13:10.059538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:13:10.059953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:10.059953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:13:10.059953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:10.059953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:13:10.059953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:13:10.059953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:10.059953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:13:10.059953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:10.059953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:13:10.060383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:10.060383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:13:10.060383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:10.060383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:13:10.060383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:13:10.060383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:13:10.060383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:13:10.060383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:13:10.060383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:13:10.060383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:10.060383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:10.060748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:10.060748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:13:10.060748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:10.060748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:10.060748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:10.060748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:13:10.060748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:10.060748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:13:10.060748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:13:10.060748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:10.060748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:13:10.060748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:10.060748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:13:10.060748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:13:10.061134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:10.061134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:13:10.061134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:10.061134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:13:10.061425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:10.061425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:13:10.061425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:13:10.061425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:13:10.061425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:13:10.061425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:13:10.061425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:10.061425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:13:10.061425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:13:10.061425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:13:10.061425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:13:10.061425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:13:10.061425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:10.061425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:10.061425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:13:10.061425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:13:10.061425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:13:10.061908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:13:10.061908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:13:10.061908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:13:10.061908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:10.061908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:10.061908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:10.061908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:13:10.061908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:10.061908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:10.061908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:10.061908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:13:10.061908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:10.061908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:13:10.061908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:13:10.061908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:10.061908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:13:10.061908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:10.061908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:13:10.061908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:13:10.062486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:10.062486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:13:10.062486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:10.062486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:13:10.062486 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:13:10.062486 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:13:10.062486 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:13:10.062486 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:13:10.062897 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:13:10.063252 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:13:10.063528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:10.064253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:13:10.064253 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:10.064253 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:13:10.064253 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:13:10.064253 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:10.064253 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:13:10.064253 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:10.064253 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:13:10.065109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:10.065109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:13:10.065109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:10.065109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:13:10.065109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:13:10.065109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:13:10.065109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:13:10.065109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:10.065109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:10.065109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:10.065109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:13:10.065109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:10.065504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:10.065504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:10.065504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:13:10.065504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:10.065504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:13:10.065504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:13:10.065504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:10.065504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:13:10.065504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:10.065504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:13:10.065504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:13:10.065504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:10.065504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:13:10.065872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:10.065872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:13:10.065966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:10.065966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:13:10.065966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:13:10.065966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:13:10.065966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:13:10.065966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:13:10.065966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:10.065966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:13:10.065966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:13:10.065966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:13:10.065966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:13:10.065966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:13:10.065966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:10.065966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:10.065966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:13:10.065966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:13:10.066469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:13:10.066469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:13:10.066469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:10.066469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:10.066469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:10.066469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:13:10.066469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:10.066469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:10.066469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:10.066469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:13:10.066469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:10.066469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:13:10.066469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:13:10.066469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:10.066469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:13:10.066469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:10.066469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:13:10.066469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:13:10.066469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:10.066992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:13:10.066992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:10.066992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:13:14.803489 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:13:14.804611 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:13:14.804611 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:13:14.807688 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:13:14.835026 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:13:14.835315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:14.835869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:13:14.835869 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:14.835869 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:13:14.835869 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:13:14.835869 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:14.835869 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:13:14.835869 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:14.835869 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:13:14.836630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:14.836630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:13:14.836630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:14.836630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:13:14.836630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:13:14.836630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:13:14.836630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:13:14.836630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:14.836630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:14.836630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:14.836630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:13:14.836630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:14.837267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:14.837267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:14.837267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:13:14.837267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:14.837267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:13:14.837267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:13:14.837267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:14.837267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:13:14.837267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:14.837267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:13:14.837267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:13:14.837267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:14.837267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:13:14.838266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:14.838266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:13:14.838266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:14.838266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:13:14.838266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:13:14.838266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:13:14.838266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:13:14.838266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:13:14.838266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:14.838266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:13:14.838266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:13:14.838266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:13:14.838266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:13:14.838266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:13:14.838266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:14.839516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:14.839516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:13:14.839516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:13:14.839516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:13:14.839516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:13:14.839516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:14.839516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:14.839516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:14.839516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:13:14.839516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:14.839516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:14.839516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:14.839516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:13:14.839516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:14.839516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:13:14.839516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:13:14.839516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:14.839516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:13:14.839516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:14.840386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:13:14.840386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:13:14.840386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:14.840386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:13:14.840386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:14.840386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:13:15.041721 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:13:15.042200 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:13:15.042318 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:13:15.043138 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:13:15.043370 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:13:15.043512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:15.044016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:13:15.044016 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:15.044016 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:13:15.044016 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:13:15.044016 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:15.044016 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:13:15.044016 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:15.044016 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:13:15.044906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:15.044906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:13:15.044906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:15.044906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:13:15.044906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:13:15.044906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:13:15.044906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:13:15.044906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:15.044906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:15.044906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:15.044906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:13:15.044906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:15.045762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:15.045762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:15.045762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:13:15.045762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:15.045762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:13:15.045762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:13:15.045762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:15.045762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:13:15.045762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:15.045762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:13:15.045762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:13:15.045762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:15.045762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:13:15.046473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:15.046473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:13:15.046473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:15.046473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:13:15.046473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:13:15.046473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:13:15.046473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:13:15.046473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:13:15.046473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:15.046473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:13:15.046473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:13:15.046473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:13:15.046473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:13:15.046473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:13:15.046473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:15.047311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:15.047311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:13:15.047311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:13:15.047311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:13:15.047311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:13:15.047311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:15.047311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:15.047311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:15.047311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:13:15.047311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:15.047311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:15.047311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:15.047311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:13:15.047311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:15.047311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:13:15.047311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:13:15.047311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:15.047311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:13:15.047311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:15.048323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:13:15.048323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:13:15.048323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:15.048323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:13:15.048323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:15.048323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:13:15.050839 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:13:15.051112 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:13:15.051200 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:13:15.051321 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:13:15.051627 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:13:15.051746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:15.052165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:13:15.052165 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:15.052165 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:13:15.052165 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:13:15.052165 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:15.052165 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:13:15.052165 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:15.052165 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:13:15.052711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:15.052711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:13:15.052711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:13:15.052711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:13:15.052711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:15.052711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:13:15.052711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:15.052711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:13:15.052711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:15.052711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:13:15.052711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:15.052711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:13:15.052711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:15.052711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:13:15.052711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:13:15.053167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:15.053167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:13:15.053167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:15.053167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:13:15.053167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:13:15.053167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:15.053167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:13:15.053167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:15.053167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:13:15.053439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:15.053439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:13:15.053439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:15.053439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:13:15.053439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:13:15.053439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:13:15.053439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:13:15.053439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:13:15.053439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:13:15.053439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:13:15.053439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:15.053439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:15.053860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:15.053860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:13:15.053860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:15.053860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:15.053860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:15.053860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:13:15.053860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:15.053860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:13:15.053860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:13:15.053860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:15.053860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:13:15.053860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:15.053860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:13:15.053860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:13:15.054395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:15.054395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:13:15.054395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:15.054395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:13:15.054395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:15.054395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:13:15.054395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:13:15.054395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:13:15.054395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:13:15.054395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:13:15.054395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:15.054395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:13:15.054395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:13:15.054395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:13:15.054863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:13:15.054863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:13:15.054863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:15.054863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:15.054863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:13:15.054863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:13:15.054863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:13:15.054863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:13:15.054863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:13:15.054863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:13:15.054863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:13:15.054863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:15.054863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:15.054863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:15.054863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:13:15.054863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:15.054863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:15.054863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:15.054863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:13:15.055513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:15.055513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:13:15.055513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:13:15.055513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:15.055513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:13:15.055513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:15.055513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:13:15.055513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:13:15.055513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:15.055513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:13:15.055513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:15.055513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:13:15.057012 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:13:15.057250 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:13:15.057323 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:13:15.057432 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:13:15.057693 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:13:15.057855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:15.058236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:13:15.058236 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:15.058236 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:13:15.058236 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:13:15.058236 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:15.058236 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:13:15.058236 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:15.058236 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:13:15.058773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:15.058773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:13:15.058773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:13:15.058773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:13:15.058773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:15.058773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:13:15.058773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:15.058773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:13:15.058773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:15.058773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:13:15.058773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:15.058773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:13:15.058773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:15.058773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:13:15.058773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:13:15.059163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:15.059163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:13:15.059163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:15.059163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:13:15.059163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:13:15.059163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:15.059163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:13:15.059163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:15.059163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:13:15.059454 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:15.059454 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:13:15.059454 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:15.059454 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:13:15.059454 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:13:15.059454 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:13:15.059454 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:13:15.059454 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:13:15.059454 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:13:15.059454 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:15.059454 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:15.059807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:15.059807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:13:15.059807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:15.059807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:15.059807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:15.059807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:13:15.059807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:15.059807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:13:15.059807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:13:15.059807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:15.059807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:13:15.059807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:15.059807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:13:15.059807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:13:15.060186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:15.060186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:13:15.060186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:15.060186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:13:15.060186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:15.060186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:13:15.060186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:13:15.060186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:13:15.060186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:13:15.060186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:13:15.060186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:15.060186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:13:15.060186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:13:15.060186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:13:15.060580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:13:15.060580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:13:15.060580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:15.060580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:15.060580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:13:15.060580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:13:15.060580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:13:15.060580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:13:15.060580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:13:15.060580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:13:15.060580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:15.060580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:15.060580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:15.060580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:13:15.060580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:15.060580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:15.060580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:15.060580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:13:15.060580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:15.061110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:13:15.061110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:13:15.061110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:15.061110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:13:15.061110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:15.061110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:13:15.061110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:13:15.061110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:15.061110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:13:15.061110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:15.061110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:13:15.062305 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:13:15.062515 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:13:15.062589 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:13:15.062686 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:13:15.063203 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:13:15.063203 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:13:15.063203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:15.063622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:13:15.063622 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:15.063622 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:13:15.063622 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:13:15.063622 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:15.063622 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:13:15.063622 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:15.063622 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:13:15.064183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:15.064183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:13:15.064183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:15.064183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:13:15.064183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:13:15.064183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:13:15.064183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:13:15.064183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:15.064183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:15.064183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:15.064183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:13:15.064183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:15.064812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:15.064812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:15.064812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:13:15.064812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:15.064812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:13:15.064812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:13:15.064812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:15.064812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:13:15.064812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:15.064812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:13:15.064812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:13:15.064812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:15.064812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:13:15.065447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:15.065447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:13:15.065447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:15.065447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:13:15.065447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:13:15.065447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:13:15.065447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:13:15.065447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:13:15.065447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:15.065447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:13:15.065447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:13:15.065447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:13:15.065447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:13:15.065447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:13:15.065447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:15.066203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:15.066203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:13:15.066203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:13:15.066203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:13:15.066203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:13:15.066203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:15.066203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:15.066203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:15.066203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:13:15.066203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:15.066203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:15.066203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:15.066203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:13:15.066203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:15.066203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:13:15.066203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:13:15.066203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:15.066203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:13:15.066203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:15.067463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:13:15.067463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:13:15.067463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:15.067463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:13:15.067463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:15.067463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:13:19.803978 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:13:19.804909 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:13:19.804909 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:13:19.806507 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:13:19.836699 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:13:19.836953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:19.838248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:13:19.838248 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:19.838248 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:13:19.838248 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:13:19.838248 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:19.838248 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:13:19.838248 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:19.838248 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:13:19.838581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:19.838581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:13:19.838581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:19.838581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:13:19.838581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:13:19.838581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:13:19.838581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:13:19.838581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:19.838581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:19.838581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:19.838581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:13:19.838581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:19.839522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:19.839522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:19.839522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:13:19.839522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:19.839522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:13:19.839522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:13:19.839522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:19.839522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:13:19.839522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:19.839522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:13:19.839522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:13:19.839522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:19.839522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:13:19.840073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:19.840073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:13:19.840073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:19.840073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:13:19.840073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:13:19.840073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:13:19.840073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:13:19.840073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:13:19.840073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:19.840073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:13:19.840073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:13:19.840073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:13:19.840073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:13:19.840073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:13:19.840073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:19.840710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:19.840710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:13:19.840710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:13:19.840710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:13:19.840710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:13:19.840710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:19.840710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:19.840710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:19.840710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:13:19.840710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:19.840710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:19.840710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:19.840710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:13:19.840710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:19.840710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:13:19.840710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:13:19.840710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:19.840710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:13:19.840710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:19.841539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:13:19.841539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:13:19.841539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:19.841539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:13:19.841539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:19.841539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:13:20.042325 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:13:20.042852 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:13:20.043218 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:13:20.044031 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:13:20.044423 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:13:20.044763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:20.045897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:13:20.045897 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:20.045897 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:13:20.045897 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:13:20.045897 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:20.045897 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:13:20.045897 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:20.045897 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:13:20.046744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:20.046744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:13:20.046744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:20.046744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:13:20.046744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:13:20.046744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:13:20.046744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:13:20.046744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:20.046744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:20.046744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:20.046744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:13:20.046744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:20.047428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:20.047428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:20.047428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:13:20.047428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:20.047428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:13:20.047428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:13:20.047428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:20.047428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:13:20.047428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:20.047428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:13:20.047428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:13:20.047428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:20.047428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:13:20.048173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:20.048173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:13:20.048173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:20.048173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:13:20.048173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:13:20.048173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:13:20.048173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:13:20.048173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:13:20.048173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:20.048173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:13:20.048173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:13:20.048173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:13:20.048173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:13:20.048173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:13:20.048173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:20.048789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:20.048789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:13:20.048789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:13:20.048789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:13:20.048789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:13:20.048789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:20.048789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:20.048789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:20.048789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:13:20.048789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:20.048789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:20.048789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:20.048789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:13:20.048789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:20.048789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:13:20.048789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:13:20.048789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:20.048789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:13:20.048789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:20.049291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:13:20.049291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:13:20.049291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:20.049291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:13:20.049291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:20.049291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:13:20.051356 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:13:20.051509 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:13:20.051618 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:13:20.051748 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:13:20.052134 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:13:20.052347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:20.053713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:13:20.053713 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:20.053713 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:13:20.053713 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:13:20.053713 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:20.053713 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:13:20.053713 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:20.053713 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:13:20.054032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:20.054032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:13:20.054032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:13:20.054032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:13:20.054032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:20.054032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:13:20.054032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:20.054032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:13:20.054032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:20.054032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:13:20.054032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:20.054032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:13:20.054032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:20.054032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:13:20.054032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:13:20.054601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:20.054601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:13:20.054601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:20.054601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:13:20.054601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:13:20.054601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:20.054601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:13:20.054601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:20.054601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:13:20.054885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:20.054885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:13:20.054885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:20.054885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:13:20.054885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:13:20.054885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:13:20.054885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:13:20.054885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:13:20.054885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:13:20.054885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:13:20.054885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:20.054885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:20.055254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:20.055254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:13:20.055254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:20.055254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:20.055254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:20.055254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:13:20.055254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:20.055254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:13:20.055254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:13:20.055254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:20.055254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:13:20.055254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:20.055254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:13:20.055254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:13:20.055676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:20.055676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:13:20.055676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:20.055676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:13:20.056440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:20.056440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:13:20.056440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:13:20.056440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:13:20.056440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:13:20.056440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:13:20.056440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:20.056440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:13:20.056440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:13:20.056440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:13:20.056440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:13:20.056440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:13:20.056440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:20.056440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:20.056440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:13:20.056440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:13:20.056984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:13:20.056984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:13:20.056984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:13:20.056984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:13:20.056984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:13:20.056984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:20.056984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:20.056984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:20.056984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:13:20.056984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:20.056984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:20.056984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:20.056984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:13:20.056984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:20.056984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:13:20.056984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:13:20.056984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:20.056984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:13:20.057552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:20.057552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:13:20.057552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:13:20.057552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:20.057552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:13:20.057552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:20.057552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:13:20.057552 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:13:20.057806 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:13:20.059125 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:13:20.059125 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:13:20.059125 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:13:20.059125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:20.059125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:13:20.059125 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:20.059125 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:13:20.059125 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:13:20.059125 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:20.059125 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:13:20.059125 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:20.059125 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:13:20.059864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:20.059864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:13:20.059864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:13:20.059864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:13:20.059864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:20.059864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:13:20.059864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:20.059864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:13:20.059864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:20.059864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:13:20.059864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:20.059864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:13:20.059864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:20.059864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:13:20.059864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:13:20.060234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:20.060234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:13:20.060234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:20.060234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:13:20.060234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:13:20.060234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:20.060234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:13:20.060234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:20.060234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:13:20.060721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:20.060721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:13:20.060721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:20.060721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:13:20.060721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:13:20.060721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:13:20.060721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:13:20.060721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:13:20.060721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:13:20.060721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:20.060721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:20.061093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:20.061093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:13:20.061093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:20.061093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:20.061093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:20.061093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:13:20.061093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:20.061093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:13:20.061093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:13:20.061093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:20.061093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:13:20.061093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:20.061093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:13:20.061093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:13:20.061550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:20.061550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:13:20.061550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:20.061550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:13:20.061761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:20.061761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:13:20.061761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:13:20.061761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:13:20.061761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:13:20.061761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:13:20.061761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:20.061761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:13:20.061761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:13:20.061761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:13:20.061761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:13:20.061761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:13:20.061761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:20.061761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:20.061761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:13:20.061761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:13:20.061761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:13:20.062117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:13:20.062117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:13:20.062117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:13:20.062117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:20.062117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:20.062117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:20.062117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:13:20.062117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:20.062117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:20.062117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:20.062117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:13:20.062117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:20.062117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:13:20.062117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:13:20.062117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:20.062117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:13:20.062117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:20.062117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:13:20.062117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:13:20.062599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:20.062599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:13:20.062599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:20.062599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:13:20.062688 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:13:20.062938 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:13:20.063181 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:13:20.063334 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:13:20.063652 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:13:20.063873 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:13:20.064017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:20.064621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:13:20.064621 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:20.064621 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:13:20.064621 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:13:20.064621 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:20.064621 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:13:20.064621 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:20.064621 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:13:20.065455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:20.065455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:13:20.065455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:20.065455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:13:20.065455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:13:20.065455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:13:20.065455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:13:20.065455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:20.065455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:20.065455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:20.065455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:13:20.065455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:20.065888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:20.065888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:20.065888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:13:20.065888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:20.065888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:13:20.065888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:13:20.065888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:20.065888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:13:20.065888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:20.065888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:13:20.065888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:13:20.065888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:20.065888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:13:20.066331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:20.066331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:13:20.066627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:20.066627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:13:20.066627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:13:20.066627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:13:20.066627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:13:20.066627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:13:20.066627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:20.066627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:13:20.066627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:13:20.066627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:13:20.066627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:13:20.066627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:13:20.066627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:20.066627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:20.066627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:13:20.066627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:13:20.067139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:13:20.067139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:13:20.067139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:20.067139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:20.067139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:20.067139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:13:20.067139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:20.067139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:20.067139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:20.067139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:13:20.067139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:20.067139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:13:20.067139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:13:20.067139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:20.067139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:13:20.067139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:20.067139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:13:20.067139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:13:20.067139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:20.067740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:13:20.067740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:20.067740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:13:24.805485 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:13:24.806183 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:13:24.806183 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:13:24.806987 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:13:24.815003 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG dbcounter [-] [105581] Writing DB stats neutron:SELECT=241 {{(pid=105581) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 14:13:24.845175 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:13:24.845532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:24.846200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:13:24.846200 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:24.846200 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:13:24.846200 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:13:24.846200 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:24.846200 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:13:24.846200 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:24.846200 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:13:24.847198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:24.847198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:13:24.847198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:24.847198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:13:24.847198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:13:24.847198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:13:24.847198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:13:24.847198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:24.847198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:24.847198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:24.847198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:13:24.847198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:24.847886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:24.847886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:24.847886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:13:24.847886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:24.847886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:13:24.847886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:13:24.847886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:24.847886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:13:24.847886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:24.847886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:13:24.847886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:13:24.847886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:24.847886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:13:24.848528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:24.848528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:13:24.848528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:24.848528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:13:24.848528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:13:24.848528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:13:24.848528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:13:24.848528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:13:24.848528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:24.848528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:13:24.848528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:13:24.848528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:13:24.848528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:13:24.848528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:13:24.848528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:24.849268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:24.849268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:13:24.849268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:13:24.849268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:13:24.849268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:13:24.849268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:24.849268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:24.849268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:24.849268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:13:24.849268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:24.849268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:24.849268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:24.849268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:13:24.849268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:24.849268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:13:24.849268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:13:24.849268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:24.849268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:13:24.849268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:24.850148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:13:24.850148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:13:24.850148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:24.850148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:13:24.850148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:24.850148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:13:25.042965 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:13:25.043578 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:13:25.043774 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:13:25.044835 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:13:25.045099 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:13:25.045292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:25.045929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:13:25.045929 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:25.045929 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:13:25.045929 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:13:25.045929 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:25.045929 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:13:25.045929 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:25.045929 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:13:25.047573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:25.047573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:13:25.047573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:25.047573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:13:25.047573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:13:25.047573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:13:25.047573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:13:25.047573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:25.047573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:25.047573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:25.047573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:13:25.047573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:25.047941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:25.047941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:25.047941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:13:25.047941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:25.047941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:13:25.047941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:13:25.047941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:25.047941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:13:25.047941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:25.047941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:13:25.047941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:13:25.047941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:25.047941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:13:25.048302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:25.048302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:13:25.049328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:25.049328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:13:25.049328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:13:25.049328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:13:25.049328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:13:25.049328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:13:25.049328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:25.049328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:13:25.049328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:13:25.049328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:13:25.049328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:13:25.049328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:13:25.049328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:25.049328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:25.049328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:13:25.049328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:13:25.049779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:13:25.049779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:13:25.049779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:25.049779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:25.049779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:25.049779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:13:25.049779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:25.049779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:25.049779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:25.049779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:13:25.049779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:25.049779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:13:25.049779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:13:25.049779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:25.049779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:13:25.049779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:25.049779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:13:25.049779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:13:25.049779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:25.050251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:13:25.050251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:25.050251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:13:25.051957 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:13:25.052934 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:13:25.053247 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:13:25.053465 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:13:25.053816 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:13:25.054010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:25.054638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:13:25.054638 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:25.054638 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:13:25.054638 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:13:25.054638 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:25.054638 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:13:25.054638 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:25.054638 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:13:25.055572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:25.055572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:13:25.055572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:13:25.055572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:13:25.055572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:25.055572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:13:25.055572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:25.055572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:13:25.055572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:25.055572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:13:25.055572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:25.055572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:13:25.055572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:25.055572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:13:25.055572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:13:25.056053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:25.056053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:13:25.056053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:25.056053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:13:25.056053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:13:25.056053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:25.056053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:13:25.056053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:25.056053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:13:25.056589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:25.056589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:13:25.056589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:25.056589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:13:25.056589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:13:25.056589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:13:25.056589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:13:25.056589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:13:25.056589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:13:25.056589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:13:25.056589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:25.056589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:25.056918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:25.056918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:13:25.056918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:25.056918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:25.056918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:25.056918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:13:25.056918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:25.056918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:13:25.056918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:13:25.056918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:25.056918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:13:25.056918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:25.056918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:13:25.056918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:13:25.057657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:25.057657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:13:25.057657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:25.057657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:13:25.057657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:25.057657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:13:25.057657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:13:25.057657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:13:25.057657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:13:25.057657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:13:25.057657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:25.057657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:13:25.057657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:13:25.057657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:13:25.058011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:13:25.058011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:13:25.058011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:25.058011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:25.058011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:13:25.058011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:13:25.058011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:13:25.058011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:13:25.058011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:13:25.058011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:13:25.058011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:13:25.058011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:25.058011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:25.058011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:25.058011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:13:25.058011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:25.058011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:25.058011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:25.058011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:13:25.058485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:25.058485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:13:25.058485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:13:25.058485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:25.058485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:13:25.058485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:25.058485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:13:25.058485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:13:25.058485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:25.058485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:13:25.058485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:25.058485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:13:25.058485 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:13:25.058485 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:13:25.058825 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:13:25.058825 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:13:25.058825 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:13:25.059064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:25.059479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:13:25.059479 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:25.059479 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:13:25.059479 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:13:25.059479 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:25.059479 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:13:25.059479 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:25.059479 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:13:25.060176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:25.060176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:13:25.060176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:13:25.060176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:13:25.060176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:25.060176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:13:25.060176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:25.060176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:13:25.060176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:25.060176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:13:25.060176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:25.060176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:13:25.060176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:25.060176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:13:25.060176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:13:25.060766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:25.060766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:13:25.060766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:25.060766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:13:25.060766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:13:25.060766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:25.060766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:13:25.060766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:25.060766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:13:25.061090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:25.061090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:13:25.061090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:25.061090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:13:25.061090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:13:25.061090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:13:25.061090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:13:25.061090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:13:25.061090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:13:25.061090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:25.061090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:25.061462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:25.061462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:13:25.061462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:25.061462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:25.061462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:25.061462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:13:25.061462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:25.061462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:13:25.061462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:13:25.061462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:25.061462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:13:25.061462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:25.061462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:13:25.061462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:13:25.061826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:25.061826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:13:25.061826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:25.061826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:13:25.062247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:25.062247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:13:25.062247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:13:25.062247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:13:25.062247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:13:25.062247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:13:25.062247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:25.062247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:13:25.062247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:13:25.062247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:13:25.062247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:13:25.062247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:13:25.062247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:25.062247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:25.062247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:13:25.062247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:13:25.062247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:13:25.062726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:13:25.062726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:13:25.062726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:13:25.062726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:25.062726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:25.062726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:25.062726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:13:25.062726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:25.062726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:25.062726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:25.062726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:13:25.062726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:25.062726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:13:25.062726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:13:25.062726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:25.062726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:13:25.062726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:25.062726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:13:25.062726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:13:25.063251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:25.063251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:13:25.063251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:25.063251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:13:25.063251 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:13:25.063399 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:13:25.063399 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:13:25.063570 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:13:25.063853 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:13:25.064064 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:13:25.064205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:25.064682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:13:25.064682 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:25.064682 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:13:25.064682 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:13:25.064682 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:25.064682 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:13:25.064682 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:25.064682 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:13:25.065954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:25.065954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:13:25.065954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:25.065954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:13:25.065954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:13:25.065954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:13:25.065954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:13:25.065954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:25.065954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:25.065954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:25.065954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:13:25.065954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:25.066293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:25.066293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:25.066293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:13:25.066293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:25.066293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:13:25.066293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:13:25.066293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:25.066293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:13:25.066293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:25.066293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:13:25.066293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:13:25.066293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:25.066293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:13:25.066655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:25.066655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:13:25.066655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:25.066655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:13:25.066655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:13:25.066655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:13:25.066655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:13:25.066655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:13:25.066655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:25.066655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:13:25.066655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:13:25.066655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:13:25.066655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:13:25.066655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:13:25.066655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:25.067064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:25.067064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:13:25.067064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:13:25.067064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:13:25.067064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:13:25.067064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:25.067064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:25.067064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:25.067064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:13:25.067064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:25.067064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:25.067064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:25.067064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:13:25.067064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:25.067064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:13:25.067064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:13:25.067064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:25.067064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:13:25.067064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:25.067587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:13:25.067587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:13:25.067587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:25.067587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:13:25.067587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:25.067587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:13:29.806109 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:13:29.807042 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:13:29.807042 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:13:29.808229 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:13:29.841686 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:13:29.841868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:29.842673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:13:29.842673 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:29.842673 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:13:29.842673 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:13:29.842673 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:29.842673 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:13:29.842673 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:29.842673 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:13:29.843673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:29.843673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:13:29.843673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:29.843673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:13:29.843673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:13:29.843673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:13:29.843673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:13:29.843673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:29.843673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:29.843673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:29.843673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:13:29.843673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:29.844130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:29.844130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:29.844130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:13:29.844130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:29.844130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:13:29.844130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:13:29.844130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:29.844130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:13:29.844130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:29.844130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:13:29.844130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:13:29.844130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:29.844130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:13:29.844589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:29.844589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:13:29.844784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:29.844784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:13:29.844784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:13:29.844784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:13:29.844784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:13:29.844784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:13:29.844784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:29.844784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:13:29.844784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:13:29.844784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:13:29.844784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:13:29.844784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:13:29.844784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:29.844784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:29.844784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:13:29.844784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:13:29.845284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:13:29.845284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:13:29.845284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:29.845284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:29.845284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:29.845284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:13:29.845284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:29.845284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:29.845284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:29.845284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:13:29.845284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:29.845284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:13:29.845284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:13:29.845284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:29.845284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:13:29.845284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:29.845284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:13:29.845284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:13:29.845284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:29.845817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:13:29.845817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:29.845817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:13:30.044117 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:13:30.044513 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:13:30.044629 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:13:30.045390 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:13:30.045598 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:13:30.045750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:30.046307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:13:30.046307 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:30.046307 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:13:30.046307 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:13:30.046307 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:30.046307 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:13:30.046307 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:30.046307 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:13:30.047246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:30.047246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:13:30.047246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:30.047246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:13:30.047246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:13:30.047246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:13:30.047246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:13:30.047246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:30.047246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:30.047246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:30.047246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:13:30.047246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:30.047697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:30.047697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:30.047697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:13:30.047697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:30.047697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:13:30.047697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:13:30.047697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:30.047697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:13:30.047697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:30.047697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:13:30.047697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:13:30.047697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:30.047697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:13:30.048141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:30.048141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:13:30.048358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:30.048358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:13:30.048358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:13:30.048358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:13:30.048358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:13:30.048358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:13:30.048358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:30.048358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:13:30.048358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:13:30.048358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:13:30.048358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:13:30.048358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:13:30.048358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:30.048358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:30.048358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:13:30.048358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:13:30.048890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:13:30.048890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:13:30.048890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:30.048890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:30.048890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:30.048890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:13:30.048890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:30.048890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:30.048890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:30.048890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:13:30.048890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:30.048890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:13:30.048890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:13:30.048890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:30.048890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:13:30.048890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:30.048890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:13:30.048890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:13:30.048890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:30.049447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:13:30.049447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:30.049447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:13:30.054087 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:13:30.054495 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:13:30.054611 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:13:30.054731 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:13:30.055209 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:13:30.055529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:30.056353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:13:30.056353 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:30.056353 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:13:30.056353 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:13:30.056353 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:30.056353 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:13:30.056353 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:30.056353 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:13:30.057149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:30.057149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:13:30.057149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:13:30.057149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:13:30.057149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:30.057149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:13:30.057149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:30.057149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:13:30.057149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:30.057149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:13:30.057149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:30.057149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:13:30.057149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:30.057149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:13:30.057149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:13:30.057969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:30.057969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:13:30.057969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:30.057969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:13:30.057969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:13:30.057969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:30.057969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:13:30.057969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:30.057969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:13:30.057969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:30.057969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:13:30.057969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:30.057969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:13:30.057969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:13:30.058380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:13:30.058380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:13:30.058380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:13:30.058380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:13:30.058380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:13:30.058380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:30.058380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:30.058380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:30.058380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:13:30.058380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:30.058380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:30.058380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:30.058380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:13:30.058862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:30.058862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:13:30.058862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:13:30.058862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:30.058862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:13:30.058862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:30.058862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:13:30.058862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:13:30.058862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:30.058862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:13:30.058862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:30.058862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:13:30.059989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:30.059989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:13:30.059989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:13:30.059989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:13:30.059989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:13:30.059989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:13:30.059989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:30.059989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:13:30.059989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:13:30.059989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:13:30.059989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:13:30.059989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:13:30.059989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:30.059989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:30.059989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:13:30.059989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:13:30.060632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:13:30.060632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:13:30.060632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:13:30.060632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:13:30.060632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:13:30.060632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:30.060632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:30.060632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:30.060632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:13:30.060632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:30.060632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:30.060632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:30.060632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:13:30.060632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:30.060632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:13:30.060632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:13:30.060632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:30.060632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:13:30.061515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:30.061515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:13:30.061515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:13:30.061515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:30.061515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:13:30.061515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:30.061515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:13:30.061515 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:13:30.061515 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:13:30.061515 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:13:30.061808 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:13:30.061808 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:13:30.061808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:30.061808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:13:30.061808 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:30.061808 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:13:30.061808 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:13:30.061808 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:30.061808 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:13:30.061808 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:30.061808 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:13:30.061808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:30.062205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:13:30.062205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:13:30.062205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:13:30.062205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:30.062205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:13:30.062205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:30.062205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:13:30.062205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:30.062205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:13:30.062205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:30.062205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:13:30.062205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:30.062205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:13:30.062205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:13:30.062205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:30.062205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:13:30.062205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:30.062705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:13:30.062705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:13:30.062705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:30.062705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:13:30.062705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:30.062705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:13:30.062941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:30.062941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:13:30.062941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:30.062941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:13:30.062941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:13:30.062941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:13:30.062941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:13:30.062941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:13:30.062941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:13:30.062941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:30.062941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:30.063868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:30.063868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:13:30.063868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:30.063868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:30.063868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:30.063868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:13:30.063868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:30.063868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:13:30.063868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:13:30.063868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:30.063868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:13:30.063868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:30.063868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:13:30.063868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:13:30.064250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:30.064250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:13:30.064250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:30.064250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:13:30.064563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:30.064563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:13:30.064563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:13:30.064563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:13:30.064563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:13:30.064563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:13:30.064563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:30.064563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:13:30.064563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:13:30.064563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:13:30.064563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:13:30.064563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:13:30.064563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:30.064563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:30.064563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:13:30.064563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:13:30.064563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:13:30.064983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:13:30.064983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:13:30.064983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:13:30.064983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:30.064983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:30.064983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:30.064983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:13:30.064983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:30.064983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:30.064983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:30.064983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:13:30.064983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:30.064983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:13:30.064983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:13:30.064983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:30.064983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:13:30.064983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:30.064983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:13:30.064983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:13:30.065543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:30.065543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:13:30.065543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:30.065543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:13:30.065543 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:13:30.065543 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:13:30.065543 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:13:30.065543 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:13:30.066071 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:13:30.066071 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:13:30.066071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:30.066981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:13:30.066981 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:30.066981 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:13:30.066981 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:13:30.066981 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:30.066981 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:13:30.066981 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:30.066981 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:13:30.067835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:30.067835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:13:30.067835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:30.067835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:13:30.067835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:13:30.067835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:13:30.067835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:13:30.067835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:30.067835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:30.067835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:30.067835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:13:30.067835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:30.068115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:30.068115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:30.068115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:13:30.068115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:30.068115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:13:30.068115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:13:30.068115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:30.068115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:13:30.068115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:30.068115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:13:30.068115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:13:30.068115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:30.068115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:13:30.068360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:30.068360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:13:30.068823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:30.068823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:13:30.068823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:13:30.068823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:13:30.068823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:13:30.068823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:13:30.068823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:30.068823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:13:30.068823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:13:30.068823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:13:30.068823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:13:30.068823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:13:30.068823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:30.068823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:30.068823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:13:30.068823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:13:30.069147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:13:30.069147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:13:30.069147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:30.069147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:30.069147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:30.069147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:13:30.069147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:30.069147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:30.069147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:30.069147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:13:30.069147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:30.069147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:13:30.069147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:13:30.069147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:30.069147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:13:30.069147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:30.069147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:13:30.069147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:13:30.069147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:30.069575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:13:30.069575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:30.069575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:13:34.806779 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:13:34.807709 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:13:34.807709 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:13:34.808221 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:13:34.849028 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:13:34.849279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:34.851509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:13:34.851509 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:34.851509 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:13:34.851509 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:13:34.851509 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:34.851509 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:13:34.851509 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:34.851509 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:13:34.851509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:34.851509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:13:34.851509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:34.851509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:13:34.851509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:13:34.851509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:13:34.851509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:13:34.852123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:34.852123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:34.852123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:34.852123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:13:34.852123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:34.852123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:34.852123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:34.852123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:13:34.852123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:34.852123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:13:34.852123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:13:34.852123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:34.852123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:13:34.852123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:34.852567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:13:34.852567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:13:34.852567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:34.852567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:13:34.852567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:34.852567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:13:34.852567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:34.852567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:13:34.852567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:13:34.852567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:13:34.852567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:13:34.852567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:13:34.852567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:34.852567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:13:34.852997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:13:34.852997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:13:34.852997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:13:34.852997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:13:34.852997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:34.852997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:34.852997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:13:34.852997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:13:34.852997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:13:34.852997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:13:34.852997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:34.852997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:34.852997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:34.852997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:13:34.852997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:34.852997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:34.852997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:34.852997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:13:34.852997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:34.852997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:13:34.852997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:13:34.853634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:34.853634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:13:34.853634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:34.853634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:13:34.853634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:13:34.853634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:34.853634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:13:34.853634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:34.853634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:13:35.044952 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:13:35.045677 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:13:35.045943 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:13:35.048909 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:13:35.048909 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:13:35.048909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:35.049231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:13:35.049231 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:35.049231 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:13:35.049231 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:13:35.049231 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:35.049231 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:13:35.049231 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:35.049231 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:13:35.050446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:35.050446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:13:35.050446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:35.050446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:13:35.050446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:13:35.050446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:13:35.050446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:13:35.050446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:35.050446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:35.050446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:35.050446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:13:35.050446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:35.051268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:35.051268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:35.051268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:13:35.051268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:35.051268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:13:35.051268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:13:35.051268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:35.051268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:13:35.051268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:35.051268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:13:35.051268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:13:35.051268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:35.051268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:13:35.052068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:35.052068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:13:35.052068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:35.052068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:13:35.052068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:13:35.052068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:13:35.052068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:13:35.052068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:13:35.052068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:35.052068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:13:35.052068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:13:35.052068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:13:35.052068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:13:35.052068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:13:35.052068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:35.053085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:35.053085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:13:35.053085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:13:35.053085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:13:35.053085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:13:35.053085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:35.053085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:35.053085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:35.053085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:13:35.053085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:35.053085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:35.053085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:35.053085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:13:35.053085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:35.053085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:13:35.053085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:13:35.053085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:35.053085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:13:35.053085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:35.054243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:13:35.054243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:13:35.054243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:35.054243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:13:35.054243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:35.054243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:13:35.060262 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:13:35.060262 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:13:35.060262 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:13:35.060262 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:13:35.060262 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:13:35.060262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:35.060522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:13:35.060522 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:35.060522 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:13:35.060522 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:13:35.060522 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:35.060522 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:13:35.060522 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:35.060522 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:13:35.060522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:35.060522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:13:35.060522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:13:35.060522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:13:35.060522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:35.060522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:13:35.060522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:35.060522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:13:35.060522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:35.060522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:13:35.061015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:35.061015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:13:35.061015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:35.061015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:13:35.061015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:13:35.061015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:35.061015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:13:35.061015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:35.061015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:13:35.061015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:13:35.061015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:35.061015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:13:35.061015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:35.061015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:13:35.061015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:35.061515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:13:35.061515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:35.061515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:13:35.061515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:13:35.061515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:13:35.061515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:13:35.061515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:13:35.061515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:13:35.061515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:13:35.061515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:35.061515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:35.061515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:35.061515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:13:35.061941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:35.061941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:35.061941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:35.061941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:13:35.061941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:35.061941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:13:35.061941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:13:35.061941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:35.061941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:13:35.061941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:35.061941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:13:35.061941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:13:35.061941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:35.061941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:13:35.062284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:35.062284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:13:35.062284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:35.062284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:13:35.062284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:13:35.062284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:13:35.062284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:13:35.062284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:13:35.062284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:35.062284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:13:35.062284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:13:35.062284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:13:35.062284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:13:35.062284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:13:35.062284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:35.062774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:35.062774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:13:35.062774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:13:35.062774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:13:35.062774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:13:35.062774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:13:35.062774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:13:35.062774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:13:35.062774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:35.062774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:35.062774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:35.062774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:13:35.062774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:35.062774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:35.062774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:35.062774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:13:35.062774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:35.062774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:13:35.062774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:13:35.063481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:35.063481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:13:35.063481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:35.063481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:13:35.063481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:13:35.063481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:35.063481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:13:35.063481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:35.063481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:13:35.063481 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:13:35.063481 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:13:35.063481 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:13:35.063937 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:13:35.063937 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:13:35.063937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:35.063937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:13:35.063937 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:35.063937 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:13:35.063937 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:13:35.063937 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:35.063937 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:13:35.063937 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:35.063937 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:13:35.063937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:35.064360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:13:35.064360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:13:35.064360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:13:35.064360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:35.064360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:13:35.064360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:35.064360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:13:35.064360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:35.064360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:13:35.064360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:35.064360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:13:35.064360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:35.064360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:13:35.064360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:13:35.064360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:35.064360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:13:35.064360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:35.064959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:13:35.064959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:13:35.064959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:35.064959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:13:35.064959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:35.064959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:13:35.064959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:35.064959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:13:35.064959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:35.064959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:13:35.064959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:13:35.064959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:13:35.064959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:13:35.065508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:13:35.065508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:13:35.065508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:35.065508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:35.065508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:35.065508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:13:35.065508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:35.065508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:35.065508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:35.065508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:13:35.065508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:35.065508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:13:35.065508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:13:35.066000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:35.066000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:13:35.066000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:35.066000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:13:35.066000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:13:35.066000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:35.066000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:13:35.066000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:35.066000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:13:35.066000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:35.066000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:13:35.066000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:13:35.066493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:13:35.066493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:13:35.066493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:13:35.066493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:35.066493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:13:35.066493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:13:35.066493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:13:35.066493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:13:35.066493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:13:35.066493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:35.066493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:35.066493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:13:35.066493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:13:35.066493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:13:35.066493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:13:35.066493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:13:35.066493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:13:35.066493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:35.066493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:35.067101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:35.067101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:13:35.067101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:35.067101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:35.067101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:35.067101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:13:35.067101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:35.067101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:13:35.067101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:13:35.067101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:35.067101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:13:35.067101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:35.067101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:13:35.067101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:13:35.067101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:35.067101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:13:35.067101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:35.067101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:13:35.067707 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:13:35.067707 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:13:35.067707 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:13:35.067707 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:13:35.067707 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:13:35.067707 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:13:35.067974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:35.068069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:13:35.068069 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:35.068069 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:13:35.068069 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:13:35.068069 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:35.068069 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:13:35.068069 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:35.068069 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:13:35.069126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:35.069126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:13:35.069126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:35.069126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:13:35.069126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:13:35.069126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:13:35.069126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:13:35.069126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:35.069126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:35.069126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:35.069126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:13:35.069126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:35.069658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:35.069658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:35.069658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:13:35.069658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:35.069658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:13:35.069658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:13:35.069658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:35.069658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:13:35.069658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:35.069658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:13:35.069658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:13:35.069658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:35.069658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:13:35.070306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:35.070306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:13:35.070306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:35.070306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:13:35.070306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:13:35.070306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:13:35.070306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:13:35.070306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:13:35.070306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:35.070306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:13:35.070306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:13:35.070306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:13:35.070306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:13:35.070306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:13:35.070306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:35.070846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:35.070846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:13:35.070846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:13:35.070846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:13:35.070846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:13:35.070846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:35.070846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:35.070846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:35.070846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:13:35.070846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:35.070846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:35.070846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:35.070846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:13:35.070846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:35.070846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:13:35.070846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:13:35.070846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:35.070846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:13:35.070846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:35.071303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:13:35.071303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:13:35.071303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:35.071303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:13:35.071303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:35.071303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:13:39.807442 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:13:39.808211 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:13:39.808211 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:13:39.808801 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:13:39.839078 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:13:39.839330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:39.839868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:13:39.839868 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:39.839868 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:13:39.839868 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:13:39.839868 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:39.839868 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:13:39.839868 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:39.839868 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:13:39.840521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:39.840521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:13:39.840521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:39.840521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:13:39.840521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:13:39.840521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:13:39.840521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:13:39.840521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:39.840521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:39.840521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:39.840521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:13:39.840521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:39.840872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:39.840872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:39.840872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:13:39.840872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:39.840872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:13:39.840872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:13:39.840872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:39.840872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:13:39.840872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:39.840872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:13:39.840872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:13:39.840872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:39.840872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:13:39.841217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:39.841217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:13:39.841290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:39.841290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:13:39.841290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:13:39.841290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:13:39.841290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:13:39.841290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:13:39.841290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:39.841290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:13:39.841290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:13:39.841290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:13:39.841290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:13:39.841290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:13:39.841290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:39.841290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:39.841290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:13:39.841290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:13:39.841748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:13:39.841748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:13:39.841748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:39.841748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:39.841748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:39.841748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:13:39.841748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:39.841748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:39.841748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:39.841748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:13:39.841748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:39.841748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:13:39.841748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:13:39.841748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:39.841748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:13:39.841748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:39.841748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:13:39.841748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:13:39.841748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:39.842218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:13:39.842218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:39.842218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:13:40.045629 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:13:40.046327 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:13:40.046569 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:13:40.047679 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:13:40.048023 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:13:40.048359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:40.049061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:13:40.049061 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:40.049061 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:13:40.049061 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:13:40.049061 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:40.049061 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:13:40.049061 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:40.049061 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:13:40.050009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:40.050009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:13:40.050009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:40.050009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:13:40.050009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:13:40.050009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:13:40.050009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:13:40.050009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:40.050009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:40.050009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:40.050009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:13:40.050009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:40.050480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:40.050480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:40.050480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:13:40.050480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:40.050480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:13:40.050480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:13:40.050480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:40.050480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:13:40.050480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:40.050480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:13:40.050480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:13:40.050480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:40.050480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:13:40.050898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:40.050898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:13:40.051172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:40.051172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:13:40.051172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:13:40.051172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:13:40.051172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:13:40.051172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:13:40.051172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:40.051172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:13:40.051172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:13:40.051172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:13:40.051172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:13:40.051172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:13:40.051172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:40.051172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:40.051172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:13:40.051172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:13:40.051857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:13:40.051857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:13:40.051857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:40.051857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:40.051857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:40.051857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:13:40.051857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:40.051857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:40.051857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:40.051857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:13:40.051857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:40.051857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:13:40.051857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:13:40.051857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:40.051857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:13:40.051857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:40.051857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:13:40.051857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:13:40.051857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:40.052424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:13:40.052424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:40.052424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:13:40.055090 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:13:40.055614 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:13:40.055697 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:13:40.055785 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:13:40.056120 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:13:40.056296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:40.056851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:13:40.056851 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:40.056851 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:13:40.056851 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:13:40.056851 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:40.056851 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:13:40.056851 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:40.056851 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:13:40.057633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:40.057633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:13:40.057633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:13:40.057633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:13:40.057633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:40.057633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:13:40.057633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:40.057633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:13:40.057633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:40.057633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:13:40.057633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:40.057633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:13:40.057633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:40.057633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:13:40.057633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:13:40.058093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:40.058093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:13:40.058093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:40.058093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:13:40.058093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:13:40.058093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:40.058093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:13:40.058093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:40.058093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:13:40.058496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:40.058496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:13:40.058496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:40.058496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:13:40.058496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:13:40.058496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:13:40.058496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:13:40.058496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:13:40.058496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:13:40.058496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:13:40.058496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:40.058496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:40.058948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:40.058948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:13:40.058948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:40.058948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:40.058948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:40.058948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:13:40.058948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:40.058948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:13:40.058948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:13:40.058948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:40.058948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:13:40.058948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:40.058948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:13:40.058948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:13:40.059225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:40.059225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:13:40.059225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:40.059225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:13:40.065176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:40.065176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:13:40.065176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:13:40.065176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:13:40.065176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:13:40.065176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:13:40.065176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:40.065176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:13:40.065176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:13:40.065176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:13:40.065176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:13:40.065176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:13:40.065176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:40.065176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:40.065176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:13:40.065176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:13:40.065581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:13:40.065581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:13:40.065581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:13:40.065581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:13:40.065581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:13:40.065581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:40.065581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:40.065581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:40.065581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:13:40.065581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:40.065581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:40.065581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:40.065581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:13:40.065581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:40.065581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:13:40.065581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:13:40.065581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:40.065581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:13:40.065969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:40.065969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:13:40.065969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:13:40.065969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:40.065969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:13:40.065969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:40.065969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:13:40.065969 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:13:40.065969 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:13:40.065969 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:13:40.066223 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:13:40.066223 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:13:40.066223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:40.066589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:13:40.066589 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:40.066589 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:13:40.066589 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:13:40.066589 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:40.066589 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:13:40.066589 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:40.066589 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:13:40.067773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:40.067773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:13:40.067773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:13:40.067773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:13:40.067773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:40.067773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:13:40.067773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:40.067773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:13:40.067773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:40.067773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:13:40.067773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:40.067773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:13:40.067773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:40.067773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:13:40.067773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:13:40.068230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:40.068230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:13:40.068230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:40.068230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:13:40.068230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:13:40.068230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:40.068230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:13:40.068230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:40.068230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:13:40.068230 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:13:40.069501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:40.069501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:13:40.069501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:40.069501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:13:40.069501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:13:40.069501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:13:40.069501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:13:40.069501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:13:40.069501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:13:40.069501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:40.069501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:40.069968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:40.069968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:13:40.069968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:40.069968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:40.069968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:40.069968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:13:40.069968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:40.069968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:13:40.069968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:13:40.069968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:40.069968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:13:40.069968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:40.069968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:13:40.069968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:13:40.070392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:40.070392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:13:40.070392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:40.070392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:13:40.071363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:40.071363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:13:40.071363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:13:40.071363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:13:40.071363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:13:40.071363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:13:40.071363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:40.071363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:13:40.071363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:13:40.071363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:13:40.071363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:13:40.071363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:13:40.071363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:40.071363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:40.071363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:13:40.071363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:13:40.071363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:13:40.072063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:13:40.072063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:13:40.072063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:13:40.072063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:40.072063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:40.072063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:40.072063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:13:40.072063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:40.072063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:40.072063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:40.072063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:13:40.072063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:40.072063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:13:40.072063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:13:40.072063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:40.072063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:13:40.072063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:40.072063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:13:40.072063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:13:40.072794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:40.072794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:13:40.072794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:40.072794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:13:40.072794 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:13:40.072794 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:13:40.072794 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:13:40.072794 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:13:40.073123 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:13:40.073123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:40.073123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:13:40.073123 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:40.073123 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:13:40.073123 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:13:40.073123 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:40.073123 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:13:40.073123 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:40.073123 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:13:40.074139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:40.074139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:13:40.074139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:40.074139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:13:40.074139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:13:40.074139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:13:40.074139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:13:40.074139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:40.074139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:40.074139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:40.074139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:13:40.074139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:40.074476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:40.074476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:40.074476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:13:40.074476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:40.074476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:13:40.074476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:13:40.074476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:40.074476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:13:40.074476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:40.074476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:13:40.074476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:13:40.074476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:40.074476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:13:40.074738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:40.074738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:13:40.075126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:40.075126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:13:40.075126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:13:40.075126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:13:40.075126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:13:40.075126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:13:40.075126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:40.075126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:13:40.075126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:13:40.075126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:13:40.075126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:13:40.075126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:13:40.075126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:40.075126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:40.075126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:13:40.075126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:13:40.075515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:13:40.075515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:13:40.075515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:40.075515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:40.075515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:40.075515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:13:40.075515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:40.075515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:40.075515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:40.075515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:13:40.075515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:40.075515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:13:40.075515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:13:40.075515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:40.075515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:13:40.075515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:40.075515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:13:40.075515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:13:40.075515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:40.076287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:13:40.076287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:40.076287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:13:44.808817 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:13:44.808817 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:13:44.808817 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:13:44.809914 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:13:44.851450 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:13:44.851450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:44.851450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:13:44.851450 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:44.851450 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:13:44.851450 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:13:44.851450 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:44.851450 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:13:44.851450 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:44.851450 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:13:44.851450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:44.851450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:13:44.851450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:44.852072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:13:44.852072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:13:44.852072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:13:44.852072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:13:44.852072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:44.852072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:44.852072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:44.852072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:13:44.852072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:44.852072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:44.852072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:44.852072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:13:44.852072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:44.852072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:13:44.852370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:13:44.852370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:44.852370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:13:44.852370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:44.852370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:13:44.852370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:13:44.852370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:44.852370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:13:44.852370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:44.852370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:13:44.852370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:44.852370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:13:44.852676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:13:44.852676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:13:44.852676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:13:44.852676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:13:44.852676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:44.852676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:13:44.852676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:13:44.852676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:13:44.852676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:13:44.852676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:13:44.852676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:44.852676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:44.852676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:13:44.852676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:13:44.852676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:13:44.852676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:13:44.852676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:44.852676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:44.852676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:44.853096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:13:44.853096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:44.853096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:44.853096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:44.853096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:13:44.853096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:44.853096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:13:44.853096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:13:44.853096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:44.853096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:13:44.853096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:44.853096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:13:44.853096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:13:44.853096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:44.853096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:13:44.853096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:44.853096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:13:45.046207 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:13:45.046734 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:13:45.046839 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:13:45.047834 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:13:45.048028 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:13:45.048140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:45.048687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:13:45.048687 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:45.048687 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:13:45.048687 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:13:45.048687 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:45.048687 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:13:45.048687 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:45.048687 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:13:45.049602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:45.049602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:13:45.049602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:45.049602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:13:45.049602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:13:45.049602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:13:45.049602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:13:45.049602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:45.049602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:45.049602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:45.049602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:13:45.049602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:45.050009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:45.050009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:45.050009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:13:45.050009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:45.050009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:13:45.050009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:13:45.050009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:45.050009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:13:45.050009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:45.050009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:13:45.050009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:13:45.050009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:45.050009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:13:45.050387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:45.050387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:13:45.050777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:45.050777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:13:45.050777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:13:45.050777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:13:45.050777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:13:45.050777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:13:45.050777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:45.050777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:13:45.050777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:13:45.050777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:13:45.050777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:13:45.050777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:13:45.050777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:45.050777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:45.050777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:13:45.050777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:13:45.051288 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:13:45.051288 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:13:45.051288 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:45.051288 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:45.051288 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:45.051288 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:13:45.051288 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:45.051288 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:45.051288 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:45.051288 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:13:45.051288 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:45.051288 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:13:45.051288 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:13:45.051288 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:45.051288 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:13:45.051288 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:45.051288 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:13:45.051288 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:13:45.051288 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:45.051934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:13:45.051934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:45.051934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:13:45.055793 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:13:45.056137 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:13:45.056309 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:13:45.056516 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:13:45.059505 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:13:45.059505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:45.059505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:13:45.059505 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:45.059505 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:13:45.059505 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:13:45.059505 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:45.059505 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:13:45.059505 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:45.059505 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:13:45.059505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:45.059505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:13:45.059505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:13:45.059505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:13:45.059865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:45.059865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:13:45.059865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:45.059865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:13:45.059865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:45.059865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:13:45.059865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:45.059865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:13:45.059865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:45.059865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:13:45.059865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:13:45.059865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:45.059865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:13:45.059865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:45.059865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:13:45.059865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:13:45.059865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:45.059865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:13:45.060267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:45.060267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:13:45.060267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:45.060267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:13:45.060267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:45.060267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:13:45.060267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:13:45.060267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:13:45.060267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:13:45.060267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:13:45.060267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:13:45.060267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:13:45.060590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:45.060590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:45.060590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:45.060590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:13:45.060590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:45.060590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:45.060590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:45.060590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:13:45.060590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:45.060590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:13:45.060590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:13:45.060590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:45.060590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:13:45.060590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:45.060858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:13:45.060858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:13:45.060858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:45.060858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:13:45.060858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:45.060858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:13:45.060858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:45.060858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:13:45.060858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:13:45.060858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:13:45.060858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:13:45.060858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:13:45.060858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:45.060858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:13:45.061144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:13:45.061144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:13:45.061144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:13:45.061144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:13:45.061144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:45.061144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:45.061144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:13:45.061144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:13:45.061144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:13:45.061144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:13:45.061144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:13:45.061144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:13:45.061144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:13:45.061144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:45.061144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:45.061144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:45.061144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:13:45.061144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:45.061144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:45.061566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:45.061566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:13:45.061566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:45.061566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:13:45.061566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:13:45.061566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:45.061566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:13:45.061566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:45.061566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:13:45.061566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:13:45.061566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:45.061566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:13:45.061566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:45.061566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:13:45.064769 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:13:45.065059 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:13:45.065178 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:13:45.065316 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:13:45.065769 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:13:45.066019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:45.066514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:13:45.066514 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:45.066514 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:13:45.066514 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:13:45.066514 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:45.066514 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:13:45.066514 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:45.066514 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:13:45.067290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:45.067290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:13:45.067290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:13:45.067290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:13:45.067290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:45.067290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:13:45.067290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:45.067290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:13:45.067290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:45.067290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:13:45.067290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:45.067290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:13:45.067290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:45.067290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:13:45.067290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:13:45.067663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:45.067663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:13:45.067663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:45.067663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:13:45.067663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:13:45.067663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:45.067663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:13:45.067663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:45.067663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:13:45.068385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:45.068385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:13:45.068385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:45.068385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:13:45.068385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:13:45.068385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:13:45.068385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:13:45.068385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:13:45.068385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:13:45.068385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:45.068385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:45.068687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:45.068687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:13:45.068687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:45.068687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:45.068687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:45.068687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:13:45.068687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:45.068687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:13:45.068687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:13:45.068687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:45.068687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:13:45.068687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:45.068687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:13:45.068687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:13:45.068990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:45.068990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:13:45.068990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:45.068990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:13:45.069825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:45.069825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:13:45.069825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:13:45.069825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:13:45.069825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:13:45.069825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:13:45.069825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:45.069825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:13:45.069825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:13:45.069825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:13:45.069825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:13:45.069825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:13:45.069825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:45.069825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:45.069825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:13:45.069825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:13:45.069825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:13:45.070222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:13:45.070222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:13:45.070222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:13:45.070222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:45.070222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:45.070222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:45.070222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:13:45.070222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:45.070222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:45.070222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:45.070222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:13:45.070222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:45.070222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:13:45.070222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:13:45.070222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:45.070222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:13:45.070222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:45.070222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:13:45.070222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:13:45.070626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:45.070626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:13:45.070626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:45.070626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:13:45.071457 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:13:45.071812 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:13:45.071983 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:13:45.072220 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:13:45.072752 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:13:45.073580 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:13:45.073580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:45.073813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:13:45.073813 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:45.073813 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:13:45.073813 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:13:45.073813 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:45.073813 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:13:45.073813 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:45.073813 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:13:45.075527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:45.075527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:13:45.075527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:45.075527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:13:45.075527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:13:45.075527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:13:45.075527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:13:45.075527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:45.075527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:45.075527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:45.075527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:13:45.075527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:45.075826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:45.075826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:45.075826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:13:45.075826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:45.075826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:13:45.075826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:13:45.075826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:45.075826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:13:45.075826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:45.075826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:13:45.075826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:13:45.075826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:45.075826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:13:45.076057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:45.076057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:13:45.076337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:45.076337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:13:45.076337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:13:45.076337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:13:45.076337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:13:45.076337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:13:45.076337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:45.076337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:13:45.076337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:13:45.076337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:13:45.076337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:13:45.076337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:13:45.076337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:45.076337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:45.076337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:13:45.076337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:13:45.076652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:13:45.076652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:13:45.076652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:45.076652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:45.076652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:45.076652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:13:45.076652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:45.076652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:45.076652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:45.076652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:13:45.076652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:45.076652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:13:45.076652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:13:45.076652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:45.076652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:13:45.076652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:45.076652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:13:45.076652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:13:45.076652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:45.076991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:13:45.076991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:45.076991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:13:49.809828 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:13:49.809828 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:13:49.809828 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:13:49.810836 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:13:49.848908 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:13:49.849058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:49.849556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:13:49.849556 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:49.849556 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:13:49.849556 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:13:49.849556 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:49.849556 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:13:49.849556 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:49.849556 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:13:49.850236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:49.850236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:13:49.850236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:49.850236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:13:49.850236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:13:49.850236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:13:49.850236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:13:49.850236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:49.850236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:49.850236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:49.850236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:13:49.850236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:49.850530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:49.850530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:49.850530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:13:49.850530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:49.850530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:13:49.850530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:13:49.850530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:49.850530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:13:49.850530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:49.850530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:13:49.850530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:13:49.850530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:49.850530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:13:49.850784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:49.850784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:13:49.851077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:49.851077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:13:49.851077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:13:49.851077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:13:49.851077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:13:49.851077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:13:49.851077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:49.851077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:13:49.851077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:13:49.851077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:13:49.851077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:13:49.851077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:13:49.851077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:49.851077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:49.851077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:13:49.851077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:13:49.851421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:13:49.851421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:13:49.851421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:49.851421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:49.851421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:49.851421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:13:49.851421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:49.851421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:49.851421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:49.851421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:13:49.851421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:49.851421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:13:49.851421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:13:49.851421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:49.851421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:13:49.851421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:49.851421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:13:49.851421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:13:49.851421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:49.851780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:13:49.851780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:49.851780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:13:50.047541 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:13:50.047541 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:13:50.047541 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:13:50.048555 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:13:50.048749 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:13:50.048870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:50.049420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:13:50.049420 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:50.049420 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:13:50.049420 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:13:50.049420 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:50.049420 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:13:50.049420 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:50.049420 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:13:50.050463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:50.050463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:13:50.050463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:50.050463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:13:50.050463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:13:50.050463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:13:50.050463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:13:50.050463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:50.050463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:50.050463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:50.050463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:13:50.050463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:50.050803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:50.050803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:50.050803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:13:50.050803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:50.050803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:13:50.050803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:13:50.050803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:50.050803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:13:50.050803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:50.050803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:13:50.050803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:13:50.050803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:50.050803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:13:50.051166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:50.051166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:13:50.051499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:50.051499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:13:50.051499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:13:50.051499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:13:50.051499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:13:50.051499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:13:50.051499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:50.051499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:13:50.051499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:13:50.051499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:13:50.051499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:13:50.051499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:13:50.051499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:50.051499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:50.051499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:13:50.051499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:13:50.052017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:13:50.052017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:13:50.052017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:50.052017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:50.052017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:50.052017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:13:50.052017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:50.052017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:50.052017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:50.052017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:13:50.052017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:50.052017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:13:50.052017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:13:50.052017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:50.052017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:13:50.052017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:50.052017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:13:50.052017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:13:50.052017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:50.052556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:13:50.052556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:50.052556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:13:50.056108 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:13:50.056366 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:13:50.056499 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:13:50.056620 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:13:50.058594 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:13:50.058594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:50.058594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:13:50.058594 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:50.058594 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:13:50.058594 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:13:50.058594 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:50.058594 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:13:50.058594 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:50.058594 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:13:50.058594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:50.058594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:13:50.058594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:13:50.058594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:13:50.059027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:50.059027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:13:50.059027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:50.059027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:13:50.059027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:50.059027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:13:50.059027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:50.059027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:13:50.059027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:50.059027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:13:50.059027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:13:50.059027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:50.059027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:13:50.059027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:50.059027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:13:50.059027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:13:50.059027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:50.059027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:13:50.059503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:50.059503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:13:50.059503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:50.059503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:13:50.059503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:50.059503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:13:50.059503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:13:50.059503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:13:50.059503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:13:50.059503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:13:50.059503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:13:50.059503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:13:50.059926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:50.059926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:50.059926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:50.059926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:13:50.059926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:50.059926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:50.059926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:50.059926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:13:50.059926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:50.059926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:13:50.059926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:13:50.059926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:50.059926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:13:50.059926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:50.060297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:13:50.060297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:13:50.060297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:50.060297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:13:50.060297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:50.060297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:13:50.060297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:50.060297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:13:50.060297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:13:50.060297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:13:50.060297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:13:50.060297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:13:50.060297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:50.060297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:13:50.060717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:13:50.060717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:13:50.060717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:13:50.060717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:13:50.060717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:50.060717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:50.060717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:13:50.060717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:13:50.060717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:13:50.060717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:13:50.060717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:13:50.060717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:13:50.060717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:13:50.060717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:50.060717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:50.060717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:50.060717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:13:50.060717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:50.060717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:50.061658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:50.061658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:13:50.061658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:50.061658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:13:50.061658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:13:50.061658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:50.061658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:13:50.061658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:50.061658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:13:50.061658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:13:50.061658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:50.061658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:13:50.061658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:50.061658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:13:50.065112 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:13:50.065378 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:13:50.065509 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:13:50.065633 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:13:50.066083 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:13:50.066797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:50.067205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:13:50.067205 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:50.067205 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:13:50.067205 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:13:50.067205 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:50.067205 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:13:50.067205 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:50.067205 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:13:50.068644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:50.068644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:13:50.068644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:13:50.068644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:13:50.068644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:50.068644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:13:50.068644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:50.068644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:13:50.068644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:50.068644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:13:50.068644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:50.068644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:13:50.068644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:50.068644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:13:50.068644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:13:50.069064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:50.069064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:13:50.069064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:50.069064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:13:50.069064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:13:50.069064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:50.069064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:13:50.069064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:50.069064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:13:50.069064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:50.069064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:13:50.069064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:50.069064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:13:50.069064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:13:50.069473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:13:50.069473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:13:50.069473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:13:50.069473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:13:50.069473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:50.069473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:50.069473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:50.069473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:13:50.069473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:50.069473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:50.069473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:50.069473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:13:50.069473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:50.069796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:13:50.069796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:13:50.069796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:50.069796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:13:50.069796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:50.069796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:13:50.069796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:13:50.069796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:50.069796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:13:50.069796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:50.069796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:13:50.070115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:50.070115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:13:50.070115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:13:50.070115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:13:50.070115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:13:50.070115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:13:50.070115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:50.070115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:13:50.070115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:13:50.070115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:13:50.070115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:13:50.070115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:13:50.070115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:50.070115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:50.070115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:13:50.070115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:13:50.070115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:13:50.070640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:13:50.070640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:13:50.070640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:13:50.070640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:50.070640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:50.070640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:50.070640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:13:50.070640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:50.070640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:50.070640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:50.070640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:13:50.070640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:50.070640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:13:50.070640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:13:50.070640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:50.070640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:13:50.070640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:50.070640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:13:50.070640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:13:50.071215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:50.071215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:13:50.071215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:50.071215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:13:50.072100 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:13:50.072342 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:13:50.072473 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:13:50.072617 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:13:50.073112 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:13:50.073516 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:13:50.073764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:50.074560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:13:50.074560 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:50.074560 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:13:50.074560 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:13:50.074560 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:50.074560 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:13:50.074560 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:50.074560 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:13:50.075352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:50.075352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:13:50.075352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:50.075352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:13:50.075352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:13:50.075352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:13:50.075352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:13:50.075352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:50.075352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:50.075352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:50.075352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:13:50.075352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:50.075731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:50.075731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:50.075731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:13:50.075731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:50.075731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:13:50.075731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:13:50.075731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:50.075731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:13:50.075731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:50.075731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:13:50.075731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:13:50.075731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:50.075731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:13:50.076113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:50.076113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:13:50.076333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:50.076333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:13:50.076333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:13:50.076333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:13:50.076333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:13:50.076333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:13:50.076333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:50.076333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:13:50.076333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:13:50.076333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:13:50.076333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:13:50.076333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:13:50.076333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:50.076333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:50.076333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:13:50.076333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:13:50.076847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:13:50.076847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:13:50.076847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:50.076847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:50.076847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:50.076847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:13:50.076847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:50.076847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:50.076847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:50.076847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:13:50.076847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:50.076847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:13:50.076847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:13:50.076847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:50.076847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:13:50.076847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:50.076847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:13:50.076847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:13:50.076847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:50.077684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:13:50.077684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:50.077684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:13:54.809360 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:13:54.810556 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:13:54.810556 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:13:54.810791 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:13:54.840824 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:13:54.840984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:54.841460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:13:54.841460 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:54.841460 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:13:54.841460 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:13:54.841460 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:54.841460 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:13:54.841460 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:54.841460 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:13:54.842116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:54.842116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:13:54.842116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:54.842116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:13:54.842116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:13:54.842116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:13:54.842116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:13:54.842116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:54.842116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:54.842116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:54.842116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:13:54.842116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:54.842785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:54.842785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:54.842785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:13:54.842785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:54.842785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:13:54.842785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:13:54.842785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:54.842785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:13:54.842785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:54.842785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:13:54.842785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:13:54.842785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:54.842785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:13:54.843515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:54.843515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:13:54.843515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:54.843515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:13:54.843515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:13:54.843515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:13:54.843515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:13:54.843515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:13:54.843515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:54.843515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:13:54.843515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:13:54.843515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:13:54.843515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:13:54.843515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:13:54.843515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:54.844537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:54.844537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:13:54.844537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:13:54.844537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:13:54.844537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:13:54.844537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:54.844537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:54.844537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:54.844537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:13:54.844537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:54.844537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:54.844537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:54.844537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:13:54.844537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:54.844537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:13:54.844537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:13:54.844537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:54.844537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:13:54.844537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:54.845560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:13:54.845560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:13:54.845560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:54.845560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:13:54.845560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:54.845560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:13:55.047356 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:13:55.047898 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:13:55.048007 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:13:55.048681 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:13:55.048886 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:13:55.049038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:55.049667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:13:55.049667 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:55.049667 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:13:55.049667 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:13:55.049667 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:55.049667 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:13:55.049667 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:55.049667 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:13:55.052606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:55.052606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:13:55.052606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:55.052606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:13:55.052606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:13:55.052606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:13:55.052606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:13:55.052606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:55.052606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:55.052606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:55.052606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:13:55.052606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:55.053251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:55.053251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:55.053251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:13:55.053251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:55.053251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:13:55.053251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:13:55.053251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:55.053251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:13:55.053251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:55.053251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:13:55.053251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:13:55.053251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:55.053251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:13:55.053609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:55.053609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:13:55.053609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:55.053609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:13:55.053609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:13:55.053609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:13:55.053609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:13:55.053609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:13:55.053609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:55.053609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:13:55.053609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:13:55.053609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:13:55.053609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:13:55.053609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:13:55.053609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:55.053962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:55.053962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:13:55.053962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:13:55.053962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:13:55.053962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:13:55.053962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:55.053962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:55.053962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:55.053962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:13:55.053962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:55.053962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:55.053962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:55.053962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:13:55.053962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:55.053962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:13:55.053962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:13:55.053962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:55.053962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:13:55.053962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:55.054426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:13:55.054426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:13:55.054426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:55.054426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:13:55.054426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:55.054426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:13:55.056480 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:13:55.056962 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:13:55.057100 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:13:55.057231 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:13:55.057659 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:13:55.057888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:55.058361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:13:55.058361 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:55.058361 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:13:55.058361 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:13:55.058361 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:55.058361 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:13:55.058361 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:55.058361 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:13:55.059465 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:55.059465 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:13:55.059465 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:13:55.059465 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:13:55.059465 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:55.059465 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:13:55.059465 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:55.059465 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:13:55.059465 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:55.059465 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:13:55.059465 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:55.059465 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:13:55.059465 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:55.059465 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:13:55.059465 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:13:55.060004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:55.060004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:13:55.060004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:55.060004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:13:55.060004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:13:55.060004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:55.060004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:13:55.060004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:55.060004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:13:55.060319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:55.060319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:13:55.060319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:55.060319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:13:55.060319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:13:55.060319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:13:55.060319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:13:55.060319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:13:55.060319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:13:55.060319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:13:55.060319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:55.060319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:55.060999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:55.060999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:13:55.060999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:55.060999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:55.060999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:55.060999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:13:55.060999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:55.060999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:13:55.060999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:13:55.060999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:55.060999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:13:55.060999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:55.060999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:13:55.060999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:13:55.061436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:55.061436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:13:55.061436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:55.061436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:13:55.062318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:55.062318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:13:55.062318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:13:55.062318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:13:55.062318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:13:55.062318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:13:55.062318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:55.062318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:13:55.062318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:13:55.062318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:13:55.062318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:13:55.062318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:13:55.062318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:55.062318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:55.062318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:13:55.062318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:13:55.062886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:13:55.062886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:13:55.062886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:13:55.062886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:13:55.062886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:13:55.062886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:55.062886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:55.062886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:55.062886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:13:55.062886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:55.062886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:55.062886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:55.062886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:13:55.062886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:55.062886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:13:55.062886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:13:55.062886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:55.062886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:13:55.063440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:55.063440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:13:55.063440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:13:55.063440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:55.063440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:13:55.063440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:55.063440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:13:55.065768 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:13:55.066054 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:13:55.066193 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:13:55.066601 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:13:55.066826 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:13:55.067076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:55.067653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:13:55.067653 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:55.067653 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:13:55.067653 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:13:55.067653 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:55.067653 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:13:55.067653 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:55.067653 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:13:55.068735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:55.068735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:13:55.068735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:13:55.068735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:13:55.068735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:55.068735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:13:55.068735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:55.068735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:13:55.068735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:55.068735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:13:55.068735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:55.068735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:13:55.068735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:55.068735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:13:55.068735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:13:55.069082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:55.069082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:13:55.069082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:55.069082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:13:55.069082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:13:55.069082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:55.069082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:13:55.069082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:55.069082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:13:55.069856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:55.069856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:13:55.069856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:55.069856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:13:55.069856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:13:55.069856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:13:55.069856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:13:55.069856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:13:55.069856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:13:55.069856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:55.069856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:55.071359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:55.071359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:13:55.071359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:55.071359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:55.071359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:55.071359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:13:55.071359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:55.071359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:13:55.071359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:13:55.071359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:55.071359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:13:55.071359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:55.071359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:13:55.071359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:13:55.072053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:55.072053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:13:55.072053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:55.072053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:13:55.072053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:55.072053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:13:55.072053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:13:55.072053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:13:55.072053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:13:55.072053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:13:55.072053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:55.072053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:13:55.072053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:13:55.072053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:13:55.072753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:13:55.072753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:13:55.072753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:55.072753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:55.072753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:13:55.072753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:13:55.072753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:13:55.072753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:13:55.072753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:13:55.072753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:13:55.072753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:55.072753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:55.072753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:55.072753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:13:55.072753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:55.072753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:55.072753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:55.072753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:13:55.072753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:55.073512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:13:55.073512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:13:55.073512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:55.073512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:13:55.073512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:55.073512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:13:55.073512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:13:55.073512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:55.073512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:13:55.073512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:55.073512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:13:55.073512 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:13:55.073512 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:13:55.073899 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:13:55.073899 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:13:55.073899 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:13:55.073899 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:13:55.073899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:55.074371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:13:55.074371 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:55.074371 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:13:55.074371 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:13:55.074371 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:55.074371 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:13:55.074371 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:55.074371 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:13:55.075220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:55.075220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:13:55.075220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:55.075220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:13:55.075220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:13:55.075220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:13:55.075220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:13:55.075220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:55.075220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:55.075220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:55.075220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:13:55.075220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:55.075625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:55.075625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:55.075625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:13:55.075625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:55.075625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:13:55.075625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:13:55.075625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:55.075625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:13:55.075625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:55.075625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:13:55.075625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:13:55.075625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:55.075625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:13:55.075968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:55.075968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:13:55.076211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:55.076211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:13:55.076211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:13:55.076211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:13:55.076211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:13:55.076211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:13:55.076211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:55.076211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:13:55.076211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:13:55.076211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:13:55.076211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:13:55.076211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:13:55.076211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:55.076211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:55.076211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:13:55.076211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:13:55.077204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:13:55.077204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:13:55.077204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:55.077204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:55.077204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:55.077204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:13:55.077204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:55.077204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:55.077204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:55.077204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:13:55.077204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:55.077204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:13:55.077204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:13:55.077204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:55.077204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:13:55.077204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:55.077204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:13:55.077204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:13:55.077204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:55.077755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:13:55.077755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:55.077755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:13:59.811626 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:13:59.811626 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:13:59.811626 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:13:59.813399 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:13:59.849264 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:13:59.851432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:59.851432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:13:59.851432 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:59.851432 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:13:59.851432 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:13:59.851432 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:59.851432 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:13:59.851432 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:59.851432 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:13:59.851432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:59.851432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:13:59.851432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:59.851432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:13:59.851432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:13:59.851432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:13:59.851819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:13:59.851819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:59.851819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:59.851819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:59.851819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:13:59.851819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:59.851819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:13:59.851819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:59.851819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:13:59.851819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:59.851819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:13:59.851819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:13:59.851819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:59.851819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:13:59.852099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:59.852099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:13:59.852099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:13:59.852099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:59.852099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:13:59.852099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:59.852099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:13:59.852099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:59.852099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:13:59.852099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:13:59.852099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:13:59.852099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:13:59.852099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:13:59.852376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:59.852376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:13:59.852376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:13:59.852376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:13:59.852376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:13:59.852376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:13:59.852376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:13:59.852376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:13:59.852376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:13:59.852376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:13:59.852376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:13:59.852376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:13:59.852376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:59.852376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:59.852376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:13:59.852376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:13:59.852376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:13:59.852376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:13:59.852376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:13:59.852376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:13:59.852946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:13:59.852946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:13:59.852946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:13:59.852946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:13:59.852946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:13:59.852946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:13:59.852946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:13:59.852946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:13:59.852946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:13:59.852946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:13:59.852946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:13:59.852946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:14:00.050431 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:14:00.050431 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:14:00.050431 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:14:00.050431 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:00.050431 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:00.050431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:00.050964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:14:00.050964 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:00.050964 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:14:00.050964 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:14:00.050964 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:00.050964 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:14:00.050964 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:00.050964 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:14:00.052435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:00.052435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:14:00.052435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:00.052435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:14:00.052435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:14:00.052435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:14:00.052435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:14:00.052435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:00.052435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:00.052435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:00.052435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:14:00.052435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:00.052706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:00.052706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:00.052706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:14:00.052706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:00.052706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:14:00.052706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:14:00.052706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:00.052706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:14:00.052706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:00.052706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:14:00.052706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:14:00.052706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:00.052706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:14:00.052942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:00.052942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:14:00.054693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:00.054693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:14:00.054693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:14:00.054693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:14:00.054693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:14:00.054693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:14:00.054693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:00.054693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:14:00.054693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:14:00.054693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:14:00.054693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:14:00.054693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:14:00.054693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:00.054693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:00.054693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:14:00.054693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:14:00.055009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:14:00.055009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:14:00.055009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:00.055009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:00.055009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:00.055009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:14:00.055009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:00.055009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:00.055009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:00.055009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:14:00.055009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:00.055009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:14:00.055009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:14:00.055009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:00.055009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:14:00.055009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:00.055009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:14:00.055009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:14:00.055009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:00.055353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:14:00.055353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:00.055353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:14:00.056983 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:14:00.057291 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:14:00.057667 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:14:00.057931 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:14:00.058292 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:00.058466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:00.059023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:14:00.059023 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:00.059023 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:14:00.059023 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:14:00.059023 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:00.059023 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:14:00.059023 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:00.059023 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:14:00.060059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:00.060059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:14:00.060059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:14:00.060059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:14:00.060059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:00.060059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:14:00.060059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:00.060059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:14:00.060059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:00.060059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:14:00.060059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:00.060059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:14:00.060059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:00.060059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:14:00.060059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:14:00.060389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:00.060389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:14:00.060389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:00.060389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:14:00.060389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:14:00.060389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:00.060389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:14:00.060389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:00.060389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:14:00.062226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:00.062226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:14:00.062226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:00.062226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:14:00.062226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:14:00.062226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:14:00.062226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:14:00.062226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:14:00.062226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:14:00.062226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:14:00.062226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:00.062226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:00.062549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:00.062549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:14:00.062549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:00.062549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:00.062549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:00.062549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:14:00.062549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:00.062549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:14:00.062549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:14:00.062549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:00.062549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:14:00.062549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:00.062549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:14:00.062549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:14:00.062882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:00.062882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:14:00.062882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:00.062882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:14:00.064883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:00.064883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:14:00.064883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:14:00.064883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:14:00.064883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:14:00.064883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:14:00.064883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:00.064883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:14:00.064883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:14:00.064883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:14:00.064883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:14:00.064883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:14:00.064883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:00.064883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:00.064883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:14:00.064883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:14:00.065222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:14:00.065222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:14:00.065222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:14:00.065222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:14:00.065222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:14:00.065222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:00.065222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:00.065222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:00.065222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:14:00.065222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:00.065222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:00.065222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:00.065222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:14:00.065222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:00.065222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:14:00.065222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:14:00.065222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:00.065222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:14:00.065734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:00.065734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:14:00.065734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:14:00.065734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:00.065734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:14:00.065734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:00.065734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:14:00.068057 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:14:00.068519 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:14:00.068827 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:14:00.068994 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:14:00.070184 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:00.070556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:00.071268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:14:00.071268 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:00.071268 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:14:00.071268 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:14:00.071268 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:00.071268 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:14:00.071268 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:00.071268 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:14:00.072350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:00.072350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:14:00.072350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:14:00.072350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:14:00.072350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:00.072350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:14:00.072350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:00.072350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:14:00.072350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:00.072350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:14:00.072350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:00.072350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:14:00.072350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:00.072350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:14:00.072350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:14:00.072762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:00.072762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:14:00.072762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:00.072762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:14:00.072762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:14:00.072762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:00.072762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:14:00.072762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:00.072762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:14:00.074083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:00.074083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:14:00.074083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:00.074083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:14:00.074083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:14:00.074083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:14:00.074083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:14:00.074083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:14:00.074083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:14:00.074083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:00.074083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:00.074520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:00.074520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:14:00.074520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:00.074520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:00.074520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:00.074520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:14:00.074520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:00.074520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:14:00.074520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:14:00.074520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:00.074520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:14:00.074520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:00.074520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:14:00.074520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:14:00.074938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:00.074938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:14:00.074938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:00.074938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:14:00.077721 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:14:00.078716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:00.078716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:14:00.078716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:14:00.078716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:14:00.078716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:14:00.078716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:14:00.078716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:00.078716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:14:00.078716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:14:00.078716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:14:00.078716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:14:00.078716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:14:00.078716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:00.078716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:00.078716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:14:00.078716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:14:00.078716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:14:00.079182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:14:00.079182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:14:00.079182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:14:00.079182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:00.079182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:00.079182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:00.079182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:14:00.079182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:00.079182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:00.079182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:00.079182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:14:00.079182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:00.079182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:14:00.079182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:14:00.079182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:00.079182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:14:00.079182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:00.079182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:14:00.079182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:14:00.079699 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:00.079699 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:14:00.079699 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:00.079699 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:14:00.079699 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:14:00.079699 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:14:00.079699 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:14:00.079920 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:00.080111 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:00.080272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:00.080846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:14:00.080846 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:00.080846 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:14:00.080846 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:14:00.080846 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:00.080846 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:14:00.080846 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:00.080846 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:14:00.081738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:00.081738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:14:00.081738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:00.081738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:14:00.081738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:14:00.081738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:14:00.081738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:14:00.081738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:00.081738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:00.081738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:00.081738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:14:00.081738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:00.082147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:00.082147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:00.082147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:14:00.082147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:00.082147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:14:00.082147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:14:00.082147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:00.082147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:14:00.082147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:00.082147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:14:00.082147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:14:00.082147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:00.082147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:14:00.082538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:00.082538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:14:00.082987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:00.082987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:14:00.082987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:14:00.082987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:14:00.082987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:14:00.082987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:14:00.082987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:00.082987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:14:00.082987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:14:00.082987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:14:00.082987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:14:00.082987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:14:00.082987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:00.082987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:00.082987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:14:00.082987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:14:00.083662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:14:00.083662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:14:00.083662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:00.083662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:00.083662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:00.083662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:14:00.083662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:00.083662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:00.083662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:00.083662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:14:00.083662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:00.083662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:14:00.083662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:14:00.083662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:00.083662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:14:00.083662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:00.083662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:14:00.083662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:14:00.083662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:00.084245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:14:00.084245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:00.084245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:14:04.810835 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:14:04.811581 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:14:04.811874 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:14:04.812802 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:14:04.851946 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:04.852324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:04.853100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:14:04.853100 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:04.853100 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:14:04.853100 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:14:04.853100 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:04.853100 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:14:04.853100 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:04.853100 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:14:04.854505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:04.854505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:14:04.854505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:04.854505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:14:04.854505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:14:04.854505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:14:04.854505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:14:04.854505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:04.854505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:04.854505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:04.854505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:14:04.854505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:04.854938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:04.854938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:04.854938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:14:04.854938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:04.854938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:14:04.854938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:14:04.854938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:04.854938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:14:04.854938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:04.854938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:14:04.854938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:14:04.854938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:04.854938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:14:04.855207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:04.855207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:14:04.855207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:04.855207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:14:04.855207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:14:04.855207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:14:04.855207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:14:04.855207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:14:04.855207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:04.855207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:14:04.855207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:14:04.855207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:14:04.855207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:14:04.855207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:14:04.855207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:04.855533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:04.855533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:14:04.855533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:14:04.855533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:14:04.855533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:14:04.855533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:04.855533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:04.855533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:04.855533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:14:04.855533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:04.855533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:04.855533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:04.855533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:14:04.855533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:04.855533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:14:04.855533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:14:04.855533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:04.855533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:14:04.855533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:04.855903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:14:04.855903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:14:04.855903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:04.855903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:14:04.855903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:04.855903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:14:05.048711 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:14:05.049108 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:14:05.049231 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:14:05.050113 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:05.050282 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:05.050421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:05.050945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:14:05.050945 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:05.050945 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:14:05.050945 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:14:05.050945 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:05.050945 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:14:05.050945 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:05.050945 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:14:05.052035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:05.052035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:14:05.052035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:05.052035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:14:05.052035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:14:05.052035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:14:05.052035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:14:05.052035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:05.052035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:05.052035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:05.052035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:14:05.052035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:05.052395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:05.052395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:05.052395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:14:05.052395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:05.052395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:14:05.052395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:14:05.052395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:05.052395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:14:05.052395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:05.052395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:14:05.052395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:14:05.052395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:05.052395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:14:05.052774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:05.052774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:14:05.053108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:05.053108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:14:05.053108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:14:05.053108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:14:05.053108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:14:05.053108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:14:05.053108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:05.053108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:14:05.053108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:14:05.053108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:14:05.053108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:14:05.053108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:14:05.053108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:05.053108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:05.053108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:14:05.053108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:14:05.053580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:14:05.053580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:14:05.053580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:05.053580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:05.053580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:05.053580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:14:05.053580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:05.053580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:05.053580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:05.053580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:14:05.053580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:05.053580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:14:05.053580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:14:05.053580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:05.053580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:14:05.053580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:05.053580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:14:05.053580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:14:05.053580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:05.054079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:14:05.054079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:05.054079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:14:05.057546 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:14:05.057789 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:14:05.057909 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:14:05.058071 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:14:05.058353 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:05.058526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:05.058958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:14:05.058958 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:05.058958 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:14:05.058958 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:14:05.058958 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:05.058958 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:14:05.058958 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:05.058958 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:14:05.059778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:05.059778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:14:05.059778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:14:05.059778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:14:05.059778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:05.059778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:14:05.059778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:05.059778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:14:05.059778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:05.059778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:14:05.059778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:05.059778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:14:05.059778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:05.059778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:14:05.059778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:14:05.060199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:05.060199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:14:05.060199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:05.060199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:14:05.060199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:14:05.060199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:05.060199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:14:05.060199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:05.060199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:14:05.060688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:05.060688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:14:05.060688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:05.060688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:14:05.060688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:14:05.060688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:14:05.060688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:14:05.060688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:14:05.060688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:14:05.060688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:14:05.060688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:05.060688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:05.061047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:05.061047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:14:05.061047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:05.061047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:05.061047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:05.061047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:14:05.061047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:05.061047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:14:05.061047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:14:05.061047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:05.061047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:14:05.061047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:05.061047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:14:05.061047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:14:05.061393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:05.061393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:14:05.061393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:05.061393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:14:05.061790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:05.061790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:14:05.061790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:14:05.061790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:14:05.061790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:14:05.061790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:14:05.061790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:05.061790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:14:05.061790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:14:05.061790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:14:05.061790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:14:05.061790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:14:05.061790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:05.061790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:05.061790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:14:05.061790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:14:05.062212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:14:05.062212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:14:05.062212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:14:05.062212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:14:05.062212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:14:05.062212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:05.062212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:05.062212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:05.062212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:14:05.062212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:05.062212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:05.062212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:05.062212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:14:05.062212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:05.062212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:14:05.062212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:14:05.062212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:05.062212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:14:05.062664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:05.062664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:14:05.062664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:14:05.062664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:05.062664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:14:05.062664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:05.062664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:14:05.068597 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:14:05.068860 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:14:05.068985 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:14:05.069121 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:14:05.069421 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:05.069574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:05.070040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:14:05.070040 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:05.070040 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:14:05.070040 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:14:05.070040 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:05.070040 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:14:05.070040 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:05.070040 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:14:05.070807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:05.070807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:14:05.070807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:14:05.070807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:14:05.070807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:05.070807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:14:05.070807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:05.070807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:14:05.070807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:05.070807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:14:05.070807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:05.070807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:14:05.070807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:05.070807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:14:05.070807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:14:05.071352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:05.071352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:14:05.071352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:05.071352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:14:05.071352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:14:05.071352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:05.071352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:14:05.071352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:05.071352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:14:05.071688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:05.071688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:14:05.071688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:05.071688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:14:05.071688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:14:05.071688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:14:05.071688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:14:05.071688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:14:05.071688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:14:05.071688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:05.071688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:05.072072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:05.072072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:14:05.072072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:05.072072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:05.072072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:05.072072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:14:05.072072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:05.072072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:14:05.072072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:14:05.072072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:05.072072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:14:05.072072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:05.072072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:14:05.072072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:14:05.072550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:05.072550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:14:05.072550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:05.072550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:14:05.072818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:05.072818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:14:05.072818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:14:05.072818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:14:05.072818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:14:05.072818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:14:05.072818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:05.072818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:14:05.072818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:14:05.072818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:14:05.072818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:14:05.072818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:14:05.072818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:05.072818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:05.072818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:14:05.072818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:14:05.072818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:14:05.073437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:14:05.073437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:14:05.073437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:14:05.073437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:05.073437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:05.073437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:05.073437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:14:05.073437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:05.073437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:05.073437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:05.073437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:14:05.073437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:05.073437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:14:05.073437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:14:05.073437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:05.073437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:14:05.073437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:05.073437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:14:05.073437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:14:05.074146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:05.074146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:14:05.074146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:05.074146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:14:05.079202 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:14:05.079548 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:14:05.079648 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:14:05.079816 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:14:05.080067 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:05.080226 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:05.080329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:05.080775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:14:05.080775 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:05.080775 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:14:05.080775 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:14:05.080775 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:05.080775 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:14:05.080775 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:05.080775 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:14:05.081664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:05.081664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:14:05.081664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:05.081664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:14:05.081664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:14:05.081664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:14:05.081664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:14:05.081664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:05.081664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:05.081664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:05.081664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:14:05.081664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:05.082383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:05.082383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:05.082383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:14:05.082383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:05.082383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:14:05.082383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:14:05.082383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:05.082383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:14:05.082383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:05.082383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:14:05.082383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:14:05.082383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:05.082383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:14:05.082812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:05.082812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:14:05.082812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:05.082812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:14:05.082812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:14:05.082812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:14:05.082812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:14:05.082812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:14:05.082812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:05.082812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:14:05.082812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:14:05.082812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:14:05.082812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:14:05.082812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:14:05.082812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:05.083239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:05.083239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:14:05.083239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:14:05.083239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:14:05.083239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:14:05.083239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:05.083239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:05.083239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:05.083239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:14:05.083239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:05.083239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:05.083239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:05.083239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:14:05.083239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:05.083239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:14:05.083239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:14:05.083239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:05.083239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:14:05.083239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:05.083803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:14:05.083803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:14:05.083803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:05.083803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:14:05.083803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:05.083803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:14:09.811547 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:14:09.812271 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:14:09.812271 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:14:09.812901 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:14:09.836326 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:09.836565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:09.837119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:14:09.837119 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:09.837119 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:14:09.837119 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:14:09.837119 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:09.837119 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:14:09.837119 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:09.837119 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:14:09.838953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.025 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:09.838953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:14:09.838953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:09.838953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:14:09.838953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:14:09.838953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:14:09.838953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:14:09.838953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:09.838953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:09.838953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:09.838953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:14:09.838953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:09.839275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:09.839275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:09.839275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:14:09.839275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:09.839275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:14:09.839275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:14:09.839275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:09.839275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:14:09.839275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:09.839275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:14:09.839275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:14:09.839275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:09.839275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:14:09.839573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:09.839573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:14:09.839573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:09.839573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:14:09.839573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:14:09.839573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:14:09.839573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:14:09.839573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:14:09.839573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:09.839573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:14:09.839573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:14:09.839573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:14:09.839573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:14:09.839573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:14:09.839573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:09.839914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:09.839914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:14:09.839914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:14:09.839914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:14:09.839914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:14:09.839914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:09.839914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:09.839914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:09.839914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:14:09.839914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:09.839914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:09.839914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:09.839914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:14:09.839914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:09.839914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:14:09.839914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:14:09.839914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:09.839914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:14:09.839914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:09.840306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:14:09.840306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:14:09.840306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:09.840306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:14:09.840306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:09.840306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:14:10.049204 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:14:10.050768 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:14:10.050768 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:14:10.051019 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:10.051205 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:10.051309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:10.051924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:14:10.051924 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:10.051924 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:14:10.051924 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:14:10.051924 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:10.051924 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:14:10.051924 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:10.051924 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:14:10.053480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:10.053480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:14:10.053480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:10.053480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:14:10.053480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:14:10.053480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:14:10.053480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:14:10.053480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:10.053480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:10.053480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:10.053480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:14:10.053480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:10.053823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:10.053823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:10.053823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:14:10.053823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:10.053823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:14:10.053823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:14:10.053823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:10.053823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:14:10.053823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:10.053823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:14:10.053823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:14:10.053823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:10.053823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:14:10.054084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:10.054084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:14:10.055113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:10.055113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:14:10.055113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:14:10.055113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:14:10.055113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:14:10.055113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:14:10.055113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:10.055113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:14:10.055113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:14:10.055113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:14:10.055113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:14:10.055113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:14:10.055113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:10.055113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:10.055113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:14:10.055113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:14:10.055491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:14:10.055491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:14:10.055491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:10.055491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:10.055491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:10.055491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:14:10.055491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:10.055491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:10.055491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:10.055491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:14:10.055491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:10.055491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:14:10.055491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:14:10.055491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:10.055491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:14:10.055491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:10.055491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:14:10.055491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:14:10.055491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:10.055867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:14:10.055867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:10.055867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:14:10.059299 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:14:10.059299 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:14:10.059299 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:14:10.059299 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:14:10.059299 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:10.059299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:10.059580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:14:10.059580 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:10.059580 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:14:10.059580 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:14:10.059580 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:10.059580 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:14:10.059580 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:10.059580 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:14:10.060758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:10.060758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:14:10.060758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:14:10.060758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:14:10.060758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:10.060758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:14:10.060758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:10.060758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:14:10.060758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:10.060758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:14:10.060758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:10.060758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:14:10.060758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:10.060758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:14:10.060758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:14:10.061090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:10.061090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:14:10.061090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:10.061090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:14:10.061090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:14:10.061090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:10.061090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:14:10.061090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:10.061090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:14:10.062016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:10.062016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:14:10.062016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:10.062016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:14:10.062016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:14:10.062016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:14:10.062016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:14:10.062016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:14:10.062016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:14:10.062016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:14:10.062016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:10.062016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:10.062836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:10.062836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:14:10.062836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:10.062836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:10.062836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:10.062836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:14:10.062836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:10.062836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:14:10.062836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:14:10.062836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:10.062836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:14:10.062836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:10.062836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:14:10.062836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:14:10.063219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:10.063219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:14:10.063219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:10.063219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:14:10.063219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:10.063219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:14:10.063219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:14:10.063219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:14:10.063219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:14:10.063219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:14:10.063219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:10.063219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:14:10.063219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:14:10.063219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:14:10.063662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:14:10.063662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:14:10.063662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:10.063662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:10.063662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:14:10.063662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:14:10.063662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:14:10.063662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:14:10.063662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:14:10.063662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:14:10.063662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:14:10.063662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:10.063662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:10.063662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:10.063662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:14:10.063662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:10.063662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:10.063662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:10.063662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:14:10.064214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:10.064214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:14:10.064214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:14:10.064214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:10.064214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:14:10.064214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:10.064214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:14:10.064214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:14:10.064214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:10.064214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:14:10.064214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:10.064214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:14:10.069008 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:14:10.069293 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:14:10.069418 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:14:10.069619 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:14:10.070047 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:10.070260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:10.071055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:14:10.071055 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:10.071055 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:14:10.071055 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:14:10.071055 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:10.071055 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:14:10.071055 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:10.071055 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:14:10.072996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:10.072996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:14:10.072996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:14:10.072996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:14:10.072996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:10.072996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:14:10.072996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:10.072996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:14:10.072996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:10.072996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:14:10.072996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:10.072996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:14:10.072996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:10.072996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:14:10.072996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:14:10.073478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:10.073478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:14:10.073478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:10.073478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:14:10.073478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:14:10.073478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:10.073478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:14:10.073478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:10.073478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:14:10.073829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:10.073829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:14:10.073829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:10.073829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:14:10.073829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:14:10.073829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:14:10.073829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:14:10.073829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:14:10.073829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:14:10.073829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:10.073829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:10.074207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:10.074207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:14:10.074207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:10.074207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:10.074207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:10.074207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:14:10.074207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:10.074207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:14:10.074207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:14:10.074207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:10.074207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:14:10.074207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:10.074207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:14:10.074207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:14:10.074643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:10.074643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:14:10.074643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:10.074643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:14:10.075564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:10.075564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:14:10.075564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:14:10.075564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:14:10.075564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:14:10.075564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:14:10.075564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:10.075564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:14:10.075564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:14:10.075564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:14:10.075564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:14:10.075564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:14:10.075564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:10.075564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:10.075564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:14:10.075564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:14:10.075564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:14:10.076065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:14:10.076065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:14:10.076065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:14:10.076065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:10.076065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:10.076065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:10.076065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:14:10.076065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:10.076065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:10.076065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:10.076065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:14:10.076065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:10.076065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:14:10.076065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:14:10.076065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:10.076065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:14:10.076065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:10.076065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:14:10.076065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:14:10.076678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:10.076678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:14:10.076678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:10.076678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:14:10.079581 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:14:10.079942 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:14:10.080058 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:14:10.080434 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:14:10.080625 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:10.080831 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:10.080950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:10.081472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:14:10.081472 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:10.081472 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:14:10.081472 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:14:10.081472 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:10.081472 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:14:10.081472 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:10.081472 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:14:10.082613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:10.082613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:14:10.082613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:10.082613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:14:10.082613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:14:10.082613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:14:10.082613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:14:10.082613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:10.082613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:10.082613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:10.082613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:14:10.082613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:10.082984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:10.082984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:10.082984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:14:10.082984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:10.082984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:14:10.082984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:14:10.082984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:10.082984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:14:10.082984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:10.082984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:14:10.082984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:14:10.082984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:10.082984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:14:10.083347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:10.083347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:14:10.083638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:10.083638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:14:10.083638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:14:10.083638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:14:10.083638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:14:10.083638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:14:10.083638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:10.083638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:14:10.083638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:14:10.083638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:14:10.083638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:14:10.083638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:14:10.083638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:10.083638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:10.083638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:14:10.083638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:14:10.084260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:14:10.084260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:14:10.084260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:10.084260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:10.084260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:10.084260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:14:10.084260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:10.084260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:10.084260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:10.084260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:14:10.084260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:10.084260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:14:10.084260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:14:10.084260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:10.084260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:14:10.084260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:10.084260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:14:10.084260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:14:10.084260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:10.085686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:14:10.085686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:10.085686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:14:14.812153 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:14:14.812796 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:14:14.812921 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:14:14.813821 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:14:14.845668 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:14.845840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:14.846492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:14:14.846492 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:14.846492 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:14:14.846492 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:14:14.846492 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:14.846492 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:14:14.846492 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:14.846492 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:14:14.848224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:14.848224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:14:14.848224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:14.848224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:14:14.848224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:14:14.848224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:14:14.848224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:14:14.848224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:14.848224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:14.848224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:14.848224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:14:14.848224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:14.848578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:14.848578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:14.848578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:14:14.848578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:14.848578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:14:14.848578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:14:14.848578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:14.848578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:14:14.848578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:14.848578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:14:14.848578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:14:14.848578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:14.848578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:14:14.848859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:14.848859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:14:14.848986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:14.848986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:14:14.848986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:14:14.848986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:14:14.848986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:14:14.848986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:14:14.848986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:14.848986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:14:14.848986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:14:14.848986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:14:14.848986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:14:14.848986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:14:14.848986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:14.848986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:14.848986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:14:14.848986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:14:14.849373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:14:14.849373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:14:14.849373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:14.849373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:14.849373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:14.849373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:14:14.849373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:14.849373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:14.849373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:14.849373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:14:14.849373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:14.849373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:14:14.849373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:14:14.849373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:14.849373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:14:14.849373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:14.849373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:14:14.849373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:14:14.849373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:14.849784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:14:14.849784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:14.849784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:14:15.050036 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:14:15.050992 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:14:15.050992 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:14:15.052247 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:15.052735 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:15.052735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:15.053597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:14:15.053597 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:15.053597 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:14:15.053597 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:14:15.053597 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:15.053597 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:14:15.053597 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:15.053597 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:14:15.054886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:15.054886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:14:15.054886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:15.054886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:14:15.054886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:14:15.054886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:14:15.054886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:14:15.054886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:15.054886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:15.054886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:15.054886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:14:15.054886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:15.055694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:15.055694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:15.055694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:14:15.055694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:15.055694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:14:15.055694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:14:15.055694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:15.055694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:14:15.055694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:15.055694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:14:15.055694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:14:15.055694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:15.055694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:14:15.056197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:15.056197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:14:15.056197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:15.056197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:14:15.056197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:14:15.056197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:14:15.056197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:14:15.056197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:14:15.056197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:15.056197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:14:15.056197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:14:15.056197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:14:15.056197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:14:15.056197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:14:15.056197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:15.056674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:15.056674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:14:15.056674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:14:15.056674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:14:15.056674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:14:15.056674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:15.056674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:15.056674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:15.056674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:14:15.056674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:15.056674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:15.056674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:15.056674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:14:15.056674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:15.056674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:14:15.056674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:14:15.056674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:15.056674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:14:15.056674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:15.057311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:14:15.057311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:14:15.057311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:15.057311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:14:15.057311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:15.057311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:14:15.058708 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:14:15.058708 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:14:15.058708 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:14:15.058708 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:14:15.059228 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:15.059390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:15.059892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:14:15.059892 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:15.059892 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:14:15.059892 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:14:15.059892 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:15.059892 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:14:15.059892 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:15.059892 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:14:15.060702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:15.060702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:14:15.060702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:14:15.060702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:14:15.060702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:15.060702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:14:15.060702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:15.060702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:14:15.060702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:15.060702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:14:15.060702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:15.060702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:14:15.060702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:15.060702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:14:15.060702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:14:15.061254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:15.061254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:14:15.061254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:15.061254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:14:15.061254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:14:15.061254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:15.061254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:14:15.061254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:15.061254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:14:15.061638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:15.061638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:14:15.061638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:15.061638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:14:15.061638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:14:15.061638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:14:15.061638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:14:15.061638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:14:15.061638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:14:15.061638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:14:15.061638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:15.061638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:15.061985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:15.061985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:14:15.061985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:15.061985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:15.061985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:15.061985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:14:15.061985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:15.061985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:14:15.061985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:14:15.061985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:15.061985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:14:15.061985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:15.061985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:14:15.061985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:14:15.062472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:15.062472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:14:15.062472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:15.062472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:14:15.062610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:15.062610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:14:15.062610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:14:15.062610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:14:15.062610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:14:15.062610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:14:15.062610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:15.062610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:14:15.062610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:14:15.062610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:14:15.062610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:14:15.062610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:14:15.062610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:15.062610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:15.062610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:14:15.062610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:14:15.063094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:14:15.063094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:14:15.063094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:14:15.063094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:14:15.063094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:14:15.063094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:15.063094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:15.063094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:15.063094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:14:15.063094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:15.063094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:15.063094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:15.063094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:14:15.063094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:15.063094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:14:15.063094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:14:15.063094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:15.063094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:14:15.063679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:15.063679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:14:15.063679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:14:15.063679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:15.063679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:14:15.063679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:15.063679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:14:15.069475 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:14:15.069732 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:14:15.069850 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:14:15.069972 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:14:15.070464 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:15.070676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:15.071195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:14:15.071195 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:15.071195 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:14:15.071195 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:14:15.071195 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:15.071195 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:14:15.071195 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:15.071195 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:14:15.072000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:15.072000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:14:15.072000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:14:15.072000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:14:15.072000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:15.072000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:14:15.072000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:15.072000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:14:15.072000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:15.072000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:14:15.072000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:15.072000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:14:15.072000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:15.072000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:14:15.072000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:14:15.072399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:15.072399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:14:15.072399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:15.072399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:14:15.072399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:14:15.072399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:15.072399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:14:15.072399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:15.072399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:14:15.073352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:15.073352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:14:15.073352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:15.073352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:14:15.073352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:14:15.073352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:14:15.073352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:14:15.073352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:14:15.073352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:14:15.073352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:15.073352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:15.073632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:15.073632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:14:15.073632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:15.073632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:15.073632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:15.073632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:14:15.073632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:15.073632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:14:15.073632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:14:15.073632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:15.073632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:14:15.073632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:15.073632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:14:15.073632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:14:15.073915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:15.073915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:14:15.073915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:15.073915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:14:15.075302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:15.075302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:14:15.075302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:14:15.075302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:14:15.075302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:14:15.075302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:14:15.075302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:15.075302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:14:15.075302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:14:15.075302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:14:15.075302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:14:15.075302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:14:15.075302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:15.075302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:15.075302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:14:15.075302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:14:15.075302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:14:15.075680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:14:15.075680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:14:15.075680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:14:15.075680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:15.075680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:15.075680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:15.075680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:14:15.075680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:15.075680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:15.075680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:15.075680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:14:15.075680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:15.075680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:14:15.075680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:14:15.075680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:15.075680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:14:15.075680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:15.075680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:14:15.075680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:14:15.076072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:15.076072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:14:15.076072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:15.076072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:14:15.080061 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:14:15.080308 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:14:15.080362 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:14:15.080520 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:14:15.080911 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:15.081112 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:15.081225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:15.081719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:14:15.081719 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:15.081719 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:14:15.081719 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:14:15.081719 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:15.081719 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:14:15.081719 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:15.081719 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:14:15.082362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:15.082362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:14:15.082362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:15.082362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:14:15.082362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:14:15.082362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:14:15.082362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:14:15.082362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:15.082362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:15.082362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:15.082362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:14:15.082362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:15.082669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:15.082669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:15.082669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:14:15.082669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:15.082669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:14:15.082669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:14:15.082669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:15.082669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:14:15.082669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:15.082669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:14:15.082669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:14:15.082669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:15.082669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:14:15.082936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:15.082936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:14:15.083105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:15.083105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:14:15.083105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:14:15.083105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:14:15.083105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:14:15.083105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:14:15.083105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:15.083105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:14:15.083105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:14:15.083105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:14:15.083105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:14:15.083105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:14:15.083105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:15.083105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:15.083105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:14:15.083105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:14:15.084416 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:14:15.084416 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:14:15.084416 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:15.084416 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:15.084416 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:15.084416 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:14:15.084416 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:15.084416 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:15.084416 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:15.084416 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:14:15.084416 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:15.084416 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:14:15.084416 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:14:15.084416 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:15.084416 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:14:15.084416 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:15.084416 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:14:15.084416 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:14:15.084416 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:15.084788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:14:15.084788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:15.084788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:14:19.812980 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:14:19.813670 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:14:19.813807 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:14:19.814634 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:14:19.846164 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:19.847259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:19.847518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:14:19.847518 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:19.847518 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:14:19.847518 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:14:19.847518 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:19.847518 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:14:19.847518 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:19.847518 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:14:19.848582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:19.848582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:14:19.848582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:19.848582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:14:19.848582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:14:19.848582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:14:19.848582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:14:19.848582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:19.848582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:19.848582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:19.848582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:14:19.848582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:19.848947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:19.848947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:19.848947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:14:19.848947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:19.848947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:14:19.848947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:14:19.848947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:19.848947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:14:19.848947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:19.848947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:14:19.848947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:14:19.848947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:19.848947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:14:19.849279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:19.849279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:14:19.849719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:19.849719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:14:19.849719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:14:19.849719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:14:19.849719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:14:19.849719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:14:19.849719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:19.849719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:14:19.849719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:14:19.849719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:14:19.849719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:14:19.849719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:14:19.849719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:19.849719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:19.849719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:14:19.849719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:14:19.850171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:14:19.850171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:14:19.850171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:19.850171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:19.850171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:19.850171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:14:19.850171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:19.850171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:19.850171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:19.850171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:14:19.850171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:19.850171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:14:19.850171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:14:19.850171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:19.850171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:14:19.850171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:19.850171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:14:19.850171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:14:19.850171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:19.851208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:14:19.851208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:19.851208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:14:20.050986 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:14:20.051153 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:14:20.051226 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:14:20.052518 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:20.052841 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:20.053091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:20.054032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:14:20.054032 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:20.054032 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:14:20.054032 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:14:20.054032 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:20.054032 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:14:20.054032 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:20.054032 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:14:20.055388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:20.055388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:14:20.055388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:20.055388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:14:20.055388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:14:20.055388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:14:20.055388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:14:20.055388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:20.055388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:20.055388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:20.055388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:14:20.055388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:20.055784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:20.055784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:20.055784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:14:20.055784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:20.055784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:14:20.055784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:14:20.055784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:20.055784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:14:20.055784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:20.055784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:14:20.055784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:14:20.055784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:20.055784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:14:20.056104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:20.056104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:14:20.056462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:20.056462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:14:20.056462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:14:20.056462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:14:20.056462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:14:20.056462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:14:20.056462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:20.056462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:14:20.056462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:14:20.056462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:14:20.056462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:14:20.056462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:14:20.056462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:20.056462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:20.056462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:14:20.056462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:14:20.057032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:14:20.057032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:14:20.057032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:20.057032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:20.057032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:20.057032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:14:20.057032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:20.057032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:20.057032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:20.057032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:14:20.057032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:20.057032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:14:20.057032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:14:20.057032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:20.057032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:14:20.057032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:20.057032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:14:20.057032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:14:20.057032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:20.057637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:14:20.057637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:20.057637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:14:20.058547 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:14:20.058814 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:14:20.058927 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:14:20.059058 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:14:20.059380 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:20.059601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:20.060104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:14:20.060104 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:20.060104 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:14:20.060104 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:14:20.060104 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:20.060104 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:14:20.060104 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:20.060104 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:14:20.060908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:20.060908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:14:20.060908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:14:20.060908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:14:20.060908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:20.060908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:14:20.060908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:20.060908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:14:20.060908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:20.060908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:14:20.060908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:20.060908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:14:20.060908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:20.060908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:14:20.060908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:14:20.061439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:20.061439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:14:20.061439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:20.061439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:14:20.061439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:14:20.061439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:20.061439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:14:20.061439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:20.061439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:14:20.061926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:20.061926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:14:20.061926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:20.061926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:14:20.061926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:14:20.061926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:14:20.061926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:14:20.061926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:14:20.061926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:14:20.061926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:14:20.061926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:20.061926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:20.062335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:20.062335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:14:20.062335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:20.062335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:20.062335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:20.062335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:14:20.062335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:20.062335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:14:20.062335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:14:20.062335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:20.062335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:14:20.062335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:20.062335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:14:20.062335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:14:20.062763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:20.062763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:14:20.062763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:20.062763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:14:20.063266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:20.063266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:14:20.063266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:14:20.063266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:14:20.063266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:14:20.063266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:14:20.063266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:20.063266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:14:20.063266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:14:20.063266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:14:20.063266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:14:20.063266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:14:20.063266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:20.063266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:20.063266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:14:20.063266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:14:20.063620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:14:20.063620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:14:20.063620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:14:20.063620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:14:20.063620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:14:20.063620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:20.063620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:20.063620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:20.063620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:14:20.063620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:20.063620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:20.063620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:20.063620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:14:20.063620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:20.063620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:14:20.063620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:14:20.063620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:20.063620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:14:20.063992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:20.063992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:14:20.063992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:14:20.063992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:20.063992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:14:20.063992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:20.063992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:14:20.070066 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:14:20.070163 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:14:20.070264 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:14:20.070396 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:14:20.070895 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:20.071263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:20.071933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:14:20.071933 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:20.071933 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:14:20.071933 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:14:20.071933 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:20.071933 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:14:20.071933 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:20.071933 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:14:20.072722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:20.072722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:14:20.072722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:14:20.072722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:14:20.072722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:20.072722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:14:20.072722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:20.072722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:14:20.072722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:20.072722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:14:20.072722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:20.072722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:14:20.072722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:20.072722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:14:20.072722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:14:20.073359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:20.073359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:14:20.073359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:20.073359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:14:20.073359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:14:20.073359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:20.073359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:14:20.073359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:20.073359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:14:20.073733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:20.073733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:14:20.073733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:20.073733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:14:20.073733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:14:20.073733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:14:20.073733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:14:20.073733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:14:20.073733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:14:20.073733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:20.073733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:20.074189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:20.074189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:14:20.074189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:20.074189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:20.074189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:20.074189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:14:20.074189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:20.074189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:14:20.074189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:14:20.074189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:20.074189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:14:20.074189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:20.074189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:14:20.074189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:14:20.074746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:20.074746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:14:20.074746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:20.074746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:14:20.074746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:20.074746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:14:20.074746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:14:20.074746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:14:20.074746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:14:20.074746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:14:20.074746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:20.074746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:14:20.074746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:14:20.074746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:14:20.075376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:14:20.075376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:14:20.075376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:20.075376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:20.075376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:14:20.075376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:14:20.075376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:14:20.075376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:14:20.075376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:14:20.075376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:14:20.075376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:20.075376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:20.075376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:20.075376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:14:20.075376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:20.075376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:20.075376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:20.075376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:14:20.075376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:20.076134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:14:20.076134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:14:20.076134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:20.076134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:14:20.076134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:20.076134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:14:20.076134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:14:20.076134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:20.076134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:14:20.076134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:20.076134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:14:20.080446 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:14:20.080702 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:14:20.080846 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:14:20.080968 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:14:20.081536 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:20.081913 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:20.082136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:20.082890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:14:20.082890 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:20.082890 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:14:20.082890 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:14:20.082890 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:20.082890 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:14:20.082890 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:20.082890 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:14:20.083748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:20.083748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:14:20.083748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:20.083748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:14:20.083748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:14:20.083748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:14:20.083748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:14:20.083748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:20.083748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:20.083748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:20.083748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:14:20.083748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:20.084169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:20.084169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:20.084169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:14:20.084169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:20.084169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:14:20.084169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:14:20.084169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:20.084169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:14:20.084169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:20.084169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:14:20.084169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:14:20.084169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:20.084169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:14:20.084527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:20.084527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:14:20.084696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:20.084696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:14:20.084696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:14:20.084696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:14:20.084696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:14:20.084696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:14:20.084696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:20.084696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:14:20.084696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:14:20.084696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:14:20.084696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:14:20.084696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:14:20.084696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:20.084696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:20.084696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:14:20.084696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:14:20.085359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:14:20.085359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:14:20.085359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:20.085359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:20.085359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:20.085359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:14:20.085359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:20.085359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:20.085359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:20.085359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:14:20.085359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:20.085359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:14:20.085359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:14:20.085359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:20.085359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:14:20.085359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:20.085359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:14:20.085359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:14:20.085359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:20.085932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:14:20.085932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:20.085932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:14:24.813716 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:14:24.814366 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:14:24.814490 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:14:24.815278 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:14:24.824779 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG dbcounter [-] [105581] Writing DB stats neutron:SELECT=240 {{(pid=105581) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 14:14:24.854175 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:24.854441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:24.854816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:14:24.854816 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:24.854816 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:14:24.854816 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:14:24.854816 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:24.854816 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:14:24.854816 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:24.854816 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:14:24.855487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:24.855487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:14:24.855487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:24.855487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:14:24.855487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:14:24.855487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:14:24.855487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:14:24.855487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:24.855487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:24.855487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:24.855487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:14:24.855487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:24.856091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:24.856091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:24.856091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:14:24.856091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:24.856091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:14:24.856091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:14:24.856091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:24.856091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:14:24.856091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:24.856091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:14:24.856091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:14:24.856091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:24.856091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:14:24.856750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:24.856750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:14:24.856750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:24.856750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:14:24.856750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:14:24.856750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:14:24.856750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:14:24.856750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:14:24.856750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:24.856750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:14:24.856750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:14:24.856750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:14:24.856750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:14:24.856750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:14:24.856750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:24.857439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:24.857439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:14:24.857439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:14:24.857439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:14:24.857439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:14:24.857439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:24.857439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:24.857439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:24.857439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:14:24.857439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:24.857439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:24.857439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:24.857439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:14:24.857439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:24.857439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:14:24.857439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:14:24.857439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:24.857439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:14:24.857439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:24.858256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:14:24.858256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:14:24.858256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:24.858256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:14:24.858256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:24.858256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:14:25.051024 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:14:25.051928 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:14:25.052151 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:14:25.053101 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:25.053309 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:25.053437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:25.054030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:14:25.054030 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:25.054030 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:14:25.054030 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:14:25.054030 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:25.054030 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:14:25.054030 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:25.054030 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:14:25.056282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:25.056282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:14:25.056282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:25.056282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:14:25.056282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:14:25.056282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:14:25.056282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:14:25.056282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:25.056282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:25.056282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:25.056282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:14:25.056282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:25.056722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:25.056722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:25.056722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:14:25.056722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:25.056722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:14:25.056722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:14:25.056722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:25.056722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:14:25.056722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:25.056722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:14:25.056722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:14:25.056722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:25.056722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:14:25.057103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:25.057103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:14:25.059242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:25.059242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:14:25.059242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:14:25.059242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:14:25.059242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:14:25.059242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:14:25.059242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:25.059242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:14:25.059242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:14:25.059242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:14:25.059242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:14:25.059242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:14:25.059242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:25.059242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:25.059242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:14:25.059242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:14:25.059698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:14:25.059698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:14:25.059698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:25.059698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:25.059698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:25.059698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:14:25.059698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:25.059698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:25.059698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:25.059698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:14:25.059698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:25.059698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:14:25.059698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:14:25.059698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:25.059698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:14:25.059698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:25.059698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:14:25.059698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:14:25.059698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:25.060231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:14:25.060231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:25.060231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:14:25.060231 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:14:25.060231 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:14:25.060231 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:14:25.060463 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:14:25.060760 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:25.060903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:25.061337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:14:25.061337 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:25.061337 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:14:25.061337 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:14:25.061337 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:25.061337 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:14:25.061337 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:25.061337 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:14:25.062152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:25.062152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:14:25.062152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:14:25.062152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:14:25.062152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:25.062152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:14:25.062152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:25.062152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:14:25.062152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:25.062152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:14:25.062152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:25.062152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:14:25.062152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:25.062152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:14:25.062152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:14:25.062625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:25.062625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:14:25.062625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:25.062625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:14:25.062625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:14:25.062625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:25.062625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:14:25.062625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:25.062625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:14:25.063012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:25.063012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:14:25.063012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:25.063012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:14:25.063012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:14:25.063012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:14:25.063012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:14:25.063012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:14:25.063012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:14:25.063012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:14:25.063012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:25.063012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:25.063484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:25.063484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:14:25.063484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:25.063484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:25.063484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:25.063484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:14:25.063484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:25.063484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:14:25.063484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:14:25.063484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:25.063484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:14:25.063484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:25.063484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:14:25.063484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:14:25.063909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:25.063909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:14:25.063909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:25.063909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:14:25.064185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:25.064185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:14:25.064185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:14:25.064185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:14:25.064185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:14:25.064185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:14:25.064185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:25.064185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:14:25.064185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:14:25.064185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:14:25.064185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:14:25.064185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:14:25.064185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:25.064185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:25.064185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:14:25.064185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:14:25.064832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:14:25.064832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:14:25.064832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:14:25.064832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:14:25.064832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:14:25.064832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:25.064832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:25.064832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:25.064832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:14:25.064832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:25.064832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:25.064832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:25.064832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:14:25.064832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:25.064832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:14:25.064832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:14:25.064832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:25.064832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:14:25.065428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:25.065428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:14:25.065428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:14:25.065428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:25.065428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:14:25.065428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:25.065428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:14:25.070218 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:14:25.070751 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:14:25.070855 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:14:25.071005 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:14:25.071505 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:25.072358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:25.073121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:14:25.073121 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:25.073121 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:14:25.073121 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:14:25.073121 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:25.073121 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:14:25.073121 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:25.073121 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:14:25.074229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:25.074229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:14:25.074229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:14:25.074229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:14:25.074229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:25.074229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:14:25.074229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:25.074229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:14:25.074229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:25.074229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:14:25.074229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:25.074229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:14:25.074229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:25.074229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:14:25.074229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:14:25.074649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:25.074649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:14:25.074649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:25.074649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:14:25.074649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:14:25.074649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:25.074649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:14:25.074649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:25.074649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:14:25.075258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:25.075258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:14:25.075258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:25.075258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:14:25.075258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:14:25.075258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:14:25.075258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:14:25.075258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:14:25.075258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:14:25.075258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:25.075258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:25.075638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:25.075638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:14:25.075638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:25.075638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:25.075638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:25.075638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:14:25.075638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:25.075638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:14:25.075638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:14:25.075638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:25.075638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:14:25.075638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:25.075638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:14:25.075638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:14:25.076006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:25.076006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:14:25.076006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:25.076006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:14:25.077086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:25.077086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:14:25.077086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:14:25.077086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:14:25.077086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:14:25.077086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:14:25.077086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:25.077086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:14:25.077086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:14:25.077086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:14:25.077086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:14:25.077086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:14:25.077086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:25.077086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:25.077086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:14:25.077086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:14:25.077086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:14:25.077641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:14:25.077641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:14:25.077641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:14:25.077641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:25.077641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:25.077641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:25.077641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:14:25.077641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:25.077641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:25.077641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:25.077641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:14:25.077641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:25.077641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:14:25.077641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:14:25.077641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:25.077641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:14:25.077641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:25.077641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:14:25.077641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:14:25.078159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:25.078159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:14:25.078159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:25.078159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:14:25.080956 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:14:25.081528 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:14:25.081653 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:14:25.081854 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:14:25.082203 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:25.082427 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:25.082588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:25.083092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:14:25.083092 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:25.083092 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:14:25.083092 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:14:25.083092 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:25.083092 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:14:25.083092 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:25.083092 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:14:25.084147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:25.084147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:14:25.084147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:25.084147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:14:25.084147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:14:25.084147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:14:25.084147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:14:25.084147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:25.084147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:25.084147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:25.084147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:14:25.084147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:25.084557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:25.084557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:25.084557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:14:25.084557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:25.084557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:14:25.084557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:14:25.084557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:25.084557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:14:25.084557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:25.084557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:14:25.084557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:14:25.084557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:25.084557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:14:25.084903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:25.084903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:14:25.085232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:25.085232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:14:25.085232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:14:25.085232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:14:25.085232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:14:25.085232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:14:25.085232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:25.085232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:14:25.085232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:14:25.085232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:14:25.085232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:14:25.085232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:14:25.085232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:25.085232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:25.085232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:14:25.085232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:14:25.085658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:14:25.085658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:14:25.085658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:25.085658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:25.085658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:25.085658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:14:25.085658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:25.085658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:25.085658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:25.085658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:14:25.085658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:25.085658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:14:25.085658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:14:25.085658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:25.085658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:14:25.085658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:25.085658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:14:25.085658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:14:25.085658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:25.086140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:14:25.086140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:25.086140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:14:29.814521 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:14:29.815493 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:14:29.815493 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:14:29.816139 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:14:29.850212 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:29.850524 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:29.851226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:14:29.851226 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:29.851226 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:14:29.851226 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:14:29.851226 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:29.851226 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:14:29.851226 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:29.851226 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:14:29.852219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:29.852219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:14:29.852219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:29.852219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:14:29.852219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:14:29.852219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:14:29.852219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:14:29.852219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:29.852219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:29.852219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:29.852219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:14:29.852219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:29.852929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:29.852929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:29.852929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:14:29.852929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:29.852929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:14:29.852929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:14:29.852929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:29.852929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:14:29.852929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:29.852929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:14:29.852929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:14:29.852929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:29.852929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:14:29.853570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:29.853570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:14:29.853570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:29.853570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:14:29.853570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:14:29.853570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:14:29.853570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:14:29.853570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:14:29.853570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:29.853570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:14:29.853570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:14:29.853570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:14:29.853570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:14:29.853570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:14:29.853570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:29.854292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:29.854292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:14:29.854292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:14:29.854292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:14:29.854292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:14:29.854292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:29.854292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:29.854292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:29.854292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:14:29.854292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:29.854292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:29.854292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:29.854292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:14:29.854292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:29.854292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:14:29.854292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:14:29.854292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:29.854292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:14:29.854292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:29.855260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:14:29.855260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:14:29.855260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:29.855260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:14:29.855260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:29.855260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:14:30.051828 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:14:30.052498 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:14:30.052612 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:14:30.053551 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:30.053797 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:30.053946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:30.054690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:14:30.054690 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:30.054690 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:14:30.054690 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:14:30.054690 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:30.054690 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:14:30.054690 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:30.054690 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:14:30.056427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:30.056427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:14:30.056427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:30.056427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:14:30.056427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:14:30.056427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:14:30.056427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:14:30.056427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:30.056427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:30.056427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:30.056427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:14:30.056427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:30.056850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:30.056850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:30.056850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:14:30.056850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:30.056850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:14:30.056850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:14:30.056850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:30.056850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:14:30.056850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:30.056850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:14:30.056850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:14:30.056850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:30.056850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:14:30.057266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:30.057266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:14:30.058218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:30.058218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:14:30.058218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:14:30.058218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:14:30.058218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:14:30.058218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:14:30.058218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:30.058218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:14:30.058218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:14:30.058218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:14:30.058218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:14:30.058218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:14:30.058218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:30.058218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:30.058218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:14:30.058218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:14:30.058778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:14:30.058778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:14:30.058778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:30.058778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:30.058778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:30.058778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:14:30.058778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:30.058778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:30.058778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:30.058778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:14:30.058778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:30.058778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:14:30.058778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:14:30.058778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:30.058778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:14:30.058778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:30.058778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:14:30.058778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:14:30.058778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:30.059364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:14:30.059364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:30.059364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:14:30.060035 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:14:30.060618 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:14:30.060618 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:14:30.060813 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:14:30.061252 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:30.061425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:30.062422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:14:30.062422 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:30.062422 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:14:30.062422 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:14:30.062422 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:30.062422 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:14:30.062422 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:30.062422 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:14:30.063580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:30.063580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:14:30.063580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:14:30.063580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:14:30.063580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:30.063580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:14:30.063580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:30.063580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:14:30.063580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:30.063580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:14:30.063580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:30.063580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:14:30.063580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:30.063580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:14:30.063580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:14:30.064074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:30.064074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:14:30.064074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:30.064074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:14:30.064074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:14:30.064074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:30.064074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:14:30.064074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:30.064074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:14:30.064590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:30.064590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:14:30.064590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:30.064590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:14:30.064590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:14:30.064590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:14:30.064590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:14:30.064590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:14:30.064590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:14:30.064590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:14:30.064590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:30.064590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:30.064984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:30.064984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:14:30.064984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:30.064984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:30.064984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:30.064984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:14:30.064984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:30.064984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:14:30.064984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:14:30.064984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:30.064984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:14:30.064984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:30.064984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:14:30.064984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:14:30.065356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:30.065356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:14:30.065356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:30.065356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:14:30.065487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:30.065487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:14:30.065487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:14:30.065487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:14:30.065487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:14:30.065487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:14:30.065487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:30.065487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:14:30.065487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:14:30.065487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:14:30.065487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:14:30.065487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:14:30.065487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:30.065487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:30.065487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:14:30.065487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:14:30.065949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:14:30.065949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:14:30.065949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:14:30.065949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:14:30.065949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:14:30.065949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:30.065949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:30.065949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:30.065949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:14:30.065949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:30.065949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:30.065949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:30.065949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:14:30.065949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:30.065949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:14:30.065949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:14:30.065949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:30.065949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:14:30.066467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:30.066467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:14:30.066467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:14:30.066467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:30.066467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:14:30.066467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:30.066467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:14:30.070893 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:14:30.071327 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:14:30.071327 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:14:30.071487 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:14:30.071875 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:30.072133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:30.072697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:14:30.072697 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:30.072697 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:14:30.072697 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:14:30.072697 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:30.072697 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:14:30.072697 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:30.072697 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:14:30.073526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:30.073526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:14:30.073526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:14:30.073526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:14:30.073526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:30.073526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:14:30.073526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:30.073526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:14:30.073526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:30.073526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:14:30.073526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:30.073526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:14:30.073526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:30.073526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:14:30.073526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:14:30.073946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:30.073946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:14:30.073946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:30.073946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:14:30.073946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:14:30.073946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:30.073946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:14:30.073946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:30.073946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:14:30.074386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:30.074386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:14:30.074386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:30.074386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:14:30.074386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:14:30.074386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:14:30.074386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:14:30.074386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:14:30.074386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:14:30.074386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:30.074386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:30.074773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:30.074773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:14:30.074773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:30.074773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:30.074773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:30.074773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:14:30.074773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:30.074773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:14:30.074773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:14:30.074773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:30.074773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:14:30.074773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:30.074773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:14:30.074773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:14:30.075170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:30.075170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:14:30.075170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:30.075170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:14:30.075717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:30.075717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:14:30.075717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:14:30.075717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:14:30.075717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:14:30.075717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:14:30.075717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:30.075717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:14:30.075717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:14:30.075717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:14:30.075717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:14:30.075717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:14:30.075717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:30.075717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:30.075717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:14:30.075717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:14:30.075717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:14:30.076187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:14:30.076187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:14:30.076187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:14:30.076187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:30.076187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:30.076187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:30.076187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:14:30.076187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:30.076187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:30.076187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:30.076187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:14:30.076187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:30.076187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:14:30.076187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:14:30.076187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:30.076187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:14:30.076187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:30.076187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:14:30.076187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:14:30.076729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:30.076729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:14:30.076729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:30.076729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:14:30.081865 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:14:30.082114 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:14:30.082228 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:14:30.082500 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:14:30.083085 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:30.083493 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:30.083747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:30.084624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:14:30.084624 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:30.084624 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:14:30.084624 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:14:30.084624 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:30.084624 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:14:30.084624 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:30.084624 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:14:30.086695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:30.086695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:14:30.086695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:30.086695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:14:30.086695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:14:30.086695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:14:30.086695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:14:30.086695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:30.086695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:30.086695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:30.086695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:14:30.086695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:30.087118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:30.087118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:30.087118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:14:30.087118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:30.087118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:14:30.087118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:14:30.087118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:30.087118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:14:30.087118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:30.087118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:14:30.087118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:14:30.087118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:30.087118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:14:30.087491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:30.087491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:14:30.088506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:30.088506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:14:30.088506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:14:30.088506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:14:30.088506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:14:30.088506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:14:30.088506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:30.088506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:14:30.088506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:14:30.088506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:14:30.088506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:14:30.088506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:14:30.088506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:30.088506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:30.088506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:14:30.088506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:14:30.088961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:14:30.088961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:14:30.088961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:30.088961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:30.088961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:30.088961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:14:30.088961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:30.088961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:30.088961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:30.088961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:14:30.088961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:30.088961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:14:30.088961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:14:30.088961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:30.088961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:14:30.088961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:30.088961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:14:30.088961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:14:30.088961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:30.089589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:14:30.089589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:30.089589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:14:34.269372 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:14:34.270191 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_for_inconsistencies {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:14:34.270191 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Checking Neutron and OVN revision consistency. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:14:34.278822 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Synchronizing Neutron and OVN databases started {{(pid=105581) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:440}}
May 09 14:14:34.278938 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Number of inconsistencies found at create/update: networks=4, routers=1, router_ports=1 {{(pid=105581) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:419}}
May 09 14:14:34.279041 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Fixing resource 0b4a5724-2518-4e33-a6d3-0d92c749f303 (type: networks) at create/update {{(pid=105581) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 14:14:34.315393 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-0b4a5724-2518-4e33-a6d3-0d92c749f303, resource={'id': '0b4a5724-2518-4e33-a6d3-0d92c749f303', 'name': 'private', 'tenant_id': '5796380176014a32af8ff613b146faed', 'project_id': '5796380176014a32af8ff613b146faed', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['23eb2e2e-ec2f-4f98-bc47-7125a2f92952', 'db2636e9-0880-488d-9aa3-e100ce5eeb9f'], 'standard_attr_id': 12, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-09T14:04:20Z', 'updated_at': '2026-05-09T14:04:23Z', 'revision_number': 3, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 52278}, resource_type=networks, if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:34.315842 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-0b4a5724-2518-4e33-a6d3-0d92c749f303, col_values=(('external_ids', {'neutron:network_name': 'private', 'neutron:mtu': '1372', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:34.316432 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-f71b8cb1-fe70-438c-84f9-bbb7008a596d, external_ids_update=None, columns={'tag': 52278}, if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:34.316701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:34.317805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:14:34.317805 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:34.317805 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:14:34.317805 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:14:34.317805 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:34.317805 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:14:34.317805 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:34.317805 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:14:34.319383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Failed to fix resource 0b4a5724-2518-4e33-a6d3-0d92c749f303 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:34.319383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:14:34.319383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 14:14:34.319383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 14:14:34.319383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 14:14:34.319383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 14:14:34.319383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 09 14:14:34.319383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:14:34.319383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:34.319383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:34.319383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:34.319821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:14:34.319821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:34.319821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:34.319821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:34.319821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:14:34.319821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:34.319821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:14:34.319821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:14:34.319821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:34.319821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:14:34.319821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:34.319821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:14:34.319821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:14:34.319821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:34.320085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:14:34.320085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:34.320085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:14:34.320085 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Fixing resource a7d094cd-23f3-4e0c-9e88-24be6ee17b2c (type: networks) at create/update {{(pid=105581) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 14:14:34.351112 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-a7d094cd-23f3-4e0c-9e88-24be6ee17b2c, resource={'id': 'a7d094cd-23f3-4e0c-9e88-24be6ee17b2c', 'name': 'public', 'tenant_id': '5796380176014a32af8ff613b146faed', 'project_id': '5796380176014a32af8ff613b146faed', 'admin_state_up': True, 'mtu': 1430, 'status': 'ACTIVE', 'subnets': ['2f0821e3-c6a2-49d6-80ee-3480a17ba8d2', '644269b7-6b84-4d8a-ad28-d59a7c6b7c5e'], 'standard_attr_id': 18, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': True, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'is_default': True, 'tags': [], 'created_at': '2026-05-09T14:04:26Z', 'updated_at': '2026-05-09T14:04:34Z', 'revision_number': 3, 'provider:network_type': 'flat', 'provider:physical_network': 'public', 'provider:segmentation_id': None}, resource_type=networks, if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:34.351640 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-a7d094cd-23f3-4e0c-9e88-24be6ee17b2c, col_values=(('external_ids', {'neutron:network_name': 'public', 'neutron:mtu': '1430', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'flat', 'neutron:provnet-physical-network': 'public'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false', 'fdb_age_threshold': '300', 'broadcast-arps-to-all-routers': 'true'})), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:34.352274 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-08fea115-9b05-4667-a0d0-487c303084a5, external_ids_update=None, columns={'tag': []}, if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:34.352546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:34.353383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:14:34.353383 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:34.353383 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:14:34.353383 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:14:34.353383 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:34.353383 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:14:34.353383 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:34.353383 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:14:34.356451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Failed to fix resource a7d094cd-23f3-4e0c-9e88-24be6ee17b2c (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:34.356451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:14:34.356451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 14:14:34.356451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 14:14:34.356451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 14:14:34.356451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 14:14:34.356451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 09 14:14:34.356451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:14:34.356451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:34.356451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:34.356451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:34.356946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:14:34.356946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:34.356946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:34.356946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:34.356946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:14:34.356946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:34.356946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:14:34.356946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:14:34.356946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:34.356946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:14:34.356946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:34.356946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:14:34.356946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:14:34.356946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:34.357534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:14:34.357534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:34.357534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:14:34.357534 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Fixing resource 3e80d551-6de5-4370-806b-36d1db5f6fda (type: networks) at create/update {{(pid=105581) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 14:14:34.388518 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-3e80d551-6de5-4370-806b-36d1db5f6fda, resource={'id': '3e80d551-6de5-4370-806b-36d1db5f6fda', 'name': 'admin_net', 'tenant_id': '590c644b03ae4bed862ecf492c6b2eb2', 'project_id': '590c644b03ae4bed862ecf492c6b2eb2', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['453849e5-22a7-43ca-911e-9401ed2b0d74'], 'standard_attr_id': 42, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-09T14:07:41Z', 'updated_at': '2026-05-09T14:07:42Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 35952}, resource_type=networks, if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:34.388868 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-3e80d551-6de5-4370-806b-36d1db5f6fda, col_values=(('external_ids', {'neutron:network_name': 'admin_net', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:34.389392 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-f84050fb-ff65-46ce-b744-016e1ff15440, external_ids_update=None, columns={'tag': 35952}, if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:34.389591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:34.390423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:14:34.390423 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:34.390423 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:14:34.390423 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:14:34.390423 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:34.390423 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:14:34.390423 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:34.390423 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:14:34.391745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Failed to fix resource 3e80d551-6de5-4370-806b-36d1db5f6fda (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:34.391745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:14:34.391745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 14:14:34.391745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 14:14:34.391745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 14:14:34.391745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 14:14:34.391745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 09 14:14:34.391745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:14:34.391745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:34.391745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:34.391745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:34.392063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:14:34.392063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:34.392063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:34.392063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:34.392063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:14:34.392063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:34.392063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:14:34.392063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:14:34.392063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:34.392063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:14:34.392063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:34.392063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:14:34.392063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:14:34.392063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:34.392457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:14:34.392457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:34.392457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:14:34.392457 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Fixing resource 632a2ddb-5634-4bb3-aad0-f05d97abd45e (type: networks) at create/update {{(pid=105581) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 14:14:34.425191 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-632a2ddb-5634-4bb3-aad0-f05d97abd45e, resource={'id': '632a2ddb-5634-4bb3-aad0-f05d97abd45e', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-5urlz', 'tenant_id': '1f17f4a2a7ac4543a1253522e61993d2', 'project_id': '1f17f4a2a7ac4543a1253522e61993d2', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['32b523fe-135f-4f9e-a98b-fc0537fcdb35'], 'standard_attr_id': 52, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-09T14:08:15Z', 'updated_at': '2026-05-09T14:08:16Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 39245}, resource_type=networks, if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:34.425644 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-632a2ddb-5634-4bb3-aad0-f05d97abd45e, col_values=(('external_ids', {'neutron:network_name': 'k8s-clusterapi-cluster-magnum-system-kube-5urlz', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:34.426058 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-bca44f95-fe96-432e-84b8-1edc05e10ce0, external_ids_update=None, columns={'tag': 39245}, if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:34.426205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:34.427120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:14:34.427120 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:34.427120 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:14:34.427120 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:14:34.427120 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:34.427120 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:14:34.427120 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:34.427120 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:14:34.428351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Failed to fix resource 632a2ddb-5634-4bb3-aad0-f05d97abd45e (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:34.428351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:14:34.428351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 14:14:34.428351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 14:14:34.428351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 14:14:34.428351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 14:14:34.428351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 09 14:14:34.428351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:14:34.428351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:34.428351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:34.428351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:34.428828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:14:34.428828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:34.428828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:34.428828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:34.428828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:14:34.428828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:34.428828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:14:34.428828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:14:34.428828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:34.428828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:14:34.428828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:34.428828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:14:34.428828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:14:34.428828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:34.429348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:14:34.429348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:34.429348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:14:34.429348 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Fixing resource 09e042cf-39a9-42c8-b683-b29ffc42b049 (type: routers) at create/update {{(pid=105581) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 14:14:34.510986 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, resource={'id': '09e042cf-39a9-42c8-b683-b29ffc42b049', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-5urlz', 'project_id': '1f17f4a2a7ac4543a1253522e61993d2', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': 'a7d094cd-23f3-4e0c-9e88-24be6ee17b2c', 'external_fixed_ips': [{'subnet_id': '2f0821e3-c6a2-49d6-80ee-3480a17ba8d2', 'ip_address': '172.24.5.175'}, {'subnet_id': '644269b7-6b84-4d8a-ad28-d59a7c6b7c5e', 'ip_address': '2001:db8::d8'}], 'enable_snat': True}, 'gw_port_id': '71ded037-e812-40c0-8d8f-70c141c74323', 'description': 'Created by cluster-api-provider-openstack cluster magnum-system-kube-5urlz', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': 'a7d094cd-23f3-4e0c-9e88-24be6ee17b2c', 'external_fixed_ips': [{'ip_address': '172.24.5.175', 'subnet_id': '2f0821e3-c6a2-49d6-80ee-3480a17ba8d2'}, {'ip_address': '2001:db8::d8', 'subnet_id': '644269b7-6b84-4d8a-ad28-d59a7c6b7c5e'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2026-05-09T14:08:17Z', 'updated_at': '2026-05-09T14:08:19Z', 'revision_number': 3, 'tenant_id': '1f17f4a2a7ac4543a1253522e61993d2'}, resource_type=routers, if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:34.511249 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, columns={'external_ids': {'neutron:router_name': 'k8s-clusterapi-cluster-magnum-system-kube-5urlz', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:34.512536 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-a7d094cd-23f3-4e0c-9e88-24be6ee17b2c, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "71ded037-e812-40c0-8d8f-70c141c74323"')], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:34.512536 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-a7d094cd-23f3-4e0c-9e88-24be6ee17b2c, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "71ded037-e812-40c0-8d8f-70c141c74323"')], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:34.512536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:34.512536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:14:34.512536 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:34.512536 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:14:34.512536 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:14:34.512536 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:34.512536 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:14:34.512536 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:34.512536 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:14:34.512971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Unable to update router 09e042cf-39a9-42c8-b683-b29ffc42b049. Error: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:34.513570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Failed to fix resource 09e042cf-39a9-42c8-b683-b29ffc42b049 (type: routers): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:34.513570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:14:34.513570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 14:14:34.513570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 14:14:34.513570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 14:14:34.513570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 14:14:34.513570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1635, in update_router
May 09 14:14:34.513570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with excutils.save_and_reraise_exception():
May 09 14:14:34.513570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 271, in __exit__
May 09 14:14:34.513570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.force_reraise()
May 09 14:14:34.513570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 233, in force_reraise
May 09 14:14:34.514501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise self.value
May 09 14:14:34.514501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1569, in update_router
May 09 14:14:34.514501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:14:34.514501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:34.514501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:34.514501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:34.514501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:14:34.514501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:34.514501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:34.514501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:34.514501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:14:34.514501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:34.514501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:14:34.514501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:14:34.515016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:34.515016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:14:34.515016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:34.515016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:14:34.515016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:14:34.515016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:34.515016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:14:34.515016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:34.515016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:14:34.515016 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Fixing resource 71ded037-e812-40c0-8d8f-70c141c74323 (type: router_ports) at create/update {{(pid=105581) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 14:14:34.629582 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-71ded037-e812-40c0-8d8f-70c141c74323, resource={'id': '71ded037-e812-40c0-8d8f-70c141c74323', 'name': '', 'network_id': 'a7d094cd-23f3-4e0c-9e88-24be6ee17b2c', 'project_id': '', 'mac_address': 'fa:16:3e:c7:86:37', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '09e042cf-39a9-42c8-b683-b29ffc42b049', 'device_owner': 'network:router_gateway', 'standard_attr_id': 57, 'fixed_ips': [{'subnet_id': '2f0821e3-c6a2-49d6-80ee-3480a17ba8d2', 'ip_address': '172.24.5.175'}, {'subnet_id': '644269b7-6b84-4d8a-ad28-d59a7c6b7c5e', 'ip_address': '2001:db8::d8'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0000173183', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'ovs_create_tap': False, 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2026-05-09T14:08:17Z', 'updated_at': '2026-05-09T14:08:19Z', 'revision_number': 5, 'tenant_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:34.629769 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-71ded037-e812-40c0-8d8f-70c141c74323, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': '2f0821e3-c6a2-49d6-80ee-3480a17ba8d2 644269b7-6b84-4d8a-ad28-d59a7c6b7c5e', 'neutron:network_name': 'neutron-a7d094cd-23f3-4e0c-9e88-24be6ee17b2c', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-09e042cf-39a9-42c8-b683-b29ffc42b049'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['2001:db8::d8/64', '172.24.5.175/24'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:34.630083 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=71ded037-e812-40c0-8d8f-70c141c74323, lrouter_port=lrp-71ded037-e812-40c0-8d8f-70c141c74323, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:34.630162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:34.631334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:14:34.631334 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:34.631334 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:14:34.631334 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:14:34.631334 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:34.631334 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:14:34.631334 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:34.631334 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:14:34.632504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Failed to fix resource 71ded037-e812-40c0-8d8f-70c141c74323 (type: router_ports): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:34.632504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:14:34.632504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 14:14:34.632504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 14:14:34.632504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 14:14:34.632504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 14:14:34.632504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1976, in update_router_port
May 09 14:14:34.632504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:14:34.632504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:34.632504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:34.632504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:34.632848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:14:34.632848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:34.632848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:34.632848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:34.632848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:14:34.632848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:34.632848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:14:34.632848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:14:34.632848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:34.632848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:14:34.632848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:34.632848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:14:34.632848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:14:34.632848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:34.633219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:14:34.633219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:34.633219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:14:34.633219 np0000173183 neutron-ovn-maintenance-worker[105581]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_for_inconsistencies finished in 0.363 seconds
May 09 14:14:34.815216 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:14:34.815735 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:14:34.815929 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:14:34.817383 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:14:34.850975 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:34.851269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:34.852000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:14:34.852000 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:34.852000 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:14:34.852000 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:14:34.852000 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:34.852000 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:14:34.852000 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:34.852000 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:14:34.852785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:34.852785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:14:34.852785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:34.852785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:14:34.852785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:14:34.852785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:14:34.852785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:14:34.852785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:34.852785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:34.852785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:34.852785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:14:34.852785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:34.853112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:34.853112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:34.853112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:14:34.853112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:34.853112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:14:34.853112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:14:34.853112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:34.853112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:14:34.853112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:34.853112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:14:34.853112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:14:34.853112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:34.853112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:14:34.853475 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:34.853475 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:14:34.853538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:34.853538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:14:34.853538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:14:34.853538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:14:34.853538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:14:34.853538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:14:34.853538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:34.853538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:14:34.853538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:14:34.853538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:14:34.853538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:14:34.853538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:14:34.853538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:34.853538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:34.853538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:14:34.853538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:14:34.854485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:14:34.854485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:14:34.854485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:34.854485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:34.854485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:34.854485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:14:34.854485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:34.854485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:34.854485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:34.854485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:14:34.854485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:34.854485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:14:34.854485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:14:34.854485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:34.854485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:14:34.854485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:34.854485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:14:34.854485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:14:34.854485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:34.854930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:14:34.854930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:34.854930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:14:35.052587 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:14:35.053157 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:14:35.053351 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:14:35.054634 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:35.054961 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:35.055203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:35.056287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:14:35.056287 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:35.056287 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:14:35.056287 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:14:35.056287 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:35.056287 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:14:35.056287 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:35.056287 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:14:35.057903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:35.057903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:14:35.057903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:35.057903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:14:35.057903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:14:35.057903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:14:35.057903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:14:35.057903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:35.057903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:35.057903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:35.057903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:14:35.057903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:35.058581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:35.058581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:35.058581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:14:35.058581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:35.058581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:14:35.058581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:14:35.058581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:35.058581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:14:35.058581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:35.058581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:14:35.058581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:14:35.058581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:35.058581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:14:35.059110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:35.059110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:14:35.060039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:35.060039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:14:35.060039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:14:35.060039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:14:35.060039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:14:35.060039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:14:35.060039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:35.060039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:14:35.060039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:14:35.060039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:14:35.060039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:14:35.060039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:14:35.060039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:35.060039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:35.060039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:14:35.060039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:14:35.060744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:14:35.060744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:14:35.060744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:35.060744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:35.060744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:35.060744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:14:35.060744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:35.060744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:35.060744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:35.060744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:14:35.060744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:35.060744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:14:35.060744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:14:35.060744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:35.060744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:14:35.060744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:35.060744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:14:35.060744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:14:35.060744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:35.061365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:14:35.061365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:35.061365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:14:35.061365 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:14:35.061365 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:14:35.061365 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:14:35.061365 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:14:35.061674 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:35.062171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:35.062490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:14:35.062490 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:35.062490 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:14:35.062490 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:14:35.062490 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:35.062490 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:14:35.062490 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:35.062490 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:14:35.063659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:35.063659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:14:35.063659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:14:35.063659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:14:35.063659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:35.063659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:14:35.063659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:35.063659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:14:35.063659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:35.063659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:14:35.063659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:35.063659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:14:35.063659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:35.063659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:14:35.063659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:14:35.064876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:35.064876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:14:35.064876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:35.064876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:14:35.064876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:14:35.064876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:35.064876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:14:35.064876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:35.064876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:14:35.064876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:35.064876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:14:35.064876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:35.064876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:14:35.064876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:14:35.066018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:14:35.066018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:14:35.066018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:14:35.066018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:14:35.066018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:14:35.066018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:35.066018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:35.066018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:35.066018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:14:35.066018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:35.066018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:35.066018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:35.066018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:14:35.066916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:35.066916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:14:35.066916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:14:35.066916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:35.066916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:14:35.066916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:35.066916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:14:35.066916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:14:35.066916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:35.066916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:14:35.066916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:35.066916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:14:35.067341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:35.067341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:14:35.067341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:14:35.067341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:14:35.067341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:14:35.067341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:14:35.067341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:35.067341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:14:35.067341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:14:35.067341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:14:35.067341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:14:35.067341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:14:35.067341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:35.067341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:35.067341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:14:35.067341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:14:35.067924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:14:35.067924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:14:35.067924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:14:35.067924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:14:35.067924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:14:35.067924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:35.067924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:35.067924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:35.067924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:14:35.067924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:35.067924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:35.067924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:35.067924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:14:35.067924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:35.067924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:14:35.067924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:14:35.067924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:35.067924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:14:35.069062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:35.069062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:14:35.069062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:14:35.069062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:35.069062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:14:35.069062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:35.069062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:14:35.071549 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:14:35.071944 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:14:35.072066 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:14:35.072186 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:14:35.072760 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:35.073186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:35.073979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:14:35.073979 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:35.073979 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:14:35.073979 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:14:35.073979 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:35.073979 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:14:35.073979 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:35.073979 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:14:35.074924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:35.074924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:14:35.074924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:14:35.074924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:14:35.074924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:35.074924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:14:35.074924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:35.074924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:14:35.074924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:35.074924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:14:35.074924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:35.074924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:14:35.074924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:35.074924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:14:35.074924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:14:35.075775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:35.075775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:14:35.075775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:35.075775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:14:35.075775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:14:35.075775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:35.075775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:14:35.075775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:35.075775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:14:35.076486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:35.076486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:14:35.076486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:35.076486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:14:35.076486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:14:35.076486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:14:35.076486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:14:35.076486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:14:35.076486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:14:35.076486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:35.076486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:35.077247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:35.077247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:14:35.077247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:35.077247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:35.077247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:35.077247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:14:35.077247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:35.077247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:14:35.077247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:14:35.077247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:35.077247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:14:35.077247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:35.077247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:14:35.077247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:14:35.078189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:35.078189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:14:35.078189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:35.078189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:14:35.078189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:35.078189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:14:35.078189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:14:35.078189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:14:35.078189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:14:35.078189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:14:35.078189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:35.078189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:14:35.078189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:14:35.078189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:14:35.078662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:14:35.078662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:14:35.078662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:35.078662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:35.078662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:14:35.078662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:14:35.078662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:14:35.078662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:14:35.078662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:14:35.078662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:14:35.078662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:35.078662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:35.078662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:35.078662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:14:35.078662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:35.078662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:35.078662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:35.078662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:14:35.078662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:35.079124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:14:35.079124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:14:35.079124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:35.079124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:14:35.079124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:35.079124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:14:35.079124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:14:35.079124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:35.079124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:14:35.079124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:35.079124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:14:35.082251 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:14:35.082712 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:14:35.082936 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:14:35.083127 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:14:35.083715 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:35.084140 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:35.084575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:35.085481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:14:35.085481 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:35.085481 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:14:35.085481 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:14:35.085481 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:35.085481 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:14:35.085481 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:35.085481 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:14:35.086477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:35.086477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:14:35.086477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:35.086477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:14:35.086477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:14:35.086477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:14:35.086477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:14:35.086477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:35.086477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:35.086477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:35.086477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:14:35.086477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:35.086740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:35.086740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:35.086740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:14:35.086740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:35.086740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:14:35.086740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:14:35.086740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:35.086740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:14:35.086740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:35.086740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:14:35.086740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:14:35.086740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:35.086740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:14:35.087002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:35.087002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:14:35.088252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:35.088252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:14:35.088252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:14:35.088252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:14:35.088252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:14:35.088252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:14:35.088252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:35.088252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:14:35.088252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:14:35.088252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:14:35.088252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:14:35.088252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:14:35.088252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:35.088252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:35.088252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:14:35.088252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:14:35.088643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:14:35.088643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:14:35.088643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:35.088643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:35.088643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:35.088643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:14:35.088643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:35.088643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:35.088643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:35.088643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:14:35.088643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:35.088643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:14:35.088643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:14:35.088643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:35.088643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:14:35.088643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:35.088643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:14:35.088643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:14:35.088643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:35.088973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:14:35.088973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:35.088973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:14:39.816008 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:14:39.816886 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:14:39.816886 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:14:39.816886 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:14:39.864361 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:39.864778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:39.865746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:14:39.865746 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:39.865746 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:14:39.865746 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:14:39.865746 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:39.865746 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:14:39.865746 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:39.865746 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:14:39.866793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.050 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:39.866793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:14:39.866793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:39.866793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:14:39.866793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:14:39.866793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:14:39.866793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:14:39.866793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:39.866793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:39.866793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:39.866793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:14:39.866793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:39.867250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:39.867250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:39.867250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:14:39.867250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:39.867250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:14:39.867250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:14:39.867250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:39.867250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:14:39.867250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:39.867250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:14:39.867250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:14:39.867250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:39.867250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:14:39.868188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:39.868188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:14:39.868188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:39.868188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:14:39.868188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:14:39.868188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:14:39.868188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:14:39.868188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:14:39.868188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:39.868188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:14:39.868188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:14:39.868188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:14:39.868188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:14:39.868188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:14:39.868188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:39.868813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:39.868813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:14:39.868813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:14:39.868813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:14:39.868813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:14:39.868813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:39.868813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:39.868813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:39.868813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:14:39.868813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:39.868813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:39.868813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:39.868813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:14:39.868813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:39.868813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:14:39.868813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:14:39.868813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:39.868813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:14:39.868813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:39.870302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:14:39.870302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:14:39.870302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:39.870302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:14:39.870302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:39.870302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:14:40.053191 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:14:40.055687 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:14:40.055687 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:14:40.055687 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:40.055687 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:40.055687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:40.056357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:14:40.056357 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:40.056357 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:14:40.056357 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:14:40.056357 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:40.056357 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:14:40.056357 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:40.056357 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:14:40.057779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:40.057779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:14:40.057779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:40.057779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:14:40.057779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:14:40.057779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:14:40.057779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:14:40.057779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:40.057779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:40.057779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:40.057779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:14:40.057779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:40.058233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:40.058233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:40.058233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:14:40.058233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:40.058233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:14:40.058233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:14:40.058233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:40.058233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:14:40.058233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:40.058233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:14:40.058233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:14:40.058233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:40.058233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:14:40.059293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:40.059293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:14:40.059293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:40.059293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:14:40.059293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:14:40.059293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:14:40.059293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:14:40.059293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:14:40.059293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:40.059293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:14:40.059293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:14:40.059293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:14:40.059293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:14:40.059293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:14:40.059293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:40.059709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:40.059709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:14:40.059709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:14:40.059709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:14:40.059709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:14:40.059709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:40.059709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:40.059709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:40.059709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:14:40.059709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:40.059709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:40.059709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:40.059709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:14:40.059709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:40.059709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:14:40.059709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:14:40.059709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:40.059709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:14:40.059709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:40.060117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:14:40.060117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:14:40.060117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:40.060117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:14:40.060117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:40.060117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:14:40.060907 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:14:40.061238 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:14:40.061379 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:14:40.061567 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:14:40.061907 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:40.062093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:40.062711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:14:40.062711 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:40.062711 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:14:40.062711 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:14:40.062711 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:40.062711 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:14:40.062711 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:40.062711 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:14:40.063625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:40.063625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:14:40.063625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:14:40.063625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:14:40.063625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:40.063625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:14:40.063625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:40.063625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:14:40.063625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:40.063625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:14:40.063625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:40.063625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:14:40.063625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:40.063625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:14:40.063625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:14:40.064000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:40.064000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:14:40.064000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:40.064000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:14:40.064000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:14:40.064000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:40.064000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:14:40.064000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:40.064000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:14:40.064344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:40.064344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:14:40.064344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:40.064344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:14:40.064344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:14:40.064344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:14:40.064344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:14:40.064344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:14:40.064344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:14:40.064344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:14:40.064344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:40.064344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:40.064688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:40.064688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:14:40.064688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:40.064688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:40.064688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:40.064688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:14:40.064688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:40.064688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:14:40.064688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:14:40.064688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:40.064688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:14:40.064688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:40.064688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:14:40.064688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:14:40.065010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:40.065010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:14:40.065010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:40.065010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:14:40.065348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:40.065348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:14:40.065348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:14:40.065348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:14:40.065348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:14:40.065348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:14:40.065348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:40.065348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:14:40.065348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:14:40.065348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:14:40.065348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:14:40.065348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:14:40.065348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:40.065348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:40.065348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:14:40.065348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:14:40.065718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:14:40.065718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:14:40.065718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:14:40.065718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:14:40.065718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:14:40.065718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:40.065718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:40.065718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:40.065718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:14:40.065718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:40.065718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:40.065718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:40.065718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:14:40.065718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:40.065718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:14:40.065718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:14:40.065718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:40.065718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:14:40.066121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:40.066121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:14:40.066121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:14:40.066121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:40.066121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:14:40.066121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:40.066121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:14:40.072287 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:14:40.072682 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:14:40.072815 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:14:40.073059 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:14:40.073642 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:40.074071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:40.074957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:14:40.074957 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:40.074957 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:14:40.074957 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:14:40.074957 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:40.074957 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:14:40.074957 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:40.074957 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:14:40.076418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:40.076418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:14:40.076418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:14:40.076418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:14:40.076418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:40.076418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:14:40.076418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:40.076418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:14:40.076418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:40.076418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:14:40.076418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:40.076418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:14:40.076418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:40.076418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:14:40.076418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:14:40.076749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:40.076749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:14:40.076749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:40.076749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:14:40.076749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:14:40.076749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:40.076749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:14:40.076749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:40.076749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:14:40.077969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:40.077969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:14:40.077969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:40.077969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:14:40.077969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:14:40.077969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:14:40.077969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:14:40.077969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:14:40.077969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:14:40.077969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:40.077969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:40.078255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:40.078255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:14:40.078255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:40.078255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:40.078255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:40.078255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:14:40.078255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:40.078255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:14:40.078255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:14:40.078255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:40.078255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:14:40.078255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:40.078255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:14:40.078255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:14:40.078526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:40.078526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:14:40.078526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:40.078526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:14:40.079754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:40.079754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:14:40.079754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:14:40.079754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:14:40.079754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:14:40.079754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:14:40.079754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:40.079754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:14:40.079754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:14:40.079754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:14:40.079754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:14:40.079754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:14:40.079754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:40.079754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:40.079754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:14:40.079754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:14:40.079754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:14:40.080188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:14:40.080188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:14:40.080188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:14:40.080188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:40.080188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:40.080188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:40.080188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:14:40.080188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:40.080188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:40.080188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:40.080188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:14:40.080188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:40.080188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:14:40.080188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:14:40.080188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:40.080188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:14:40.080188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:40.080188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:14:40.080188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:14:40.080614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:40.080614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:14:40.080614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:40.080614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:14:40.083849 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:14:40.084150 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:14:40.084258 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:14:40.084487 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:14:40.084932 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:40.085261 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:40.085753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:40.086201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:14:40.086201 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:40.086201 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:14:40.086201 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:14:40.086201 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:40.086201 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:14:40.086201 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:40.086201 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:14:40.087282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:40.087282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:14:40.087282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:40.087282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:14:40.087282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:14:40.087282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:14:40.087282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:14:40.087282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:40.087282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:40.087282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:40.087282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:14:40.087282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:40.087739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:40.087739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:40.087739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:14:40.087739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:40.087739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:14:40.087739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:14:40.087739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:40.087739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:14:40.087739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:40.087739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:14:40.087739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:14:40.087739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:40.087739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:14:40.088144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:40.088144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:14:40.088144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:40.088144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:14:40.088144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:14:40.088144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:14:40.088144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:14:40.088144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:14:40.088144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:40.088144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:14:40.088144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:14:40.088144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:14:40.088144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:14:40.088144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:14:40.088144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:40.088635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:40.088635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:14:40.088635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:14:40.088635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:14:40.088635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:14:40.088635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:40.088635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:40.088635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:40.088635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:14:40.088635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:40.088635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:40.088635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:40.088635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:14:40.088635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:40.088635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:14:40.088635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:14:40.088635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:40.088635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:14:40.088635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:40.092336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:14:40.092336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:14:40.092336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:40.092336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:14:40.092336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:40.092336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:14:44.817662 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:14:44.818516 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:14:44.818516 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:14:44.819330 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:14:44.858931 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:44.859124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:44.859727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:14:44.859727 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:44.859727 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:14:44.859727 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:14:44.859727 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:44.859727 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:14:44.859727 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:44.859727 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:14:44.860552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:44.860552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:14:44.860552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:44.860552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:14:44.860552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:14:44.860552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:14:44.860552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:14:44.860552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:44.860552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:44.860552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:44.860552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:14:44.860552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:44.860999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:44.860999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:44.860999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:14:44.860999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:44.860999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:14:44.860999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:14:44.860999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:44.860999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:14:44.860999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:44.860999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:14:44.860999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:14:44.860999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:44.860999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:14:44.861454 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:44.861454 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:14:44.861523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:44.861523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:14:44.861523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:14:44.861523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:14:44.861523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:14:44.861523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:14:44.861523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:44.861523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:14:44.861523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:14:44.861523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:14:44.861523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:14:44.861523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:14:44.861523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:44.861523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:44.861523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:14:44.861523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:14:44.861960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:14:44.861960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:14:44.861960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:44.861960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:44.861960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:44.861960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:14:44.861960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:44.861960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:44.861960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:44.861960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:14:44.861960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:44.861960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:14:44.861960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:14:44.861960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:44.861960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:14:44.861960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:44.861960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:14:44.861960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:14:44.861960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:44.862580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:14:44.862580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:44.862580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:14:45.053925 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:14:45.054555 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:14:45.054676 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:14:45.055539 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:45.055744 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:45.055821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:45.056329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:14:45.056329 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:45.056329 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:14:45.056329 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:14:45.056329 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:45.056329 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:14:45.056329 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:45.056329 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:14:45.057488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:45.057488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:14:45.057488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:45.057488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:14:45.057488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:14:45.057488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:14:45.057488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:14:45.057488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:45.057488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:45.057488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:45.057488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:14:45.057488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:45.058160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:45.058160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:45.058160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:14:45.058160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:45.058160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:14:45.058160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:14:45.058160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:45.058160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:14:45.058160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:45.058160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:14:45.058160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:14:45.058160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:45.058160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:14:45.058770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:45.058770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:14:45.058770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:45.058770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:14:45.058770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:14:45.058770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:14:45.058770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:14:45.058770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:14:45.058770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:45.058770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:14:45.058770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:14:45.058770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:14:45.058770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:14:45.058770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:14:45.058770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:45.059436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:45.059436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:14:45.059436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:14:45.059436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:14:45.059436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:14:45.059436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:45.059436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:45.059436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:45.059436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:14:45.059436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:45.059436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:45.059436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:45.059436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:14:45.059436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:45.059436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:14:45.059436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:14:45.059436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:45.059436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:14:45.059436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:45.060365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:14:45.060365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:14:45.060365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:45.060365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:14:45.060365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:45.060365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:14:45.061490 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:14:45.061749 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:14:45.061888 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:14:45.062037 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:14:45.062429 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:45.062511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:45.062952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:14:45.062952 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:45.062952 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:14:45.062952 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:14:45.062952 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:45.062952 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:14:45.062952 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:45.062952 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:14:45.063731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:45.063731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:14:45.063731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:14:45.063731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:14:45.063731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:45.063731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:14:45.063731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:45.063731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:14:45.063731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:45.063731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:14:45.063731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:45.063731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:14:45.063731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:45.063731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:14:45.063731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:14:45.064487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:45.064487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:14:45.064487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:45.064487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:14:45.064487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:14:45.064487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:45.064487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:14:45.064487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:45.064487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:14:45.064820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:45.064820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:14:45.064820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:45.064820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:14:45.064820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:14:45.064820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:14:45.064820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:14:45.064820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:14:45.064820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:14:45.064820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:14:45.064820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:45.064820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:45.065204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:45.065204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:14:45.065204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:45.065204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:45.065204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:45.065204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:14:45.065204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:45.065204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:14:45.065204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:14:45.065204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:45.065204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:14:45.065204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:45.065204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:14:45.065204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:14:45.065657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:45.065657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:14:45.065657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:45.065657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:14:45.065657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:45.065657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:14:45.065657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:14:45.065657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:14:45.065657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:14:45.065657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:14:45.065657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:45.065657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:14:45.065657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:14:45.065657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:14:45.066093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:14:45.066093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:14:45.066093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:45.066093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:45.066093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:14:45.066093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:14:45.066093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:14:45.066093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:14:45.066093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:14:45.066093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:14:45.066093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:14:45.066093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:45.066093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:45.066093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:45.066093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:14:45.066093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:45.066093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:45.066093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:45.066093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:14:45.066651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:45.066651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:14:45.066651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:14:45.066651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:45.066651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:14:45.066651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:45.066651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:14:45.066651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:14:45.066651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:45.066651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:14:45.066651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:45.066651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:14:45.072480 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:14:45.072789 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:14:45.072907 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:14:45.073062 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:14:45.073548 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:45.073756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:45.074341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:14:45.074341 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:45.074341 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:14:45.074341 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:14:45.074341 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:45.074341 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:14:45.074341 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:45.074341 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:14:45.075259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:45.075259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:14:45.075259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:14:45.075259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:14:45.075259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:45.075259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:14:45.075259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:45.075259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:14:45.075259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:45.075259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:14:45.075259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:45.075259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:14:45.075259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:45.075259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:14:45.075259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:14:45.075752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:45.075752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:14:45.075752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:45.075752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:14:45.075752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:14:45.075752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:45.075752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:14:45.075752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:45.075752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:14:45.076199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:45.076199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:14:45.076199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:45.076199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:14:45.076199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:14:45.076199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:14:45.076199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:14:45.076199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:14:45.076199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:14:45.076199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:45.076199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:45.076691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:45.076691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:14:45.076691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:45.076691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:45.076691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:45.076691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:14:45.076691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:45.076691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:14:45.076691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:14:45.076691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:45.076691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:14:45.076691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:45.076691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:14:45.076691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:14:45.077084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:45.077084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:14:45.077084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:45.077084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:14:45.077295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:45.077295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:14:45.077295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:14:45.077295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:14:45.077295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:14:45.077295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:14:45.077295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:45.077295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:14:45.077295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:14:45.077295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:14:45.077295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:14:45.077295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:14:45.077295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:45.077295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:45.077295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:14:45.077295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:14:45.077295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:14:45.077794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:14:45.077794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:14:45.077794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:14:45.077794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:45.077794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:45.077794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:45.077794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:14:45.077794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:45.077794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:45.077794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:45.077794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:14:45.077794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:45.077794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:14:45.077794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:14:45.077794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:45.077794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:14:45.077794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:45.077794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:14:45.077794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:14:45.078369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:45.078369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:14:45.078369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:45.078369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:14:45.084240 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:14:45.084783 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:14:45.084926 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:14:45.085178 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:14:45.086643 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:45.086643 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:45.086643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:45.086643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:14:45.086643 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:45.086643 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:14:45.086643 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:14:45.086643 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:45.086643 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:14:45.086643 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:45.086643 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:14:45.087888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:45.087888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:14:45.087888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:45.087888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:14:45.087888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:14:45.087888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:14:45.087888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:14:45.087888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:45.087888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:45.087888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:45.087888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:14:45.087888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:45.088308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:45.088308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:45.088308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:14:45.088308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:45.088308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:14:45.088308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:14:45.088308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:45.088308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:14:45.088308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:45.088308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:14:45.088308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:14:45.088308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:45.088308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:14:45.088790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:45.088790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:14:45.089471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:45.089471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:14:45.089471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:14:45.089471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:14:45.089471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:14:45.089471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:14:45.089471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:45.089471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:14:45.089471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:14:45.089471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:14:45.089471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:14:45.089471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:14:45.089471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:45.089471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:45.089471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:14:45.089471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:14:45.090226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:14:45.090226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:14:45.090226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:45.090226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:45.090226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:45.090226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:14:45.090226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:45.090226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:45.090226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:45.090226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:14:45.090226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:45.090226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:14:45.090226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:14:45.090226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:45.090226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:14:45.090226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:45.090226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:14:45.090226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:14:45.090226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:45.091022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:14:45.091022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:45.091022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:14:49.818668 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:14:49.819782 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:14:49.819782 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:14:49.820110 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:14:49.857915 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:49.858563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:49.858795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:14:49.858795 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:49.858795 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:14:49.858795 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:14:49.858795 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:49.858795 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:14:49.858795 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:49.858795 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:14:49.860473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:49.860473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:14:49.860473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:49.860473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:14:49.860473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:14:49.860473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:14:49.860473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:14:49.860473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:49.860473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:49.860473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:49.860473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:14:49.860473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:49.861389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:49.861389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:49.861389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:14:49.861389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:49.861389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:14:49.861389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:14:49.861389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:49.861389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:14:49.861389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:49.861389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:14:49.861389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:14:49.861389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:49.861389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:14:49.861702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:49.861702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:14:49.861702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:49.861702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:14:49.861702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:14:49.861702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:14:49.861702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:14:49.861702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:14:49.861702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:49.861702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:14:49.861702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:14:49.861702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:14:49.861702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:14:49.861702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:14:49.861702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:49.862040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:49.862040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:14:49.862040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:14:49.862040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:14:49.862040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:14:49.862040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:49.862040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:49.862040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:49.862040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:14:49.862040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:49.862040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:49.862040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:49.862040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:14:49.862040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:49.862040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:14:49.862040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:14:49.862040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:49.862040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:14:49.862040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:49.862439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:14:49.862439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:14:49.862439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:49.862439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:14:49.862439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:49.862439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:14:50.054599 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:14:50.056839 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:14:50.057070 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:14:50.057670 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:50.057885 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:50.058000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:50.058473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:14:50.058473 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:50.058473 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:14:50.058473 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:14:50.058473 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:50.058473 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:14:50.058473 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:50.058473 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:14:50.059220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:50.059220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:14:50.059220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:50.059220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:14:50.059220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:14:50.059220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:14:50.059220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:14:50.059220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:50.059220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:50.059220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:50.059220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:14:50.059220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:50.059663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:50.059663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:50.059663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:14:50.059663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:50.059663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:14:50.059663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:14:50.059663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:50.059663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:14:50.059663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:50.059663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:14:50.059663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:14:50.059663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:50.059663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:14:50.060023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:50.060023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:14:50.060023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:50.060023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:14:50.060023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:14:50.060023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:14:50.060023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:14:50.060023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:14:50.060023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:50.060023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:14:50.060023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:14:50.060023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:14:50.060023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:14:50.060023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:14:50.060023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:50.060503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:50.060503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:14:50.060503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:14:50.060503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:14:50.060503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:14:50.060503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:50.060503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:50.060503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:50.060503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:14:50.060503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:50.060503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:50.060503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:50.060503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:14:50.060503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:50.060503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:14:50.060503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:14:50.060503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:50.060503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:14:50.060503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:50.061078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:14:50.061078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:14:50.061078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:50.061078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:14:50.061078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:50.061078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:14:50.062523 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:14:50.062523 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:14:50.062523 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:14:50.062523 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:14:50.062523 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:50.062985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:50.063075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:14:50.063075 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:50.063075 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:14:50.063075 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:14:50.063075 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:50.063075 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:14:50.063075 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:50.063075 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:14:50.063687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:50.063687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:14:50.063687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:14:50.063687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:14:50.063687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:50.063687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:14:50.063687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:50.063687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:14:50.063687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:50.063687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:14:50.063687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:50.063687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:14:50.063687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:50.063687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:14:50.063687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:14:50.065002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:50.065002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:14:50.065002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:50.065002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:14:50.065002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:14:50.065002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:50.065002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:14:50.065002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:50.065002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:14:50.065002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:50.065002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:14:50.065002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:50.065002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:14:50.065002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:14:50.066207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:14:50.066207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:14:50.066207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:14:50.066207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:14:50.066207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:14:50.066207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:50.066207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:50.066207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:50.066207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:14:50.066207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:50.066207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:50.066207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:50.066207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:14:50.066705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:50.066705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:14:50.066705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:14:50.066705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:50.066705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:14:50.066705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:50.066705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:14:50.066705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:14:50.066705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:50.066705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:14:50.066705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:50.066705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:14:50.067602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:50.067602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:14:50.067602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:14:50.067602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:14:50.067602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:14:50.067602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:14:50.067602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:50.067602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:14:50.067602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:14:50.067602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:14:50.067602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:14:50.067602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:14:50.067602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:50.067602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:50.067602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:14:50.067602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:14:50.068425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:14:50.068425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:14:50.068425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:14:50.068425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:14:50.068425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:14:50.068425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:50.068425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:50.068425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:50.068425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:14:50.068425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:50.068425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:50.068425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:50.068425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:14:50.068425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:50.068425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:14:50.068425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:14:50.068425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:50.068425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:14:50.069033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:50.069033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:14:50.069033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:14:50.069033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:50.069033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:14:50.069033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:50.069033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:14:50.072741 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:14:50.073231 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:14:50.073335 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:14:50.073484 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:14:50.073833 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:50.074078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:50.075285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:14:50.075285 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:50.075285 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:14:50.075285 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:14:50.075285 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:50.075285 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:14:50.075285 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:50.075285 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:14:50.076149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:50.076149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:14:50.076149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:14:50.076149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:14:50.076149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:50.076149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:14:50.076149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:50.076149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:14:50.076149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:50.076149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:14:50.076149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:50.076149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:14:50.076149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:50.076149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:14:50.076149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:14:50.076474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:50.076474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:14:50.076474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:50.076474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:14:50.076474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:14:50.076474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:50.076474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:14:50.076474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:50.076474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:14:50.077149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:50.077149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:14:50.077149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:50.077149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:14:50.077149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:14:50.077149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:14:50.077149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:14:50.077149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:14:50.077149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:14:50.077149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:50.077149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:50.077396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:50.077396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:14:50.077396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:50.077396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:50.077396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:50.077396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:14:50.077396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:50.077396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:14:50.077396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:14:50.077396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:50.077396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:14:50.077396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:50.077396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:14:50.077396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:14:50.077723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:50.077723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:14:50.077723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:50.077723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:14:50.078121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:50.078121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:14:50.078121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:14:50.078121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:14:50.078121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:14:50.078121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:14:50.078121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:50.078121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:14:50.078121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:14:50.078121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:14:50.078121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:14:50.078121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:14:50.078121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:50.078121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:50.078121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:14:50.078121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:14:50.078121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:14:50.078776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:14:50.078776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:14:50.078776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:14:50.078776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:50.078776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:50.078776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:50.078776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:14:50.078776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:50.078776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:50.078776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:50.078776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:14:50.078776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:50.078776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:14:50.078776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:14:50.078776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:50.078776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:14:50.078776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:50.078776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:14:50.078776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:14:50.079254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:50.079254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:14:50.079254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:50.079254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:14:50.085918 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:14:50.086250 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:14:50.086365 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:14:50.086564 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:14:50.089703 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:50.089939 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:50.090014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:50.090685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:14:50.090685 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:50.090685 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:14:50.090685 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:14:50.090685 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:50.090685 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:14:50.090685 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:50.090685 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:14:50.091667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:50.091667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:14:50.091667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:50.091667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:14:50.091667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:14:50.091667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:14:50.091667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:14:50.091667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:50.091667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:50.091667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:50.091667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:14:50.091667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:50.092240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:50.092240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:50.092240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:14:50.092240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:50.092240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:14:50.092240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:14:50.092240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:50.092240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:14:50.092240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:50.092240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:14:50.092240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:14:50.092240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:50.092240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:14:50.092635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:50.092635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:14:50.093126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:50.093126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:14:50.093126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:14:50.093126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:14:50.093126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:14:50.093126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:14:50.093126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:50.093126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:14:50.093126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:14:50.093126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:14:50.093126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:14:50.093126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:14:50.093126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:50.093126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:50.093126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:14:50.093126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:14:50.094592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:14:50.094592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:14:50.094592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:50.094592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:50.094592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:50.094592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:14:50.094592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:50.094592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:50.094592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:50.094592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:14:50.094592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:50.094592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:14:50.094592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:14:50.094592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:50.094592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:14:50.094592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:50.094592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:14:50.094592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:14:50.094592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:50.095222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:14:50.095222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:50.095222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:14:54.819117 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:14:54.819715 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:14:54.819715 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:14:54.820292 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:14:54.849265 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:54.849492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:54.850053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:14:54.850053 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:54.850053 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:14:54.850053 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:14:54.850053 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:54.850053 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:14:54.850053 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:54.850053 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:14:54.850821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.030 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:54.850821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:14:54.850821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:54.850821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:14:54.850821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:14:54.850821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:14:54.850821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:14:54.850821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:54.850821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:54.850821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:54.850821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:14:54.850821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:54.851097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:54.851097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:54.851097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:14:54.851097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:54.851097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:14:54.851097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:14:54.851097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:54.851097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:14:54.851097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:54.851097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:14:54.851097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:14:54.851097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:54.851097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:14:54.851327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:54.851327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:14:54.851615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:54.851615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:14:54.851615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:14:54.851615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:14:54.851615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:14:54.851615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:14:54.851615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:54.851615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:14:54.851615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:14:54.851615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:14:54.851615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:14:54.851615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:14:54.851615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:54.851615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:54.851615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:14:54.851615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:14:54.851924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:14:54.851924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:14:54.851924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:54.851924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:54.851924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:54.851924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:14:54.851924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:54.851924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:54.851924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:54.851924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:14:54.851924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:54.851924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:14:54.851924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:14:54.851924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:54.851924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:14:54.851924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:54.851924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:14:54.851924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:14:54.851924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:54.852266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:14:54.852266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:54.852266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:14:55.056907 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:14:55.057520 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:14:55.057724 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:14:55.058680 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:55.058867 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:55.058987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:55.059626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:14:55.059626 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:55.059626 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:14:55.059626 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:14:55.059626 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:55.059626 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:14:55.059626 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:55.059626 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:14:55.061222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:55.061222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:14:55.061222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:55.061222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:14:55.061222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:14:55.061222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:14:55.061222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:14:55.061222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:55.061222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:55.061222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:55.061222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:14:55.061222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:55.061524 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:55.061524 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:55.061524 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:14:55.061524 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:55.061524 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:14:55.061524 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:14:55.061524 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:55.061524 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:14:55.061524 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:55.061524 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:14:55.061524 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:14:55.061524 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:55.061524 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:14:55.061788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:55.061788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:14:55.063102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:55.063102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:14:55.063102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:14:55.063102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:14:55.063102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:14:55.063102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:14:55.063102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:55.063102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:14:55.063102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:14:55.063102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:14:55.063102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:14:55.063102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:14:55.063102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:55.063102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:55.063102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:14:55.063102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:14:55.063428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:14:55.063428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:14:55.063428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:55.063428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:55.063428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:55.063428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:14:55.063428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:55.063428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:55.063428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:55.063428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:14:55.063428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:55.063428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:14:55.063428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:14:55.063428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:55.063428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:14:55.063428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:55.063428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:14:55.063428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:14:55.063428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:55.063888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:14:55.063888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:55.063888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:14:55.063888 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:14:55.063888 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:14:55.063888 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:14:55.063888 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:14:55.064056 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:55.064193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:55.064689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:14:55.064689 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:55.064689 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:14:55.064689 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:14:55.064689 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:55.064689 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:14:55.064689 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:55.064689 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:14:55.065413 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:55.065413 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:14:55.065413 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:14:55.065413 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:14:55.065413 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:55.065413 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:14:55.065413 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:55.065413 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:14:55.065413 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:55.065413 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:14:55.065413 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:55.065413 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:14:55.065413 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:55.065413 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:14:55.065413 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:14:55.065746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:55.065746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:14:55.065746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:55.065746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:14:55.065746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:14:55.065746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:55.065746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:14:55.065746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:55.065746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:14:55.066212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:55.066212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:14:55.066212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:55.066212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:14:55.066212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:14:55.066212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:14:55.066212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:14:55.066212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:14:55.066212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:14:55.066212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:14:55.066212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:55.066212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:55.066475 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:55.066475 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:14:55.066475 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:55.066475 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:55.066475 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:55.066475 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:14:55.066475 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:55.066475 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:14:55.066475 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:14:55.066475 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:55.066475 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:14:55.066475 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:55.066475 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:14:55.066475 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:14:55.066747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:55.066747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:14:55.066747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:55.066747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:14:55.067193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:55.067193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:14:55.067193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:14:55.067193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:14:55.067193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:14:55.067193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:14:55.067193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:55.067193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:14:55.067193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:14:55.067193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:14:55.067193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:14:55.067193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:14:55.067193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:55.067193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:55.067193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:14:55.067193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:14:55.067520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:14:55.067520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:14:55.067520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:14:55.067520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:14:55.067520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:14:55.067520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:55.067520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:55.067520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:55.067520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:14:55.067520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:55.067520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:55.067520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:55.067520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:14:55.067520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:55.067520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:14:55.067520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:14:55.067520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:55.067520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:14:55.067881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:55.067881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:14:55.067881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:14:55.067881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:55.067881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:14:55.067881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:55.067881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:14:55.073559 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:14:55.073808 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:14:55.073924 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:14:55.074052 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:14:55.074354 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:55.074552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:55.074968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:14:55.074968 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:55.074968 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:14:55.074968 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:14:55.074968 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:55.074968 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:14:55.074968 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:55.074968 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:14:55.075720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:55.075720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:14:55.075720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:14:55.075720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:14:55.075720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:55.075720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:14:55.075720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:55.075720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:14:55.075720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:55.075720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:14:55.075720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:55.075720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:14:55.075720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:55.075720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:14:55.075720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:14:55.076066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:55.076066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:14:55.076066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:55.076066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:14:55.076066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:14:55.076066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:55.076066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:14:55.076066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:55.076066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:14:55.076659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:55.076659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:14:55.076659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:55.076659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:14:55.076659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:14:55.076659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:14:55.076659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:14:55.076659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:14:55.076659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:14:55.076659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:55.076659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:55.076887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:55.076887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:14:55.076887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:55.076887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:55.076887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:55.076887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:14:55.076887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:55.076887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:14:55.076887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:14:55.076887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:55.076887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:14:55.076887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:55.076887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:14:55.076887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:14:55.077159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:55.077159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:14:55.077159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:55.077159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:14:55.077746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:55.077746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:14:55.077746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:14:55.077746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:14:55.077746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:14:55.077746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:14:55.077746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:55.077746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:14:55.077746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:14:55.077746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:14:55.077746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:14:55.077746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:14:55.077746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:55.077746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:55.077746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:14:55.077746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:14:55.077746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:14:55.078071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:14:55.078071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:14:55.078071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:14:55.078071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:55.078071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:55.078071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:55.078071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:14:55.078071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:55.078071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:55.078071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:55.078071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:14:55.078071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:55.078071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:14:55.078071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:14:55.078071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:55.078071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:14:55.078071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:55.078071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:14:55.078071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:14:55.078473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:55.078473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:14:55.078473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:55.078473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:14:55.086206 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:14:55.086484 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:14:55.086689 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:14:55.086859 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:14:55.087209 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:55.087393 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:55.087540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:55.088086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:14:55.088086 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:55.088086 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:14:55.088086 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:14:55.088086 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:55.088086 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:14:55.088086 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:55.088086 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:14:55.089430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:55.089430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:14:55.089430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:55.089430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:14:55.089430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:14:55.089430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:14:55.089430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:14:55.089430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:55.089430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:55.089430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:55.089430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:14:55.089430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:55.089691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:55.089691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:55.089691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:14:55.089691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:55.089691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:14:55.089691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:14:55.089691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:55.089691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:14:55.089691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:55.089691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:14:55.089691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:14:55.089691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:55.089691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:14:55.089932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:55.089932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:14:55.090994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:55.090994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:14:55.090994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:14:55.090994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:14:55.090994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:14:55.090994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:14:55.090994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:55.090994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:14:55.090994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:14:55.090994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:14:55.090994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:14:55.090994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:14:55.090994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:55.090994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:55.090994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:14:55.090994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:14:55.091304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:14:55.091304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:14:55.091304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:55.091304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:55.091304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:55.091304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:14:55.091304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:55.091304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:55.091304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:55.091304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:14:55.091304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:55.091304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:14:55.091304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:14:55.091304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:55.091304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:14:55.091304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:55.091304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:14:55.091304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:14:55.091304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:55.091691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:14:55.091691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:55.091691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:14:59.822462 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:14:59.822462 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:14:59.822462 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:14:59.822462 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:14:59.853131 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:14:59.853583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:59.854432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:14:59.854432 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:59.854432 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:14:59.854432 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:14:59.854432 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:59.854432 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:14:59.854432 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:59.854432 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:14:59.855174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:59.855174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:14:59.855174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:59.855174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:14:59.855174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:14:59.855174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:14:59.855174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:14:59.855174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:59.855174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:59.855174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:59.855174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:14:59.855174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:59.855560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:14:59.855560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:59.855560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:14:59.855560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:59.855560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:14:59.855560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:14:59.855560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:59.855560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:14:59.855560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:59.855560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:14:59.855560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:14:59.855560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:59.855560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:14:59.855865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:59.855865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:14:59.855865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:59.855865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:14:59.855865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:14:59.855865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:14:59.855865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:14:59.855865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:14:59.855865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:59.855865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:14:59.855865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:14:59.855865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:14:59.855865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:14:59.855865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:14:59.855865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:14:59.856259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:14:59.856259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:14:59.856259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:14:59.856259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:14:59.856259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:14:59.856259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:59.856259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:59.856259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:14:59.856259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:14:59.856259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:14:59.856259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:14:59.856259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:14:59.856259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:14:59.856259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:14:59.856259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:14:59.856259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:14:59.856259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:14:59.856259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:14:59.856259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:14:59.856767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:14:59.856767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:14:59.856767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:14:59.856767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:14:59.856767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:14:59.856767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:15:00.057581 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:15:00.058119 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:15:00.058119 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:15:00.059047 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:15:00.059587 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:15:00.059672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:00.060481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:15:00.060481 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:00.060481 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:15:00.060481 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:15:00.060481 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:00.060481 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:15:00.060481 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:00.060481 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:15:00.061283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:00.061283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:15:00.061283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:00.061283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:15:00.061283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:15:00.061283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:15:00.061283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:15:00.061283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:00.061283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:00.061283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:00.061283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:15:00.061283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:00.061722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:00.061722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:00.061722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:15:00.061722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:00.061722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:15:00.061722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:15:00.061722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:00.061722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:15:00.061722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:00.061722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:15:00.061722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:15:00.061722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:00.061722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:15:00.062084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:00.062084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:15:00.062084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:00.062084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:15:00.062084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:15:00.062084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:15:00.062084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:15:00.062084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:15:00.062084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:00.062084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:15:00.062084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:15:00.062084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:15:00.062084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:15:00.062084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:15:00.062084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:00.062561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:00.062561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:15:00.062561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:15:00.062561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:15:00.062561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:15:00.062561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:00.062561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:00.062561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:00.062561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:15:00.062561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:00.062561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:00.062561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:00.062561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:15:00.062561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:00.062561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:15:00.062561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:15:00.062561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:00.062561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:15:00.062561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:00.063152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:15:00.063152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:15:00.063152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:00.063152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:15:00.063152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:00.063152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:15:00.064550 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:15:00.064550 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:15:00.064550 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:15:00.064741 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:15:00.065124 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:15:00.065313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:00.066009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:15:00.066009 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:00.066009 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:15:00.066009 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:15:00.066009 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:00.066009 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:15:00.066009 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:00.066009 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:15:00.067637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:00.067637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:15:00.067637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:15:00.067637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:15:00.067637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:00.067637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:15:00.067637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:00.067637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:15:00.067637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:00.067637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:15:00.067637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:00.067637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:15:00.067637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:00.067637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:15:00.067637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:15:00.068173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:00.068173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:15:00.068173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:00.068173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:15:00.068173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:15:00.068173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:00.068173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:15:00.068173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:00.068173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:15:00.068497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:00.068497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:15:00.068497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:00.068497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:15:00.068497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:15:00.068497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:15:00.068497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:15:00.068497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:15:00.068497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:15:00.068497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:15:00.068497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:00.068497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:00.068980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:00.068980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:15:00.068980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:00.068980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:00.068980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:00.068980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:15:00.068980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:00.068980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:15:00.068980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:15:00.068980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:00.068980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:15:00.068980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:00.068980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:15:00.068980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:15:00.069392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:00.069392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:15:00.069392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:00.069392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:15:00.069392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:00.069392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:15:00.069392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:15:00.069392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:15:00.069392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:15:00.069392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:15:00.069392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:00.069392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:15:00.069392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:15:00.069392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:15:00.069736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:15:00.069736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:15:00.069736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:00.069736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:00.069736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:15:00.069736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:15:00.069736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:15:00.069736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:15:00.069736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:15:00.069736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:15:00.069736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:15:00.069736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:00.069736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:00.069736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:00.069736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:15:00.069736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:00.069736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:00.069736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:00.069736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:15:00.070122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:00.070122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:15:00.070122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:15:00.070122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:00.070122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:15:00.070122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:00.070122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:15:00.070122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:15:00.070122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:00.070122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:15:00.070122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:00.070122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:15:00.073865 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:15:00.074077 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:15:00.074237 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:15:00.074334 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:15:00.074665 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:15:00.074907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:00.077897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:15:00.077897 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:00.077897 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:15:00.077897 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:15:00.077897 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:00.077897 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:15:00.077897 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:00.077897 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:15:00.078397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:00.078397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:15:00.078397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:15:00.078397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:15:00.078397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:00.078397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:15:00.078397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:00.078397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:15:00.078397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:00.078397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:15:00.078397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:00.078397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:15:00.078397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:00.078397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:15:00.078397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:15:00.078836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:00.078836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:15:00.078836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:00.078836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:15:00.078836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:15:00.078836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:00.078836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:15:00.078836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:00.078836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:15:00.079023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:00.079023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:15:00.079023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:00.079023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:15:00.079023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:15:00.079023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:15:00.079023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:15:00.079023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:15:00.079023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:15:00.079023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:00.079023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:00.079281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:00.079281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:15:00.079281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:00.079281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:00.079281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:00.079281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:15:00.079281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:00.079281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:15:00.079281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:15:00.079281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:00.079281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:15:00.079281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:00.079281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:15:00.079281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:15:00.079604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:00.079604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:15:00.079604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:00.079604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:15:00.079857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:00.079857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:15:00.079857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:15:00.079857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:15:00.079857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:15:00.079857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:15:00.079857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:00.079857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:15:00.079857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:15:00.079857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:15:00.079857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:15:00.079857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:15:00.079857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:00.079857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:00.079857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:15:00.079857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:15:00.079857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:15:00.080848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:15:00.080848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:15:00.080848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:15:00.080848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:00.080848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:00.080848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:00.080848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:15:00.080848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:00.080848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:00.080848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:00.080848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:15:00.080848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:00.080848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:15:00.080848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:15:00.080848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:00.080848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:15:00.080848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:00.080848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:15:00.080848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:15:00.081797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:00.081797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:15:00.081797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:00.081797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:15:00.086557 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:15:00.086950 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:15:00.086950 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:15:00.086950 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:15:00.087299 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:15:00.087554 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:15:00.087706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:00.088156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:15:00.088156 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:00.088156 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:15:00.088156 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:15:00.088156 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:00.088156 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:15:00.088156 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:00.088156 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:15:00.091451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:00.091451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:15:00.091451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:00.091451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:15:00.091451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:15:00.091451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:15:00.091451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:15:00.091451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:00.091451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:00.091451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:00.091451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:15:00.091451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:00.091871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:00.091871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:00.091871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:15:00.091871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:00.091871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:15:00.091871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:15:00.091871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:00.091871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:15:00.091871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:00.091871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:15:00.091871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:15:00.091871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:00.091871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:15:00.092285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:00.092285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:15:00.092285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:00.092285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:15:00.092285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:15:00.092285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:15:00.092285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:15:00.092285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:15:00.092285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:00.092285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:15:00.092285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:15:00.092285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:15:00.092285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:15:00.092285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:15:00.092285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:00.092910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:00.092910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:15:00.092910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:15:00.092910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:15:00.092910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:15:00.092910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:00.092910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:00.092910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:00.092910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:15:00.092910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:00.092910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:00.092910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:00.092910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:15:00.092910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:00.092910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:15:00.092910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:15:00.092910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:00.092910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:15:00.092910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:00.093462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:15:00.093462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:15:00.093462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:00.093462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:15:00.093462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:00.093462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:15:04.820291 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:15:04.821464 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:15:04.821464 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:15:04.821970 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:15:04.861424 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:15:04.861706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:04.862502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:15:04.862502 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:04.862502 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:15:04.862502 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:15:04.862502 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:04.862502 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:15:04.862502 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:04.862502 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:15:04.863523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:04.863523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:15:04.863523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:04.863523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:15:04.863523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:15:04.863523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:15:04.863523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:15:04.863523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:04.863523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:04.863523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:04.863523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:15:04.863523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:04.864155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:04.864155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:04.864155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:15:04.864155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:04.864155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:15:04.864155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:15:04.864155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:04.864155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:15:04.864155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:04.864155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:15:04.864155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:15:04.864155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:04.864155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:15:04.864879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:04.864879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:15:04.864879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:04.864879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:15:04.864879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:15:04.864879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:15:04.864879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:15:04.864879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:15:04.864879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:04.864879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:15:04.864879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:15:04.864879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:15:04.864879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:15:04.864879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:15:04.864879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:04.865714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:04.865714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:15:04.865714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:15:04.865714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:15:04.865714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:15:04.865714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:04.865714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:04.865714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:04.865714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:15:04.865714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:04.865714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:04.865714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:04.865714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:15:04.865714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:04.865714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:15:04.865714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:15:04.865714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:04.865714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:15:04.865714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:04.866854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:15:04.866854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:15:04.866854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:04.866854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:15:04.866854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:04.866854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:15:05.058458 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:15:05.059192 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:15:05.059383 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:15:05.060375 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:15:05.060652 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:15:05.061036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:05.061452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:15:05.061452 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:05.061452 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:15:05.061452 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:15:05.061452 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:05.061452 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:15:05.061452 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:05.061452 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:15:05.062752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:05.062752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:15:05.062752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:05.062752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:15:05.062752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:15:05.062752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:15:05.062752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:15:05.062752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:05.062752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:05.062752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:05.062752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:15:05.062752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:05.063155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:05.063155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:05.063155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:15:05.063155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:05.063155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:15:05.063155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:15:05.063155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:05.063155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:15:05.063155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:05.063155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:15:05.063155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:15:05.063155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:05.063155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:15:05.063614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:05.063614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:15:05.064786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:05.064786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:15:05.064786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:15:05.064786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:15:05.064786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:15:05.064786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:15:05.064786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:05.064786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:15:05.064786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:15:05.064786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:15:05.064786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:15:05.064786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:15:05.064786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:05.064786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:05.064786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:15:05.064786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:15:05.065377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:15:05.065377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:15:05.065377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:05.065377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:05.065377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:05.065377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:15:05.065377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:05.065377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:05.065377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:05.065377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:15:05.065377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:05.065377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:15:05.065377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:15:05.065377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:05.065377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:15:05.065377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:05.065377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:15:05.065377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:15:05.065377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:05.066427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:15:05.066427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:05.066427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:15:05.066427 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:15:05.066427 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:15:05.066427 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:15:05.066427 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:15:05.066427 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:15:05.066690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:05.067058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:15:05.067058 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:05.067058 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:15:05.067058 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:15:05.067058 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:05.067058 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:15:05.067058 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:05.067058 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:15:05.068019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:05.068019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:15:05.068019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:15:05.068019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:15:05.068019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:05.068019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:15:05.068019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:05.068019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:15:05.068019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:05.068019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:15:05.068019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:05.068019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:15:05.068019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:05.068019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:15:05.068019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:15:05.068509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:05.068509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:15:05.068509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:05.068509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:15:05.068509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:15:05.068509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:05.068509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:15:05.068509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:05.068509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:15:05.069493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:05.069493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:15:05.069493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:05.069493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:15:05.069493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:15:05.069493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:15:05.069493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:15:05.069493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:15:05.069493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:15:05.069493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:15:05.069493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:05.069493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:05.069863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:05.069863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:15:05.069863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:05.069863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:05.069863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:05.069863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:15:05.069863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:05.069863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:15:05.069863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:15:05.069863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:05.069863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:15:05.069863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:05.069863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:15:05.069863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:15:05.070215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:05.070215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:15:05.070215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:05.070215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:15:05.070215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:05.070215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:15:05.070215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:15:05.070215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:15:05.070215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:15:05.070215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:15:05.070215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:05.070215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:15:05.070215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:15:05.070215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:15:05.070612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:15:05.070612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:15:05.070612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:05.070612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:05.070612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:15:05.070612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:15:05.070612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:15:05.070612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:15:05.070612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:15:05.070612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:15:05.070612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:15:05.070612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:05.070612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:05.070612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:05.070612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:15:05.070612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:05.070612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:05.070612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:05.070612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:15:05.071133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:05.071133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:15:05.071133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:15:05.071133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:05.071133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:15:05.071133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:05.071133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:15:05.071133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:15:05.071133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:05.071133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:15:05.071133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:05.071133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:15:05.074143 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:15:05.074355 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:15:05.074441 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:15:05.074542 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:15:05.074894 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:15:05.075113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:05.075633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:15:05.075633 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:05.075633 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:15:05.075633 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:15:05.075633 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:05.075633 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:15:05.075633 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:05.075633 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:15:05.076162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:05.076162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:15:05.076162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:15:05.076162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:15:05.076162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:05.076162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:15:05.076162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:05.076162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:15:05.076162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:05.076162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:15:05.076162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:05.076162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:15:05.076162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:05.076162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:15:05.076162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:15:05.076545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:05.076545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:15:05.076545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:05.076545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:15:05.076545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:15:05.076545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:05.076545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:15:05.076545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:05.076545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:15:05.076826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:05.076826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:15:05.076826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:05.076826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:15:05.076826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:15:05.076826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:15:05.076826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:15:05.076826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:15:05.076826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:15:05.076826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:05.076826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:05.077114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:05.077114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:15:05.077114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:05.077114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:05.077114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:05.077114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:15:05.077114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:05.077114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:15:05.077114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:15:05.077114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:05.077114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:15:05.077114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:05.077114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:15:05.077114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:15:05.077483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:05.077483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:15:05.077483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:05.077483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:15:05.077666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:05.077666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:15:05.077666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:15:05.077666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:15:05.077666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:15:05.077666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:15:05.077666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:05.077666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:15:05.077666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:15:05.077666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:15:05.077666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:15:05.077666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:15:05.077666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:05.077666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:05.077666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:15:05.077666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:15:05.077666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:15:05.078091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:15:05.078091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:15:05.078091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:15:05.078091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:05.078091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:05.078091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:05.078091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:15:05.078091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:05.078091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:05.078091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:05.078091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:15:05.078091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:05.078091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:15:05.078091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:15:05.078091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:05.078091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:15:05.078091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:05.078091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:15:05.078091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:15:05.078599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:05.078599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:15:05.078599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:05.078599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:15:05.086895 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:15:05.087162 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:15:05.087271 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:15:05.087419 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:15:05.087792 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:15:05.088026 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:15:05.088183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:05.088808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:15:05.088808 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:05.088808 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:15:05.088808 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:15:05.088808 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:05.088808 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:15:05.088808 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:05.088808 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:15:05.089652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:05.089652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:15:05.089652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:05.089652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:15:05.089652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:15:05.089652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:15:05.089652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:15:05.089652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:05.089652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:05.089652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:05.089652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:15:05.089652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:05.090021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:05.090021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:05.090021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:15:05.090021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:05.090021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:15:05.090021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:15:05.090021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:05.090021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:15:05.090021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:05.090021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:15:05.090021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:15:05.090021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:05.090021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:15:05.090437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:05.090437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:15:05.090577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:05.090577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:15:05.090577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:15:05.090577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:15:05.090577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:15:05.090577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:15:05.090577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:05.090577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:15:05.090577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:15:05.090577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:15:05.090577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:15:05.090577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:15:05.090577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:05.090577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:05.090577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:15:05.090577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:15:05.091010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:15:05.091010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:15:05.091010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:05.091010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:05.091010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:05.091010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:15:05.091010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:05.091010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:05.091010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:05.091010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:15:05.091010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:05.091010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:15:05.091010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:15:05.091010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:05.091010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:15:05.091010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:05.091010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:15:05.091010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:15:05.091010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:05.091522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:15:05.091522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:05.091522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:15:09.821043 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:15:09.821690 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:15:09.821905 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:15:09.823093 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:15:09.858392 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:15:09.858645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:09.859308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:15:09.859308 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:09.859308 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:15:09.859308 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:15:09.859308 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:09.859308 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:15:09.859308 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:09.859308 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:15:09.860039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:09.860039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:15:09.860039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:09.860039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:15:09.860039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:15:09.860039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:15:09.860039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:15:09.860039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:09.860039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:09.860039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:09.860039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:15:09.860039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:09.860580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:09.860580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:09.860580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:15:09.860580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:09.860580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:15:09.860580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:15:09.860580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:09.860580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:15:09.860580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:09.860580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:15:09.860580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:15:09.860580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:09.860580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:15:09.860911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:09.860911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:15:09.860911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:09.860911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:15:09.860911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:15:09.860911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:15:09.860911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:15:09.860911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:15:09.860911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:09.860911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:15:09.860911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:15:09.860911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:15:09.860911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:15:09.860911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:15:09.860911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:09.861308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:09.861308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:15:09.861308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:15:09.861308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:15:09.861308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:15:09.861308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:09.861308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:09.861308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:09.861308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:15:09.861308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:09.861308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:09.861308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:09.861308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:15:09.861308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:09.861308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:15:09.861308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:15:09.861308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:09.861308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:15:09.861308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:09.862090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:15:09.862090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:15:09.862090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:09.862090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:15:09.862090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:09.862090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:15:10.058939 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:15:10.059573 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:15:10.059768 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:15:10.060968 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:15:10.061257 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:15:10.061426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:10.062070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:15:10.062070 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:10.062070 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:15:10.062070 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:15:10.062070 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:10.062070 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:15:10.062070 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:10.062070 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:15:10.063137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:10.063137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:15:10.063137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:10.063137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:15:10.063137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:15:10.063137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:15:10.063137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:15:10.063137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:10.063137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:10.063137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:10.063137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:15:10.063137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:10.063693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:10.063693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:10.063693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:15:10.063693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:10.063693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:15:10.063693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:15:10.063693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:10.063693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:15:10.063693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:10.063693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:15:10.063693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:15:10.063693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:10.063693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:15:10.064488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:10.064488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:15:10.064488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:10.064488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:15:10.064488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:15:10.064488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:15:10.064488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:15:10.064488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:15:10.064488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:10.064488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:15:10.064488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:15:10.064488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:15:10.064488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:15:10.064488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:15:10.064488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:10.065147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:10.065147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:15:10.065147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:15:10.065147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:15:10.065147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:15:10.065147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:10.065147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:10.065147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:10.065147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:15:10.065147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:10.065147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:10.065147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:10.065147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:15:10.065147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:10.065147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:15:10.065147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:15:10.065147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:10.065147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:15:10.065147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:10.066178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:15:10.066178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:15:10.066178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:10.066178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:15:10.066178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:10.066178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:15:10.066178 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:15:10.066178 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:15:10.066178 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:15:10.066178 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:15:10.066797 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:15:10.066867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:10.067383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:15:10.067383 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:10.067383 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:15:10.067383 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:15:10.067383 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:10.067383 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:15:10.067383 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:10.067383 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:15:10.068171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:10.068171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:15:10.068171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:15:10.068171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:15:10.068171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:10.068171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:15:10.068171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:10.068171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:15:10.068171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:10.068171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:15:10.068171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:10.068171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:15:10.068171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:10.068171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:15:10.068171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:15:10.068975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:10.068975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:15:10.068975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:10.068975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:15:10.068975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:15:10.068975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:10.068975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:15:10.068975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:10.068975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:15:10.069461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:10.069461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:15:10.069461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:10.069461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:15:10.069461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:15:10.069461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:15:10.069461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:15:10.069461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:15:10.069461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:15:10.069461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:15:10.069461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:10.069461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:10.069836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:10.069836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:15:10.069836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:10.069836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:10.069836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:10.069836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:15:10.069836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:10.069836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:15:10.069836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:15:10.069836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:10.069836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:15:10.069836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:10.069836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:15:10.069836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:15:10.070175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:10.070175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:15:10.070175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:10.070175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:15:10.070175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:10.070175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:15:10.070175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:15:10.070175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:15:10.070175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:15:10.070175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:15:10.070175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:10.070175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:15:10.070175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:15:10.070175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:15:10.070529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:15:10.070529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:15:10.070529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:10.070529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:10.070529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:15:10.070529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:15:10.070529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:15:10.070529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:15:10.070529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:15:10.070529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:15:10.070529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:15:10.070529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:10.070529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:10.070529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:10.070529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:15:10.070529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:10.070529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:10.070529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:10.070529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:15:10.070991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:10.070991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:15:10.070991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:15:10.070991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:10.070991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:15:10.070991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:10.070991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:15:10.070991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:15:10.070991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:10.070991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:15:10.070991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:10.070991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:15:10.074551 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:15:10.074729 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:15:10.074803 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:15:10.074910 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:15:10.075252 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:15:10.075519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:10.076078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:15:10.076078 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:10.076078 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:15:10.076078 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:15:10.076078 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:10.076078 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:15:10.076078 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:10.076078 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:15:10.076816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:10.076816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:15:10.076816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:15:10.076816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:15:10.076816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:10.076816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:15:10.076816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:10.076816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:15:10.076816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:10.076816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:15:10.076816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:10.076816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:15:10.076816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:10.076816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:15:10.076816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:15:10.077226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:10.077226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:15:10.077226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:10.077226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:15:10.077226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:15:10.077226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:10.077226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:15:10.077226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:10.077226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:15:10.077477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:10.077477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:15:10.077477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:10.077477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:15:10.077477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:15:10.077477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:15:10.077477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:15:10.077477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:15:10.077477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:15:10.077477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:10.077477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:10.077747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:10.077747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:15:10.077747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:10.077747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:10.077747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:10.077747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:15:10.077747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:10.077747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:15:10.077747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:15:10.077747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:10.077747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:15:10.077747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:10.077747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:15:10.077747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:15:10.078102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:10.078102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:15:10.078102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:10.078102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:15:10.078207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:10.078207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:15:10.078207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:15:10.078207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:15:10.078207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:15:10.078207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:15:10.078207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:10.078207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:15:10.078207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:15:10.078207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:15:10.078207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:15:10.078207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:15:10.078207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:10.078207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:10.078207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:15:10.078207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:15:10.078207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:15:10.078653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:15:10.078653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:15:10.078653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:15:10.078653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:10.078653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:10.078653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:10.078653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:15:10.078653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:10.078653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:10.078653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:10.078653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:15:10.078653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:10.078653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:15:10.078653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:15:10.078653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:10.078653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:15:10.078653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:10.078653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:15:10.078653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:15:10.079097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:10.079097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:15:10.079097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:10.079097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:15:10.087756 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:15:10.088127 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:15:10.088296 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:15:10.088571 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:15:10.089217 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:15:10.090195 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:15:10.090319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:10.090891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:15:10.090891 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:10.090891 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:15:10.090891 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:15:10.090891 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:10.090891 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:15:10.090891 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:10.090891 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:15:10.092221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:10.092221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:15:10.092221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:10.092221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:15:10.092221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:15:10.092221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:15:10.092221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:15:10.092221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:10.092221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:10.092221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:10.092221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:15:10.092221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:10.092626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:10.092626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:10.092626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:15:10.092626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:10.092626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:15:10.092626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:15:10.092626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:10.092626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:15:10.092626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:10.092626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:15:10.092626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:15:10.092626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:10.092626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:15:10.093042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:10.093042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:15:10.093921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:10.093921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:15:10.093921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:15:10.093921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:15:10.093921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:15:10.093921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:15:10.093921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:10.093921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:15:10.093921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:15:10.093921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:15:10.093921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:15:10.093921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:15:10.093921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:10.093921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:10.093921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:15:10.093921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:15:10.094675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:15:10.094675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:15:10.094675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:10.094675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:10.094675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:10.094675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:15:10.094675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:10.094675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:10.094675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:10.094675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:15:10.094675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:10.094675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:15:10.094675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:15:10.094675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:10.094675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:15:10.094675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:10.094675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:15:10.094675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:15:10.094675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:10.095486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:15:10.095486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:10.095486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:15:14.821878 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:15:14.823206 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:15:14.823317 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:15:14.824440 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:15:14.857717 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:15:14.857897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:14.858804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:15:14.858804 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:14.858804 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:15:14.858804 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:15:14.858804 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:14.858804 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:15:14.858804 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:14.858804 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:15:14.859389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:14.859389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:15:14.859389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:14.859389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:15:14.859389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:15:14.859389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:15:14.859389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:15:14.859389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:14.859389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:14.859389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:14.859389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:15:14.859389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:14.859819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:14.859819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:14.859819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:15:14.859819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:14.859819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:15:14.859819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:15:14.859819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:14.859819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:15:14.859819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:14.859819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:15:14.859819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:15:14.859819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:14.859819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:15:14.860200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:14.860200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:15:14.860200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:14.860200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:15:14.860200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:15:14.860200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:15:14.860200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:15:14.860200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:15:14.860200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:14.860200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:15:14.860200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:15:14.860200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:15:14.860200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:15:14.860200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:15:14.860200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:14.860669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:14.860669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:15:14.860669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:15:14.860669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:15:14.860669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:15:14.860669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:14.860669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:14.860669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:14.860669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:15:14.860669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:14.860669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:14.860669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:14.860669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:15:14.860669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:14.860669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:15:14.860669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:15:14.860669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:14.860669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:15:14.860669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:14.861183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:15:14.861183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:15:14.861183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:14.861183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:15:14.861183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:14.861183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:15:15.059879 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:15:15.060167 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:15:15.060289 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:15:15.061360 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:15:15.061667 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:15:15.061868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:15.062726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:15:15.062726 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:15.062726 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:15:15.062726 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:15:15.062726 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:15.062726 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:15:15.062726 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:15.062726 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:15:15.064065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:15.064065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:15:15.064065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:15.064065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:15:15.064065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:15:15.064065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:15:15.064065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:15:15.064065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:15.064065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:15.064065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:15.064065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:15:15.064065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:15.064678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:15.064678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:15.064678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:15:15.064678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:15.064678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:15:15.064678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:15:15.064678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:15.064678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:15:15.064678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:15.064678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:15:15.064678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:15:15.064678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:15.064678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:15:15.065318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:15.065318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:15:15.065318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:15.065318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:15:15.065318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:15:15.065318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:15:15.065318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:15:15.065318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:15:15.065318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:15.065318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:15:15.065318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:15:15.065318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:15:15.065318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:15:15.065318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:15:15.065318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:15.065830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:15.065830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:15:15.065830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:15:15.065830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:15:15.065830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:15:15.065830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:15.065830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:15.065830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:15.065830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:15:15.065830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:15.065830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:15.065830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:15.065830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:15:15.065830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:15.065830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:15:15.065830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:15:15.065830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:15.065830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:15:15.065830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:15.066341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:15:15.066341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:15:15.066341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:15.066341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:15:15.066341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:15.066341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:15:15.066341 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:15:15.066341 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:15:15.066341 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:15:15.066625 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:15:15.066833 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:15:15.067086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:15.067854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:15:15.067854 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:15.067854 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:15:15.067854 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:15:15.067854 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:15.067854 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:15:15.067854 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:15.067854 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:15:15.068626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:15.068626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:15:15.068626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:15:15.068626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:15:15.068626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:15.068626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:15:15.068626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:15.068626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:15:15.068626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:15.068626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:15:15.068626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:15.068626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:15:15.068626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:15.068626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:15:15.068626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:15:15.069057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:15.069057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:15:15.069057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:15.069057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:15:15.069057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:15:15.069057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:15.069057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:15:15.069057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:15.069057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:15:15.069469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:15.069469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:15:15.069469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:15.069469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:15:15.069469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:15:15.069469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:15:15.069469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:15:15.069469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:15:15.069469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:15:15.069469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:15:15.069469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:15.069469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:15.069804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:15.069804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:15:15.069804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:15.069804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:15.069804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:15.069804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:15:15.069804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:15.069804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:15:15.069804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:15:15.069804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:15.069804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:15:15.069804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:15.069804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:15:15.069804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:15:15.070136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:15.070136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:15:15.070136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:15.070136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:15:15.070426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:15.070426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:15:15.070426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:15:15.070426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:15:15.070426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:15:15.070426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:15:15.070426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:15.070426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:15:15.070426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:15:15.070426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:15:15.070426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:15:15.070426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:15:15.070426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:15.070426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:15.070426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:15:15.070426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:15:15.070827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:15:15.070827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:15:15.070827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:15:15.070827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:15:15.070827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:15:15.070827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:15.070827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:15.070827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:15.070827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:15:15.070827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:15.070827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:15.070827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:15.070827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:15:15.070827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:15.070827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:15:15.070827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:15:15.070827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:15.070827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:15:15.071286 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:15.071286 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:15:15.071286 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:15:15.071286 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:15.071286 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:15:15.071286 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:15.071286 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:15:15.075446 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:15:15.075446 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:15:15.075446 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:15:15.075615 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:15:15.076021 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:15:15.076680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:15.077381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:15:15.077381 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:15.077381 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:15:15.077381 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:15:15.077381 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:15.077381 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:15:15.077381 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:15.077381 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:15:15.078042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:15.078042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:15:15.078042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:15:15.078042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:15:15.078042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:15.078042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:15:15.078042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:15.078042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:15:15.078042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:15.078042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:15:15.078042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:15.078042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:15:15.078042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:15.078042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:15:15.078042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:15:15.078448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:15.078448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:15:15.078448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:15.078448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:15:15.078448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:15:15.078448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:15.078448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:15:15.078448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:15.078448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:15:15.078689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:15.078689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:15:15.078689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:15.078689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:15:15.078689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:15:15.078689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:15:15.078689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:15:15.078689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:15:15.078689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:15:15.078689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:15.078689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:15.079023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:15.079023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:15:15.079023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:15.079023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:15.079023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:15.079023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:15:15.079023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:15.079023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:15:15.079023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:15:15.079023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:15.079023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:15:15.079023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:15.079023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:15:15.079023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:15:15.079389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:15.079389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:15:15.079389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:15.079389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:15:15.079389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:15.079389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:15:15.079389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:15:15.079389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:15:15.079389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:15:15.079389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:15:15.079389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:15.079389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:15:15.079389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:15:15.079389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:15:15.079801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:15:15.079801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:15:15.079801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:15.079801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:15.079801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:15:15.079801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:15:15.079801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:15:15.079801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:15:15.079801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:15:15.079801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:15:15.079801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:15.079801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:15.079801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:15.079801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:15:15.079801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:15.079801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:15.079801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:15.079801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:15:15.079801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:15.080302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:15:15.080302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:15:15.080302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:15.080302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:15:15.080302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:15.080302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:15:15.080302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:15:15.080302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:15.080302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:15:15.080302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:15.080302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:15:15.088186 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:15:15.088603 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:15:15.088798 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:15:15.089063 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:15:15.089661 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:15:15.089874 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:15:15.089991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:15.090605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:15:15.090605 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:15.090605 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:15:15.090605 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:15:15.090605 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:15.090605 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:15:15.090605 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:15.090605 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:15:15.091946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:15.091946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:15:15.091946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:15.091946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:15:15.091946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:15:15.091946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:15:15.091946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:15:15.091946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:15.091946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:15.091946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:15.091946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:15:15.091946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:15.092507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:15.092507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:15.092507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:15:15.092507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:15.092507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:15:15.092507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:15:15.092507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:15.092507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:15:15.092507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:15.092507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:15:15.092507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:15:15.092507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:15.092507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:15:15.093035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:15.093035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:15:15.093643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:15.093643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:15:15.093643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:15:15.093643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:15:15.093643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:15:15.093643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:15:15.093643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:15.093643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:15:15.093643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:15:15.093643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:15:15.093643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:15:15.093643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:15:15.093643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:15.093643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:15.093643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:15:15.093643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:15:15.094324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:15:15.094324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:15:15.094324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:15.094324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:15.094324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:15.094324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:15:15.094324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:15.094324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:15.094324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:15.094324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:15:15.094324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:15.094324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:15:15.094324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:15:15.094324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:15.094324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:15:15.094324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:15.094324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:15:15.094324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:15:15.094324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:15.095183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:15:15.095183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:15.095183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:15:19.823522 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:15:19.824100 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:15:19.824186 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:15:19.824819 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:15:19.853546 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:15:19.853833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:19.854549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:15:19.854549 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:19.854549 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:15:19.854549 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:15:19.854549 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:19.854549 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:15:19.854549 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:19.854549 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:15:19.855325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.030 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:19.855325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:15:19.855325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:19.855325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:15:19.855325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:15:19.855325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:15:19.855325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:15:19.855325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:19.855325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:19.855325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:19.855325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:15:19.855325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:19.855725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:19.855725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:19.855725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:15:19.855725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:19.855725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:15:19.855725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:15:19.855725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:19.855725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:15:19.855725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:19.855725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:15:19.855725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:15:19.855725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:19.855725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:15:19.856059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:19.856059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:15:19.856122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:19.856122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:15:19.856122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:15:19.856122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:15:19.856122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:15:19.856122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:15:19.856122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:19.856122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:15:19.856122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:15:19.856122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:15:19.856122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:15:19.856122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:15:19.856122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:19.856122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:19.856122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:15:19.856122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:15:19.856708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:15:19.856708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:15:19.856708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:19.856708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:19.856708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:19.856708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:15:19.856708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:19.856708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:19.856708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:19.856708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:15:19.856708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:19.856708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:15:19.856708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:15:19.856708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:19.856708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:15:19.856708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:19.856708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:15:19.856708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:15:19.856708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:19.857350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:15:19.857350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:19.857350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:15:20.060252 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:15:20.060891 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:15:20.061171 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:15:20.062151 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:15:20.063617 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:15:20.063617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:20.063617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:15:20.063617 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:20.063617 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:15:20.063617 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:15:20.063617 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:20.063617 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:15:20.063617 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:20.063617 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:15:20.065064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:20.065064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:15:20.065064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:20.065064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:15:20.065064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:15:20.065064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:15:20.065064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:15:20.065064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:20.065064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:20.065064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:20.065064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:15:20.065064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:20.065348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:20.065348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:20.065348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:15:20.065348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:20.065348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:15:20.065348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:15:20.065348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:20.065348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:15:20.065348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:20.065348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:15:20.065348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:15:20.065348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:20.065348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:15:20.065628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:20.065628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:15:20.066921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:20.066921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:15:20.066921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:15:20.066921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:15:20.066921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:15:20.066921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:15:20.066921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:20.066921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:15:20.066921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:15:20.066921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:15:20.066921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:15:20.066921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:15:20.066921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:20.066921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:20.066921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:15:20.066921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:15:20.067266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:15:20.067266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:15:20.067266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:20.067266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:20.067266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:20.067266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:15:20.067266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:20.067266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:20.067266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:20.067266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:15:20.067266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:20.067266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:15:20.067266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:15:20.067266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:20.067266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:15:20.067266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:20.067266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:15:20.067266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:15:20.067266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:20.067631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:15:20.067631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:20.067631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:15:20.067893 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:15:20.068265 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:15:20.068507 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:15:20.068724 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:15:20.069181 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:15:20.069507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:20.070785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:15:20.070785 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:20.070785 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:15:20.070785 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:15:20.070785 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:20.070785 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:15:20.070785 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:20.070785 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:15:20.071616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:20.071616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:15:20.071616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:15:20.071616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:15:20.071616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:20.071616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:15:20.071616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:20.071616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:15:20.071616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:20.071616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:15:20.071616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:20.071616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:15:20.071616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:20.071616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:15:20.071616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:15:20.071908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:20.071908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:15:20.071908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:20.071908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:15:20.071908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:15:20.071908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:20.071908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:15:20.071908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:20.071908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:15:20.072610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:20.072610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:15:20.072610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:20.072610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:15:20.072610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:15:20.072610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:15:20.072610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:15:20.072610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:15:20.072610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:15:20.072610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:15:20.072610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:20.072610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:20.072870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:20.072870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:15:20.072870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:20.072870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:20.072870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:20.072870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:15:20.072870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:20.072870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:15:20.072870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:15:20.072870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:20.072870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:15:20.072870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:20.072870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:15:20.072870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:15:20.073121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:20.073121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:15:20.073121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:20.073121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:15:20.073918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:20.073918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:15:20.073918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:15:20.073918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:15:20.073918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:15:20.073918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:15:20.073918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:20.073918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:15:20.073918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:15:20.073918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:15:20.073918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:15:20.073918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:15:20.073918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:20.073918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:20.073918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:15:20.073918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:15:20.074378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:15:20.074378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:15:20.074378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:15:20.074378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:15:20.074378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:15:20.074378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:20.074378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:20.074378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:20.074378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:15:20.074378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:20.074378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:20.074378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:20.074378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:15:20.074378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:20.074378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:15:20.074378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:15:20.074378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:20.074378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:15:20.074859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:20.074859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:15:20.074859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:15:20.074859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:20.074859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:15:20.074859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:20.074859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:15:20.075460 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:15:20.075801 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:15:20.075913 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:15:20.076055 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:15:20.076580 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:15:20.076900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:20.077566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:15:20.077566 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:20.077566 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:15:20.077566 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:15:20.077566 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:20.077566 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:15:20.077566 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:20.077566 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:15:20.078507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:20.078507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:15:20.078507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:15:20.078507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:15:20.078507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:20.078507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:15:20.078507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:20.078507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:15:20.078507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:20.078507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:15:20.078507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:20.078507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:15:20.078507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:20.078507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:15:20.078507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:15:20.078969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:20.078969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:15:20.078969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:20.078969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:15:20.078969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:15:20.078969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:20.078969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:15:20.078969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:20.078969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:15:20.079216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:20.079216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:15:20.079216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:20.079216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:15:20.079216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:15:20.079216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:15:20.079216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:15:20.079216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:15:20.079216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:15:20.079216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:20.079216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:20.079579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:20.079579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:15:20.079579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:20.079579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:20.079579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:20.079579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:15:20.079579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:20.079579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:15:20.079579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:15:20.079579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:20.079579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:15:20.079579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:20.079579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:15:20.079579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:15:20.079940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:20.079940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:15:20.079940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:20.079940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:15:20.080195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:20.080195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:15:20.080195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:15:20.080195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:15:20.080195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:15:20.080195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:15:20.080195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:20.080195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:15:20.080195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:15:20.080195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:15:20.080195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:15:20.080195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:15:20.080195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:20.080195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:20.080195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:15:20.080195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:15:20.080195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:15:20.080726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:15:20.080726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:15:20.080726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:15:20.080726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:20.080726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:20.080726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:20.080726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:15:20.080726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:20.080726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:20.080726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:20.080726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:15:20.080726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:20.080726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:15:20.080726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:15:20.080726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:20.080726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:15:20.080726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:20.080726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:15:20.080726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:15:20.081238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:20.081238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:15:20.081238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:20.081238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:15:20.088627 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:15:20.088929 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:15:20.089111 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:15:20.089354 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:15:20.089814 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:15:20.090148 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:15:20.090365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:20.091249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:15:20.091249 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:20.091249 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:15:20.091249 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:15:20.091249 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:20.091249 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:15:20.091249 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:20.091249 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:15:20.092827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:20.092827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:15:20.092827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:20.092827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:15:20.092827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:15:20.092827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:15:20.092827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:15:20.092827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:20.092827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:20.092827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:20.092827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:15:20.092827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:20.093388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:20.093388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:20.093388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:15:20.093388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:20.093388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:15:20.093388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:15:20.093388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:20.093388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:15:20.093388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:20.093388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:15:20.093388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:15:20.093388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:20.093388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:15:20.094015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:20.094015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:15:20.094671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:20.094671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:15:20.094671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:15:20.094671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:15:20.094671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:15:20.094671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:15:20.094671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:20.094671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:15:20.094671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:15:20.094671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:15:20.094671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:15:20.094671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:15:20.094671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:20.094671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:20.094671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:15:20.094671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:15:20.095381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:15:20.095381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:15:20.095381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:20.095381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:20.095381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:20.095381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:15:20.095381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:20.095381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:20.095381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:20.095381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:15:20.095381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:20.095381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:15:20.095381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:15:20.095381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:20.095381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:15:20.095381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:20.095381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:15:20.095381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:15:20.095381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:20.096212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:15:20.096212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:20.096212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:15:24.824182 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:15:24.824693 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:15:24.824773 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:15:24.825639 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:15:24.832993 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG dbcounter [-] [105581] Writing DB stats neutron:SELECT=435 {{(pid=105581) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 14:15:24.861131 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:15:24.861348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:24.862005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:15:24.862005 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:24.862005 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:15:24.862005 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:15:24.862005 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:24.862005 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:15:24.862005 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:24.862005 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:15:24.862792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:24.862792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:15:24.862792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:24.862792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:15:24.862792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:15:24.862792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:15:24.862792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:15:24.862792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:24.862792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:24.862792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:24.862792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:15:24.862792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:24.863106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:24.863106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:24.863106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:15:24.863106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:24.863106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:15:24.863106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:15:24.863106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:24.863106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:15:24.863106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:24.863106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:15:24.863106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:15:24.863106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:24.863106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:15:24.863838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:24.863838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:15:24.863838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:24.863838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:15:24.863838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:15:24.863838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:15:24.863838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:15:24.863838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:15:24.863838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:24.863838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:15:24.863838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:15:24.863838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:15:24.863838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:15:24.863838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:15:24.863838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:24.864157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:24.864157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:15:24.864157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:15:24.864157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:15:24.864157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:15:24.864157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:24.864157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:24.864157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:24.864157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:15:24.864157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:24.864157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:24.864157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:24.864157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:15:24.864157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:24.864157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:15:24.864157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:15:24.864157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:24.864157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:15:24.864157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:24.864549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:15:24.864549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:15:24.864549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:24.864549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:15:24.864549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:24.864549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:15:25.062298 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:15:25.062298 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:15:25.062298 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:15:25.063315 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:15:25.063673 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:15:25.063911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:25.064791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:15:25.064791 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:25.064791 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:15:25.064791 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:15:25.064791 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:25.064791 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:15:25.064791 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:25.064791 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:15:25.066317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:25.066317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:15:25.066317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:25.066317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:15:25.066317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:15:25.066317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:15:25.066317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:15:25.066317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:25.066317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:25.066317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:25.066317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:15:25.066317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:25.066597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:25.066597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:25.066597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:15:25.066597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:25.066597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:15:25.066597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:15:25.066597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:25.066597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:15:25.066597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:25.066597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:15:25.066597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:15:25.066597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:25.066597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:15:25.066832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:25.066832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:15:25.067451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:25.067451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:15:25.067451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:15:25.067451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:15:25.067451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:15:25.067451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:15:25.067451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:25.067451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:15:25.067451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:15:25.067451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:15:25.067451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:15:25.067451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:15:25.067451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:25.067451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:25.067451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:15:25.067451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:15:25.067799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:15:25.067799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:15:25.067799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:25.067799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:25.067799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:25.067799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:15:25.067799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:25.067799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:25.067799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:25.067799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:15:25.067799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:25.067799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:15:25.067799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:15:25.067799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:25.067799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:15:25.067799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:25.067799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:15:25.067799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:15:25.067799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:25.068133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:15:25.068133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:25.068133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:15:25.068320 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:15:25.068600 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:15:25.068757 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:15:25.068897 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:15:25.069360 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:15:25.069675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:25.070308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:15:25.070308 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:25.070308 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:15:25.070308 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:15:25.070308 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:25.070308 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:15:25.070308 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:25.070308 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:15:25.071181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:25.071181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:15:25.071181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:15:25.071181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:15:25.071181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:25.071181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:15:25.071181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:25.071181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:15:25.071181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:25.071181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:15:25.071181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:25.071181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:15:25.071181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:25.071181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:15:25.071181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:15:25.071961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:25.071961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:15:25.071961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:25.071961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:15:25.071961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:15:25.071961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:25.071961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:15:25.071961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:25.071961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:15:25.072205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:25.072205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:15:25.072205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:25.072205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:15:25.072205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:15:25.072205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:15:25.072205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:15:25.072205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:15:25.072205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:15:25.072205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:15:25.072205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:25.072205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:25.072553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:25.072553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:15:25.072553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:25.072553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:25.072553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:25.072553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:15:25.072553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:25.072553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:15:25.072553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:15:25.072553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:25.072553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:15:25.072553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:25.072553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:15:25.072553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:15:25.072902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:25.072902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:15:25.072902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:25.072902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:15:25.073363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:25.073363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:15:25.073363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:15:25.073363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:15:25.073363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:15:25.073363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:15:25.073363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:25.073363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:15:25.073363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:15:25.073363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:15:25.073363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:15:25.073363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:15:25.073363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:25.073363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:25.073363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:15:25.073363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:15:25.074709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:15:25.074709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:15:25.074709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:15:25.074709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:15:25.074709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:15:25.074709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:25.074709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:25.074709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:25.074709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:15:25.074709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:25.074709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:25.074709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:25.074709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:15:25.074709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:25.074709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:15:25.074709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:15:25.074709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:25.074709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:15:25.075945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:25.075945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:15:25.075945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:15:25.075945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:25.075945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:15:25.075945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:25.075945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:15:25.075945 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:15:25.076251 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:15:25.076395 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:15:25.076528 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:15:25.076899 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:15:25.077147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:25.077770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:15:25.077770 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:25.077770 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:15:25.077770 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:15:25.077770 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:25.077770 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:15:25.077770 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:25.077770 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:15:25.078607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:25.078607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:15:25.078607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:15:25.078607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:15:25.078607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:25.078607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:15:25.078607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:25.078607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:15:25.078607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:25.078607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:15:25.078607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:25.078607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:15:25.078607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:25.078607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:15:25.078607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:15:25.079008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:25.079008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:15:25.079008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:25.079008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:15:25.079008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:15:25.079008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:25.079008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:15:25.079008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:25.079008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:15:25.080781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:25.080781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:15:25.080781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:25.080781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:15:25.080781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:15:25.080781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:15:25.080781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:15:25.080781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:15:25.080781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:15:25.080781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:25.080781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:25.081109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:25.081109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:15:25.081109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:25.081109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:25.081109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:25.081109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:15:25.081109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:25.081109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:15:25.081109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:15:25.081109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:25.081109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:15:25.081109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:25.081109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:15:25.081109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:15:25.081649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:25.081649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:15:25.081649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:25.081649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:15:25.082132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:25.082132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:15:25.082132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:15:25.082132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:15:25.082132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:15:25.082132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:15:25.082132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:25.082132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:15:25.082132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:15:25.082132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:15:25.082132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:15:25.082132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:15:25.082132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:25.082132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:25.082132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:15:25.082132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:15:25.082132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:15:25.083303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:15:25.083303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:15:25.083303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:15:25.083303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:25.083303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:25.083303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:25.083303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:15:25.083303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:25.083303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:25.083303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:25.083303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:15:25.083303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:25.083303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:15:25.083303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:15:25.083303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:25.083303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:15:25.083303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:25.083303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:15:25.083303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:15:25.084048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:25.084048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:15:25.084048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:25.084048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:15:25.089055 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:15:25.089378 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:15:25.089502 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:15:25.089711 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:15:25.090183 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:15:25.090574 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:15:25.090828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:25.091540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:15:25.091540 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:25.091540 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:15:25.091540 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:15:25.091540 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:25.091540 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:15:25.091540 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:25.091540 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:15:25.092721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:25.092721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:15:25.092721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:25.092721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:15:25.092721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:15:25.092721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:15:25.092721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:15:25.092721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:25.092721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:25.092721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:25.092721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:15:25.092721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:25.093053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:25.093053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:25.093053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:15:25.093053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:25.093053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:15:25.093053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:15:25.093053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:25.093053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:15:25.093053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:25.093053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:15:25.093053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:15:25.093053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:25.093053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:15:25.093364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:25.093364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:15:25.094523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:25.094523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:15:25.094523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:15:25.094523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:15:25.094523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:15:25.094523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:15:25.094523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:25.094523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:15:25.094523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:15:25.094523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:15:25.094523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:15:25.094523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:15:25.094523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:25.094523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:25.094523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:15:25.094523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:15:25.095223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:15:25.095223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:15:25.095223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:25.095223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:25.095223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:25.095223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:15:25.095223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:25.095223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:25.095223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:25.095223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:15:25.095223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:25.095223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:15:25.095223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:15:25.095223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:25.095223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:15:25.095223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:25.095223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:15:25.095223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:15:25.095223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:25.096014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:15:25.096014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:25.096014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:15:29.824863 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:15:29.825705 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:15:29.825705 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:15:29.825829 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:15:29.854864 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:15:29.855723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:29.856428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:15:29.856428 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:29.856428 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:15:29.856428 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:15:29.856428 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:29.856428 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:15:29.856428 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:29.856428 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:15:29.857164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.031 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:29.857164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:15:29.857164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:29.857164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:15:29.857164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:15:29.857164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:15:29.857164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:15:29.857164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:29.857164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:29.857164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:29.857164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:15:29.857164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:29.857797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:29.857797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:29.857797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:15:29.857797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:29.857797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:15:29.857797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:15:29.857797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:29.857797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:15:29.857797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:29.857797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:15:29.857797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:15:29.857797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:29.857797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:15:29.858994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:29.858994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:15:29.858994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:29.858994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:15:29.858994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:15:29.858994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:15:29.858994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:15:29.858994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:15:29.858994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:29.858994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:15:29.858994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:15:29.858994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:15:29.858994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:15:29.858994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:15:29.858994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:29.859670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:29.859670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:15:29.859670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:15:29.859670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:15:29.859670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:15:29.859670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:29.859670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:29.859670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:29.859670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:15:29.859670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:29.859670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:29.859670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:29.859670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:15:29.859670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:29.859670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:15:29.859670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:15:29.859670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:29.859670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:15:29.859670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:29.860429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:15:29.860429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:15:29.860429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:29.860429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:15:29.860429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:29.860429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:15:30.061782 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:15:30.062247 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:15:30.062470 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:15:30.063583 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:15:30.063926 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:15:30.064222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:30.065088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:15:30.065088 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:30.065088 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:15:30.065088 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:15:30.065088 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:30.065088 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:15:30.065088 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:30.065088 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:15:30.066531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:30.066531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:15:30.066531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:30.066531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:15:30.066531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:15:30.066531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:15:30.066531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:15:30.066531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:30.066531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:30.066531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:30.066531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:15:30.066531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:30.067218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:30.067218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:30.067218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:15:30.067218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:30.067218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:15:30.067218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:15:30.067218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:30.067218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:15:30.067218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:30.067218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:15:30.067218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:15:30.067218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:30.067218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:15:30.067790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:30.067790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:15:30.067790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:30.067790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:15:30.067790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:15:30.067790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:15:30.067790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:15:30.067790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:15:30.067790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:30.067790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:15:30.067790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:15:30.067790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:15:30.067790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:15:30.067790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:15:30.067790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:30.068382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:30.068382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:15:30.068382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:15:30.068382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:15:30.068382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:15:30.068382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:30.068382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:30.068382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:30.068382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:15:30.068382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:30.068382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:30.068382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:30.068382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:15:30.068382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:30.068382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:15:30.068382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:15:30.068382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:30.068382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:15:30.068382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:30.069156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:15:30.069156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:15:30.069156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:30.069156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:15:30.069156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:30.069156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:15:30.069156 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:15:30.069156 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:15:30.069156 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:15:30.069156 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:15:30.069991 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:15:30.069991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:30.070544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:15:30.070544 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:30.070544 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:15:30.070544 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:15:30.070544 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:30.070544 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:15:30.070544 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:30.070544 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:15:30.071332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:30.071332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:15:30.071332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:15:30.071332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:15:30.071332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:30.071332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:15:30.071332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:30.071332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:15:30.071332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:30.071332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:15:30.071332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:30.071332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:15:30.071332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:30.071332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:15:30.071332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:15:30.072725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:30.072725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:15:30.072725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:30.072725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:15:30.072725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:15:30.072725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:30.072725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:15:30.072725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:30.072725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:15:30.072725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:30.072725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:15:30.072725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:30.072725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:15:30.072725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:15:30.074137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:15:30.074137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:15:30.074137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:15:30.074137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:15:30.074137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:15:30.074137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:30.074137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:30.074137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:30.074137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:15:30.074137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:30.074137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:30.074137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:30.074137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:15:30.075933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:30.075933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:15:30.075933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:15:30.075933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:30.075933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:15:30.075933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:30.075933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:15:30.075933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:15:30.075933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:30.075933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:15:30.075933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:30.075933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:15:30.077525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:30.077525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:15:30.077525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:15:30.077525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:15:30.077525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:15:30.077525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:15:30.077525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:30.077525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:15:30.077525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:15:30.077525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:15:30.077525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:15:30.077525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:15:30.077525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:30.077525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:30.077525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:15:30.077525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:15:30.078628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:15:30.078628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:15:30.078628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:15:30.078628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:15:30.078628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:15:30.078628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:30.078628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:30.078628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:30.078628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:15:30.078628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:30.078628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:30.078628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:30.078628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:15:30.078628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:30.078628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:15:30.078628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:15:30.078628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:30.078628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:15:30.078982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:30.078982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:15:30.078982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:15:30.078982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:30.078982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:15:30.078982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:30.078982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:15:30.078982 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:15:30.078982 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:15:30.078982 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:15:30.079174 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:15:30.079174 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:15:30.079174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:30.079174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:15:30.079174 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:30.079174 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:15:30.079174 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:15:30.079174 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:30.079174 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:15:30.079174 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:30.079174 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:15:30.079174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:30.079394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:15:30.079394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:15:30.079394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:15:30.079394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:30.079394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:15:30.079394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:30.079394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:15:30.079394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:30.079394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:15:30.079394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:30.079394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:15:30.079394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:30.079394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:15:30.079394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:15:30.079394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:30.079394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:15:30.079394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:30.079695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:15:30.079695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:15:30.079695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:30.079695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:15:30.079695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:30.079695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:15:30.079695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:30.079695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:15:30.079695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:30.079695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:15:30.079695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:15:30.079695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:15:30.079695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:15:30.079945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:15:30.079945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:15:30.079945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:30.079945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:30.079945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:30.079945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:15:30.079945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:30.079945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:30.079945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:30.079945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:15:30.079945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:30.079945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:15:30.079945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:15:30.080171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:30.080171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:15:30.080171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:30.080171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:15:30.080171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:15:30.080171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:30.080171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:15:30.080171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:30.080171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:15:30.080931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:30.080931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:15:30.080931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:15:30.080931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:15:30.080931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:15:30.080931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:15:30.080931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:30.080931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:15:30.080931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:15:30.080931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:15:30.080931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:15:30.080931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:15:30.080931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:30.080931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:30.080931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:15:30.080931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:15:30.080931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:15:30.081590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:15:30.081590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:15:30.081590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:15:30.081590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:30.081590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:30.081590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:30.081590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:15:30.081590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:30.081590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:30.081590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:30.081590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:15:30.081590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:30.081590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:15:30.081590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:15:30.081590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:30.081590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:15:30.081590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:30.081590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:15:30.081590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:15:30.082530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:30.082530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:15:30.082530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:30.082530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:15:30.089453 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:15:30.089713 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:15:30.089921 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:15:30.090308 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:15:30.090789 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:15:30.091130 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:15:30.091324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:30.092209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:15:30.092209 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:30.092209 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:15:30.092209 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:15:30.092209 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:30.092209 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:15:30.092209 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:30.092209 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:15:30.094163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:30.094163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:15:30.094163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:30.094163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:15:30.094163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:15:30.094163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:15:30.094163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:15:30.094163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:30.094163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:30.094163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:30.094163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:15:30.094163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:30.094769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:30.094769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:30.094769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:15:30.094769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:30.094769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:15:30.094769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:15:30.094769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:30.094769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:15:30.094769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:30.094769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:15:30.094769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:15:30.094769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:30.094769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:15:30.095087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:30.095087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:15:30.095448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:30.095448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:15:30.095448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:15:30.095448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:15:30.095448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:15:30.095448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:15:30.095448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:30.095448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:15:30.095448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:15:30.095448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:15:30.095448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:15:30.095448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:15:30.095448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:30.095448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:30.095448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:15:30.095448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:15:30.096021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:15:30.096021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:15:30.096021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:30.096021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:30.096021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:30.096021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:15:30.096021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:30.096021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:30.096021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:30.096021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:15:30.096021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:30.096021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:15:30.096021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:15:30.096021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:30.096021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:15:30.096021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:30.096021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:15:30.096021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:15:30.096021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:30.096611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:15:30.096611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:30.096611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:15:34.825469 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:15:34.826262 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:15:34.826262 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:15:34.826823 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:15:34.864611 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:15:34.865105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:34.866065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:15:34.866065 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:34.866065 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:15:34.866065 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:15:34.866065 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:34.866065 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:15:34.866065 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:34.866065 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:15:34.866901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:34.866901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:15:34.866901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:34.866901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:15:34.866901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:15:34.866901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:15:34.866901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:15:34.866901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:34.866901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:34.866901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:34.866901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:15:34.866901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:34.867505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:34.867505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:34.867505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:15:34.867505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:34.867505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:15:34.867505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:15:34.867505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:34.867505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:15:34.867505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:34.867505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:15:34.867505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:15:34.867505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:34.867505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:15:34.868069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:34.868069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:15:34.868069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:34.868069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:15:34.868069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:15:34.868069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:15:34.868069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:15:34.868069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:15:34.868069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:34.868069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:15:34.868069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:15:34.868069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:15:34.868069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:15:34.868069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:15:34.868069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:34.868731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:34.868731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:15:34.868731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:15:34.868731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:15:34.868731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:15:34.868731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:34.868731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:34.868731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:34.868731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:15:34.868731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:34.868731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:34.868731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:34.868731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:15:34.868731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:34.868731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:15:34.868731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:15:34.868731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:34.868731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:15:34.868731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:34.869564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:15:34.869564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:15:34.869564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:34.869564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:15:34.869564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:34.869564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:15:35.062350 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:15:35.062923 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:15:35.063118 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:15:35.064263 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:15:35.064529 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:15:35.064747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:35.065482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:15:35.065482 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:35.065482 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:15:35.065482 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:15:35.065482 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:35.065482 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:15:35.065482 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:35.065482 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:15:35.067198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:35.067198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:15:35.067198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:35.067198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:15:35.067198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:15:35.067198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:15:35.067198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:15:35.067198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:35.067198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:35.067198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:35.067198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:15:35.067198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:35.067713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:35.067713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:35.067713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:15:35.067713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:35.067713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:15:35.067713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:15:35.067713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:35.067713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:15:35.067713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:35.067713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:15:35.067713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:15:35.067713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:35.067713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:15:35.068152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:35.068152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:15:35.068921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:35.068921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:15:35.068921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:15:35.068921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:15:35.068921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:15:35.068921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:15:35.068921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:35.068921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:15:35.068921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:15:35.068921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:15:35.068921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:15:35.068921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:15:35.068921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:35.068921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:35.068921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:15:35.068921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:15:35.069592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:15:35.069592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:15:35.069592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:35.069592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:35.069592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:35.069592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:15:35.069592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:35.069592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:35.069592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:35.069592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:15:35.069592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:35.069592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:15:35.069592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:15:35.069592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:35.069592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:15:35.069592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:35.069592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:15:35.069592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:15:35.069592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:35.070239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:15:35.070239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:35.070239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:15:35.070239 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:15:35.070239 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:15:35.070239 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:15:35.070239 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:15:35.070995 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:15:35.071262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:35.071931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:15:35.071931 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:35.071931 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:15:35.071931 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:15:35.071931 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:35.071931 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:15:35.071931 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:35.071931 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:15:35.073180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:35.073180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:15:35.073180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:15:35.073180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:15:35.073180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:35.073180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:15:35.073180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:35.073180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:15:35.073180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:35.073180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:15:35.073180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:35.073180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:15:35.073180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:35.073180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:15:35.073180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:15:35.073950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:35.073950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:15:35.073950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:35.073950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:15:35.073950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:15:35.073950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:35.073950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:15:35.073950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:35.073950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:15:35.074534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:35.074534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:15:35.074534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:35.074534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:15:35.074534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:15:35.074534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:15:35.074534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:15:35.074534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:15:35.074534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:15:35.074534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:15:35.074534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:35.074534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:35.075147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:35.075147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:15:35.075147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:35.075147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:35.075147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:35.075147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:15:35.075147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:35.075147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:15:35.075147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:15:35.075147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:35.075147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:15:35.075147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:35.075147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:15:35.075147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:15:35.076162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:35.076162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:15:35.076162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:35.076162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:15:35.076162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:35.076162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:15:35.076162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:15:35.076162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:15:35.076162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:15:35.076162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:15:35.076162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:35.076162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:15:35.076162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:15:35.076162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:15:35.076803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:15:35.076803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:15:35.076803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:35.076803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:35.076803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:15:35.076803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:15:35.076803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:15:35.076803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:15:35.076803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:15:35.076803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:15:35.076803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:15:35.076803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:35.076803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:35.076803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:35.076803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:15:35.076803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:35.076803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:35.076803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:35.076803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:15:35.077710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:35.077710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:15:35.077710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:15:35.077710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:35.077710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:15:35.077710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:35.077710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:15:35.077710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:15:35.077710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:35.077710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:15:35.077710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:35.077710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:15:35.077710 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:15:35.077710 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:15:35.078249 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:15:35.078249 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:15:35.078249 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:15:35.078249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:35.078249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:15:35.078249 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:35.078249 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:15:35.078249 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:15:35.078249 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:35.078249 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:15:35.078249 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:35.078249 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:15:35.078628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:35.078628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:15:35.078628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:15:35.078628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:15:35.078628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:35.078628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:15:35.078628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:35.078628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:15:35.078628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:35.078628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:15:35.078628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:35.078628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:15:35.078628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:35.078628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:15:35.078628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:15:35.079032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:35.079032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:15:35.079032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:35.079032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:15:35.079032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:15:35.079032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:35.079032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:15:35.079032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:35.079032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:15:35.079486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:35.079486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:15:35.079486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:35.079486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:15:35.079486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:15:35.079486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:15:35.079486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:15:35.079486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:15:35.079486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:15:35.079486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:35.079486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:35.079839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:35.079839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:15:35.079839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:35.079839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:35.079839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:35.079839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:15:35.079839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:35.079839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:15:35.079839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:15:35.079839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:35.079839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:15:35.079839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:35.079839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:15:35.079839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:15:35.080232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:35.080232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:15:35.080232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:35.080232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:15:35.080511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:35.080511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:15:35.080511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:15:35.080511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:15:35.080511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:15:35.080511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:15:35.080511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:35.080511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:15:35.080511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:15:35.080511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:15:35.080511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:15:35.080511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:15:35.080511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:35.080511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:35.080511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:15:35.080511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:15:35.080511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:15:35.080963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:15:35.080963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:15:35.080963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:15:35.080963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:35.080963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:35.080963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:35.080963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:15:35.080963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:35.080963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:35.080963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:35.080963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:15:35.080963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:35.080963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:15:35.080963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:15:35.080963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:35.080963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:15:35.080963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:35.080963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:15:35.080963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:15:35.081463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:35.081463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:15:35.081463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:35.081463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:15:35.089760 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:15:35.090018 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:15:35.090094 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:15:35.090480 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:15:35.090693 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:15:35.090908 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:15:35.091055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:35.091684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:15:35.091684 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:35.091684 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:15:35.091684 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:15:35.091684 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:35.091684 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:15:35.091684 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:35.091684 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:15:35.092973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:35.092973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:15:35.092973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:35.092973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:15:35.092973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:15:35.092973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:15:35.092973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:15:35.092973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:35.092973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:35.092973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:35.092973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:15:35.092973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:35.093292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:35.093292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:35.093292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:15:35.093292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:35.093292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:15:35.093292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:15:35.093292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:35.093292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:15:35.093292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:35.093292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:15:35.093292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:15:35.093292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:35.093292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:15:35.093586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:35.093586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:15:35.094936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:35.094936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:15:35.094936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:15:35.094936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:15:35.094936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:15:35.094936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:15:35.094936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:35.094936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:15:35.094936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:15:35.094936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:15:35.094936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:15:35.094936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:15:35.094936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:35.094936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:35.094936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:15:35.094936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:15:35.095481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:15:35.095481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:15:35.095481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:35.095481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:35.095481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:35.095481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:15:35.095481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:35.095481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:35.095481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:35.095481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:15:35.095481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:35.095481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:15:35.095481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:15:35.095481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:35.095481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:15:35.095481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:35.095481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:15:35.095481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:15:35.095481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:35.096420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:15:35.096420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:35.096420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:15:39.825999 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:15:39.826818 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:15:39.826938 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:15:39.827778 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:15:39.855771 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:15:39.856133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:39.856946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:15:39.856946 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:39.856946 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:15:39.856946 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:15:39.856946 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:39.856946 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:15:39.856946 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:39.856946 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:15:39.857698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.030 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:39.857698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:15:39.857698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:39.857698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:15:39.857698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:15:39.857698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:15:39.857698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:15:39.857698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:39.857698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:39.857698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:39.857698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:15:39.857698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:39.858318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:39.858318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:39.858318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:15:39.858318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:39.858318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:15:39.858318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:15:39.858318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:39.858318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:15:39.858318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:39.858318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:15:39.858318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:15:39.858318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:39.858318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:15:39.858692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:39.858692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:15:39.858692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:39.858692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:15:39.858692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:15:39.858692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:15:39.858692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:15:39.858692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:15:39.858692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:39.858692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:15:39.858692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:15:39.858692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:15:39.858692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:15:39.858692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:15:39.858692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:39.859091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:39.859091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:15:39.859091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:15:39.859091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:15:39.859091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:15:39.859091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:39.859091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:39.859091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:39.859091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:15:39.859091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:39.859091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:39.859091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:39.859091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:15:39.859091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:39.859091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:15:39.859091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:15:39.859091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:39.859091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:15:39.859091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:39.859636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:15:39.859636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:15:39.859636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:39.859636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:15:39.859636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:39.859636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:15:40.063083 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:15:40.063548 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:15:40.063724 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:15:40.064696 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:15:40.064926 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:15:40.065091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:40.065777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:15:40.065777 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:40.065777 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:15:40.065777 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:15:40.065777 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:40.065777 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:15:40.065777 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:40.065777 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:15:40.067184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:40.067184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:15:40.067184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:40.067184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:15:40.067184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:15:40.067184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:15:40.067184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:15:40.067184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:40.067184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:40.067184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:40.067184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:15:40.067184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:40.067618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:40.067618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:40.067618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:15:40.067618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:40.067618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:15:40.067618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:15:40.067618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:40.067618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:15:40.067618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:40.067618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:15:40.067618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:15:40.067618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:40.067618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:15:40.067974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:40.067974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:15:40.068366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:40.068366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:15:40.068366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:15:40.068366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:15:40.068366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:15:40.068366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:15:40.068366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:40.068366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:15:40.068366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:15:40.068366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:15:40.068366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:15:40.068366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:15:40.068366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:40.068366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:40.068366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:15:40.068366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:15:40.068894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:15:40.068894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:15:40.068894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:40.068894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:40.068894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:40.068894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:15:40.068894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:40.068894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:40.068894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:40.068894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:15:40.068894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:40.068894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:15:40.068894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:15:40.068894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:40.068894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:15:40.068894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:40.068894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:15:40.068894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:15:40.068894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:40.069454 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:15:40.069454 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:40.069454 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:15:40.069554 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:15:40.069936 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:15:40.070096 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:15:40.070264 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:15:40.070616 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:15:40.070778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:40.071362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:15:40.071362 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:40.071362 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:15:40.071362 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:15:40.071362 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:40.071362 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:15:40.071362 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:40.071362 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:15:40.072338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:40.072338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:15:40.072338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:15:40.072338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:15:40.072338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:40.072338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:15:40.072338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:40.072338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:15:40.072338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:40.072338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:15:40.072338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:40.072338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:15:40.072338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:40.072338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:15:40.072338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:15:40.072959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:40.072959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:15:40.072959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:40.072959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:15:40.072959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:15:40.072959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:40.072959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:15:40.072959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:40.072959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:15:40.073531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:40.073531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:15:40.073531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:40.073531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:15:40.073531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:15:40.073531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:15:40.073531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:15:40.073531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:15:40.073531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:15:40.073531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:15:40.073531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:40.073531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:40.074000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:40.074000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:15:40.074000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:40.074000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:40.074000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:40.074000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:15:40.074000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:40.074000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:15:40.074000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:15:40.074000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:40.074000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:15:40.074000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:40.074000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:15:40.074000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:15:40.074541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:40.074541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:15:40.074541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:40.074541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:15:40.074705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:40.074705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:15:40.074705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:15:40.074705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:15:40.074705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:15:40.074705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:15:40.074705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:40.074705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:15:40.074705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:15:40.074705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:15:40.074705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:15:40.074705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:15:40.074705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:40.074705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:40.074705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:15:40.074705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:15:40.075198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:15:40.075198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:15:40.075198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:15:40.075198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:15:40.075198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:15:40.075198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:40.075198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:40.075198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:40.075198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:15:40.075198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:40.075198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:40.075198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:40.075198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:15:40.075198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:40.075198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:15:40.075198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:15:40.075198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:40.075198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:15:40.075678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:40.075678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:15:40.075678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:15:40.075678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:40.075678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:15:40.075678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:40.075678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:15:40.077973 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:15:40.077973 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:15:40.077973 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:15:40.077973 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:15:40.078091 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:15:40.078381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:40.078979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:15:40.078979 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:40.078979 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:15:40.078979 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:15:40.078979 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:40.078979 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:15:40.078979 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:40.078979 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:15:40.080108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:40.080108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:15:40.080108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:15:40.080108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:15:40.080108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:40.080108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:15:40.080108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:40.080108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:15:40.080108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:40.080108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:15:40.080108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:40.080108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:15:40.080108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:40.080108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:15:40.080108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:15:40.080461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:40.080461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:15:40.080461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:40.080461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:15:40.080461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:15:40.080461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:40.080461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:15:40.080461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:40.080461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:15:40.081299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:40.081299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:15:40.081299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:40.081299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:15:40.081299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:15:40.081299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:15:40.081299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:15:40.081299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:15:40.081299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:15:40.081299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:40.081299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:40.081573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:40.081573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:15:40.081573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:40.081573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:40.081573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:40.081573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:15:40.081573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:40.081573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:15:40.081573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:15:40.081573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:40.081573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:15:40.081573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:40.081573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:15:40.081573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:15:40.081835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:40.081835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:15:40.081835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:40.081835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:15:40.082745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:40.082745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:15:40.082745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:15:40.082745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:15:40.082745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:15:40.082745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:15:40.082745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:40.082745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:15:40.082745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:15:40.082745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:15:40.082745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:15:40.082745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:15:40.082745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:40.082745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:40.082745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:15:40.082745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:15:40.082745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:15:40.083093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:15:40.083093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:15:40.083093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:15:40.083093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:40.083093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:40.083093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:40.083093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:15:40.083093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:40.083093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:40.083093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:40.083093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:15:40.083093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:40.083093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:15:40.083093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:15:40.083093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:40.083093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:15:40.083093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:40.083093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:15:40.083093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:15:40.083488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:40.083488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:15:40.083488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:40.083488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:15:40.090156 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:15:40.090468 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:15:40.090726 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:15:40.090914 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:15:40.091424 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:15:40.092248 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:15:40.092248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:40.092433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:15:40.092433 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:40.092433 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:15:40.092433 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:15:40.092433 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:40.092433 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:15:40.092433 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:40.092433 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:15:40.093652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:40.093652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:15:40.093652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:40.093652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:15:40.093652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:15:40.093652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:15:40.093652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:15:40.093652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:40.093652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:40.093652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:40.093652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:15:40.093652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:40.094040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:40.094040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:40.094040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:15:40.094040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:40.094040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:15:40.094040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:15:40.094040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:40.094040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:15:40.094040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:40.094040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:15:40.094040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:15:40.094040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:40.094040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:15:40.094396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:40.094396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:15:40.095053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:40.095053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:15:40.095053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:15:40.095053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:15:40.095053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:15:40.095053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:15:40.095053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:40.095053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:15:40.095053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:15:40.095053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:15:40.095053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:15:40.095053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:15:40.095053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:40.095053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:40.095053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:15:40.095053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:15:40.095528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:15:40.095528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:15:40.095528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:40.095528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:40.095528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:40.095528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:15:40.095528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:40.095528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:40.095528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:40.095528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:15:40.095528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:40.095528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:15:40.095528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:15:40.095528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:40.095528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:15:40.095528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:40.095528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:15:40.095528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:15:40.095528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:40.096097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:15:40.096097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:40.096097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:15:44.828484 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:15:44.828484 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:15:44.828484 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:15:44.828484 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:15:44.864890 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:15:44.865120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:44.865673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:15:44.865673 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:44.865673 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:15:44.865673 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:15:44.865673 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:44.865673 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:15:44.865673 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:44.865673 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:15:44.866351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:44.866351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:15:44.866351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:44.866351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:15:44.866351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:15:44.866351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:15:44.866351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:15:44.866351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:44.866351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:44.866351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:44.866351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:15:44.866351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:44.867261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:44.867261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:44.867261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:15:44.867261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:44.867261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:15:44.867261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:15:44.867261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:44.867261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:15:44.867261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:44.867261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:15:44.867261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:15:44.867261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:44.867261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:15:44.867957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:44.867957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:15:44.867957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:44.867957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:15:44.867957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:15:44.867957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:15:44.867957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:15:44.867957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:15:44.867957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:44.867957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:15:44.867957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:15:44.867957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:15:44.867957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:15:44.867957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:15:44.867957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:44.869011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:44.869011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:15:44.869011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:15:44.869011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:15:44.869011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:15:44.869011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:44.869011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:44.869011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:44.869011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:15:44.869011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:44.869011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:44.869011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:44.869011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:15:44.869011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:44.869011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:15:44.869011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:15:44.869011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:44.869011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:15:44.869011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:44.870088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:15:44.870088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:15:44.870088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:44.870088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:15:44.870088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:44.870088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:15:45.063817 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:15:45.064251 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:15:45.064381 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:15:45.065256 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:15:45.065577 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:15:45.065864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:45.066738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:15:45.066738 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:45.066738 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:15:45.066738 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:15:45.066738 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:45.066738 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:15:45.066738 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:45.066738 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:15:45.068210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:45.068210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:15:45.068210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:45.068210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:15:45.068210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:15:45.068210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:15:45.068210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:15:45.068210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:45.068210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:45.068210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:45.068210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:15:45.068210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:45.068515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:45.068515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:45.068515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:15:45.068515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:45.068515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:15:45.068515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:15:45.068515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:45.068515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:15:45.068515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:45.068515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:15:45.068515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:15:45.068515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:45.068515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:15:45.068786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:45.068786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:15:45.070734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:45.070734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:15:45.070734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:15:45.070734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:15:45.070734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:15:45.070734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:15:45.070734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:45.070734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:15:45.070734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:15:45.070734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:15:45.070734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:15:45.070734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:15:45.070734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:45.070734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:45.070734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:15:45.070734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:15:45.071607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:15:45.071607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:15:45.071607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:45.071607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:45.071607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:45.071607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:15:45.071607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:45.071607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:45.071607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:45.071607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:15:45.071607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:45.071607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:15:45.071607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:15:45.071607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:45.071607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:15:45.071607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:45.071607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:15:45.071607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:15:45.071607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:45.072455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:15:45.072455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:45.072455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:15:45.072455 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:15:45.072455 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:15:45.072455 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:15:45.072455 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:15:45.072455 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:15:45.072845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:45.072845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:15:45.072845 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:45.072845 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:15:45.072845 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:15:45.072845 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:45.072845 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:15:45.072845 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:45.072845 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:15:45.073906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:45.073906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:15:45.073906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:15:45.073906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:15:45.073906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:45.073906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:15:45.073906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:45.073906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:15:45.073906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:45.073906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:15:45.073906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:45.073906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:15:45.073906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:45.073906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:15:45.073906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:15:45.074655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:45.074655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:15:45.074655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:45.074655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:15:45.074655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:15:45.074655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:45.074655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:15:45.074655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:45.074655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:15:45.075637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:45.075637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:15:45.075637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:45.075637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:15:45.075637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:15:45.075637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:15:45.075637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:15:45.075637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:15:45.075637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:15:45.075637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:15:45.075637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:45.075637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:45.076330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:45.076330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:15:45.076330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:45.076330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:45.076330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:45.076330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:15:45.076330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:45.076330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:15:45.076330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:15:45.076330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:45.076330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:15:45.076330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:45.076330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:15:45.076330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:15:45.076995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:45.076995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:15:45.076995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:45.076995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:15:45.077202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:45.077202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:15:45.077202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:15:45.077202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:15:45.077202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:15:45.077202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:15:45.077202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:45.077202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:15:45.077202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:15:45.077202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:15:45.077202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:15:45.077202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:15:45.077202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:45.077202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:45.077202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:15:45.077202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:15:45.078037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:15:45.078037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:15:45.078037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:15:45.078037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:15:45.078037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:15:45.078037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:45.078037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:45.078037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:45.078037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:15:45.078037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:45.078037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:45.078037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:45.078037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:15:45.078037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:45.078037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:15:45.078037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:15:45.078037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:45.078037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:15:45.079287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:45.079287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:15:45.079287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:15:45.079287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:45.079287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:15:45.079287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:45.079287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:15:45.079287 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:15:45.079287 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:15:45.079287 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:15:45.079524 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:15:45.079524 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:15:45.079524 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:45.079524 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:15:45.079524 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:45.079524 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:15:45.079524 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:15:45.079524 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:45.079524 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:15:45.079524 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:45.079524 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:15:45.080437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:45.080437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:15:45.080437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:15:45.080437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:15:45.080437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:45.080437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:15:45.080437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:45.080437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:15:45.080437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:45.080437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:15:45.080437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:45.080437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:15:45.080437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:45.080437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:15:45.080437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:15:45.080766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:45.080766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:15:45.080766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:45.080766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:15:45.080766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:15:45.080766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:45.080766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:15:45.080766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:45.080766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:15:45.080766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:45.080766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:15:45.080766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:45.080766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:15:45.080766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:15:45.081062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:15:45.081062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:15:45.081062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:15:45.081062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:15:45.081062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:45.081062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:45.081062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:45.081062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:15:45.081062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:45.081062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:45.081062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:45.081062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:15:45.081062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:45.081329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:15:45.081329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:15:45.081329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:45.081329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:15:45.081329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:45.081329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:15:45.081329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:15:45.081329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:45.081329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:15:45.081329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:45.081329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:15:45.081329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:45.081600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:15:45.081600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:15:45.081600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:15:45.081600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:15:45.081600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:15:45.081600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:45.081600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:15:45.081600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:15:45.081600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:15:45.081600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:15:45.081600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:15:45.081600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:45.081600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:45.081600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:15:45.081600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:15:45.081600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:15:45.081600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:15:45.081600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:15:45.081600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:15:45.082001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:45.082001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:45.082001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:45.082001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:15:45.082001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:45.082001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:45.082001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:45.082001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:15:45.082001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:45.082001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:15:45.082001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:15:45.082001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:45.082001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:15:45.082001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:45.082001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:15:45.082001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:15:45.082001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:45.082001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:15:45.082001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:45.082001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:15:45.090435 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:15:45.090547 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:15:45.090629 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:15:45.090728 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:15:45.090971 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:15:45.091191 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:15:45.091329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:45.091745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:15:45.091745 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:45.091745 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:15:45.091745 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:15:45.091745 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:45.091745 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:15:45.091745 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:45.091745 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:15:45.092271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:45.092271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:15:45.092271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:45.092271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:15:45.092271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:15:45.092271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:15:45.092271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:15:45.092271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:45.092271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:45.092271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:45.092271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:15:45.092271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:45.092668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:45.092668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:45.092668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:15:45.092668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:45.092668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:15:45.092668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:15:45.092668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:45.092668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:15:45.092668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:45.092668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:15:45.092668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:15:45.092668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:45.092668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:15:45.092986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:45.092986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:15:45.092986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:45.092986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:15:45.092986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:15:45.092986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:15:45.092986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:15:45.092986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:15:45.092986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:45.092986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:15:45.092986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:15:45.092986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:15:45.092986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:15:45.092986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:15:45.092986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:45.093351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:45.093351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:15:45.093351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:15:45.093351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:15:45.093351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:15:45.093351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:45.093351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:45.093351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:45.093351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:15:45.093351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:45.093351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:45.093351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:45.093351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:15:45.093351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:45.093351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:15:45.093351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:15:45.093351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:45.093351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:15:45.093351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:45.093903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:15:45.093903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:15:45.093903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:45.093903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:15:45.093903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:45.093903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:15:49.827908 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:15:49.828833 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:15:49.828833 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:15:49.830084 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:15:49.867721 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:15:49.868200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:49.869169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:15:49.869169 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:49.869169 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:15:49.869169 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:15:49.869169 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:49.869169 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:15:49.869169 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:49.869169 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:15:49.869947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:49.869947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:15:49.869947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:49.869947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:15:49.869947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:15:49.869947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:15:49.869947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:15:49.869947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:49.869947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:49.869947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:49.869947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:15:49.869947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:49.870594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:49.870594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:49.870594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:15:49.870594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:49.870594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:15:49.870594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:15:49.870594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:49.870594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:15:49.870594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:49.870594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:15:49.870594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:15:49.870594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:49.870594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:15:49.871094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:49.871094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:15:49.871094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:49.871094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:15:49.871094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:15:49.871094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:15:49.871094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:15:49.871094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:15:49.871094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:49.871094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:15:49.871094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:15:49.871094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:15:49.871094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:15:49.871094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:15:49.871094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:49.873163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:49.873163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:15:49.873163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:15:49.873163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:15:49.873163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:15:49.873163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:49.873163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:49.873163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:49.873163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:15:49.873163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:49.873163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:49.873163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:49.873163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:15:49.873163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:49.873163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:15:49.873163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:15:49.873163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:49.873163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:15:49.873163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:49.874926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:15:49.874926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:15:49.874926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:49.874926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:15:49.874926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:49.874926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:15:50.064549 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:15:50.064927 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:15:50.065092 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:15:50.066077 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:15:50.066242 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:15:50.066393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:50.067032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:15:50.067032 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:50.067032 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:15:50.067032 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:15:50.067032 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:50.067032 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:15:50.067032 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:50.067032 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:15:50.068510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:50.068510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:15:50.068510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:50.068510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:15:50.068510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:15:50.068510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:15:50.068510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:15:50.068510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:50.068510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:50.068510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:50.068510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:15:50.068510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:50.069044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:50.069044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:50.069044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:15:50.069044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:50.069044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:15:50.069044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:15:50.069044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:50.069044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:15:50.069044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:50.069044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:15:50.069044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:15:50.069044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:50.069044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:15:50.069610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:50.069610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:15:50.070748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:50.070748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:15:50.070748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:15:50.070748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:15:50.070748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:15:50.070748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:15:50.070748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:50.070748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:15:50.070748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:15:50.070748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:15:50.070748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:15:50.070748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:15:50.070748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:50.070748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:50.070748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:15:50.070748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:15:50.071495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:15:50.071495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:15:50.071495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:50.071495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:50.071495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:50.071495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:15:50.071495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:50.071495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:50.071495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:50.071495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:15:50.071495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:50.071495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:15:50.071495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:15:50.071495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:50.071495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:15:50.071495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:50.071495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:15:50.071495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:15:50.071495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:50.072219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:15:50.072219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:50.072219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:15:50.072219 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:15:50.072219 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:15:50.072219 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:15:50.072219 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:15:50.072219 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:15:50.072603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:50.073361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:15:50.073361 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:50.073361 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:15:50.073361 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:15:50.073361 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:50.073361 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:15:50.073361 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:50.073361 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:15:50.074300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:50.074300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:15:50.074300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:15:50.074300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:15:50.074300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:50.074300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:15:50.074300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:50.074300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:15:50.074300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:50.074300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:15:50.074300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:50.074300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:15:50.074300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:50.074300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:15:50.074300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:15:50.074942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:50.074942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:15:50.074942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:50.074942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:15:50.074942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:15:50.074942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:50.074942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:15:50.074942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:50.074942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:15:50.075112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:50.075112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:15:50.075112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:50.075112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:15:50.075112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:15:50.075112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:15:50.075112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:15:50.075112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:15:50.075112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:15:50.075112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:15:50.075112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:50.075112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:50.075333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:50.075333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:15:50.075333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:50.075333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:50.075333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:50.075333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:15:50.075333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:50.075333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:15:50.075333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:15:50.075333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:50.075333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:15:50.075333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:50.075333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:15:50.075333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:15:50.075599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:50.075599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:15:50.075599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:50.075599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:15:50.076102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:50.076102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:15:50.076102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:15:50.076102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:15:50.076102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:15:50.076102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:15:50.076102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:50.076102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:15:50.076102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:15:50.076102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:15:50.076102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:15:50.076102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:15:50.076102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:50.076102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:50.076102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:15:50.076102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:15:50.076419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:15:50.076419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:15:50.076419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:15:50.076419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:15:50.076419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:15:50.076419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:50.076419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:50.076419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:50.076419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:15:50.076419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:50.076419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:50.076419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:50.076419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:15:50.076419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:50.076419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:15:50.076419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:15:50.076419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:50.076419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:15:50.076727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:50.076727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:15:50.076727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:15:50.076727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:50.076727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:15:50.076727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:50.076727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:15:50.078085 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:15:50.078241 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:15:50.078351 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:15:50.078499 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:15:50.078778 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:15:50.078948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:50.079346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:15:50.079346 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:50.079346 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:15:50.079346 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:15:50.079346 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:50.079346 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:15:50.079346 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:50.079346 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:15:50.080112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:50.080112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:15:50.080112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:15:50.080112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:15:50.080112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:50.080112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:15:50.080112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:50.080112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:15:50.080112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:50.080112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:15:50.080112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:50.080112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:15:50.080112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:50.080112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:15:50.080112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:15:50.080387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:50.080387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:15:50.080387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:50.080387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:15:50.080387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:15:50.080387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:50.080387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:15:50.080387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:50.080387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:15:50.081016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:50.081016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:15:50.081016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:50.081016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:15:50.081016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:15:50.081016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:15:50.081016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:15:50.081016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:15:50.081016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:15:50.081016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:50.081016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:50.081253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:50.081253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:15:50.081253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:50.081253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:50.081253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:50.081253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:15:50.081253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:50.081253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:15:50.081253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:15:50.081253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:50.081253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:15:50.081253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:50.081253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:15:50.081253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:15:50.081509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:50.081509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:15:50.081509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:50.081509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:15:50.082301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:50.082301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:15:50.082301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:15:50.082301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:15:50.082301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:15:50.082301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:15:50.082301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:50.082301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:15:50.082301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:15:50.082301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:15:50.082301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:15:50.082301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:15:50.082301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:50.082301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:50.082301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:15:50.082301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:15:50.082301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:15:50.082650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:15:50.082650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:15:50.082650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:15:50.082650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:50.082650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:50.082650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:50.082650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:15:50.082650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:50.082650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:50.082650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:50.082650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:15:50.082650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:50.082650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:15:50.082650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:15:50.082650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:50.082650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:15:50.082650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:50.082650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:15:50.082650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:15:50.083131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:50.083131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:15:50.083131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:50.083131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:15:50.090686 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:15:50.090900 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:15:50.091004 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:15:50.091168 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:15:50.091422 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:15:50.091610 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:15:50.091712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:50.092258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:15:50.092258 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:50.092258 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:15:50.092258 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:15:50.092258 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:50.092258 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:15:50.092258 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:50.092258 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:15:50.093653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:50.093653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:15:50.093653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:50.093653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:15:50.093653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:15:50.093653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:15:50.093653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:15:50.093653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:50.093653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:50.093653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:50.093653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:15:50.093653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:50.094083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:50.094083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:50.094083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:15:50.094083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:50.094083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:15:50.094083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:15:50.094083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:50.094083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:15:50.094083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:50.094083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:15:50.094083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:15:50.094083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:50.094083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:15:50.094596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:50.094596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:15:50.095360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:50.095360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:15:50.095360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:15:50.095360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:15:50.095360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:15:50.095360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:15:50.095360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:50.095360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:15:50.095360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:15:50.095360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:15:50.095360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:15:50.095360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:15:50.095360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:50.095360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:50.095360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:15:50.095360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:15:50.097873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:15:50.097873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:15:50.097873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:50.097873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:50.097873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:50.097873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:15:50.097873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:50.097873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:50.097873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:50.097873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:15:50.097873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:50.097873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:15:50.097873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:15:50.097873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:50.097873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:15:50.097873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:50.097873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:15:50.097873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:15:50.097873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:50.098893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:15:50.098893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:50.098893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:15:54.828634 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:15:54.829379 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:15:54.829379 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:15:54.829795 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:15:54.855223 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:15:54.855223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:54.855453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:15:54.855453 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:54.855453 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:15:54.855453 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:15:54.855453 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:54.855453 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:15:54.855453 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:54.855453 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:15:54.856122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.026 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:54.856122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:15:54.856122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:54.856122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:15:54.856122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:15:54.856122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:15:54.856122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:15:54.856122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:54.856122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:54.856122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:54.856122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:15:54.856122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:54.856398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:54.856398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:54.856398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:15:54.856398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:54.856398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:15:54.856398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:15:54.856398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:54.856398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:15:54.856398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:54.856398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:15:54.856398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:15:54.856398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:54.856398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:15:54.856687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:54.856687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:15:54.856971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:54.856971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:15:54.856971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:15:54.856971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:15:54.856971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:15:54.856971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:15:54.856971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:54.856971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:15:54.856971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:15:54.856971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:15:54.856971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:15:54.856971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:15:54.856971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:54.856971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:54.856971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:15:54.856971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:15:54.857843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:15:54.857843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:15:54.857843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:54.857843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:54.857843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:54.857843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:15:54.857843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:54.857843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:54.857843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:54.857843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:15:54.857843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:54.857843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:15:54.857843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:15:54.857843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:54.857843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:15:54.857843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:54.857843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:15:54.857843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:15:54.857843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:54.858623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:15:54.858623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:54.858623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:15:55.065069 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:15:55.065663 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:15:55.065663 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:15:55.067845 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:15:55.067845 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:15:55.067845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:55.067845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:15:55.067845 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:55.067845 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:15:55.067845 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:15:55.067845 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:55.067845 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:15:55.067845 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:55.067845 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:15:55.068903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:55.068903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:15:55.068903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:55.068903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:15:55.068903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:15:55.068903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:15:55.068903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:15:55.068903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:55.068903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:55.068903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:55.068903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:15:55.068903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:55.069534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:55.069534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:55.069534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:15:55.069534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:55.069534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:15:55.069534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:15:55.069534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:55.069534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:15:55.069534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:55.069534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:15:55.069534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:15:55.069534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:55.069534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:15:55.070124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:55.070124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:15:55.070124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:55.070124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:15:55.070124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:15:55.070124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:15:55.070124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:15:55.070124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:15:55.070124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:55.070124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:15:55.070124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:15:55.070124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:15:55.070124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:15:55.070124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:15:55.070124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:55.072137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:55.072137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:15:55.072137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:15:55.072137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:15:55.072137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:15:55.072137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:55.072137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:55.072137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:55.072137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:15:55.072137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:55.072137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:55.072137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:55.072137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:15:55.072137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:55.072137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:15:55.072137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:15:55.072137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:55.072137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:15:55.072137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:55.072911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:15:55.072911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:15:55.072911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:55.072911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:15:55.072911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:55.072911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:15:55.072911 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:15:55.072911 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:15:55.072911 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:15:55.072911 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:15:55.073210 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:15:55.073210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:55.073759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:15:55.073759 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:55.073759 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:15:55.073759 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:15:55.073759 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:55.073759 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:15:55.073759 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:55.073759 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:15:55.074497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:55.074497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:15:55.074497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:15:55.074497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:15:55.074497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:55.074497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:15:55.074497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:55.074497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:15:55.074497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:55.074497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:15:55.074497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:55.074497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:15:55.074497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:55.074497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:15:55.074497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:15:55.075002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:55.075002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:15:55.075002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:55.075002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:15:55.075002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:15:55.075002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:55.075002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:15:55.075002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:55.075002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:15:55.075270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:55.075270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:15:55.075270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:55.075270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:15:55.075270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:15:55.075270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:15:55.075270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:15:55.075270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:15:55.075270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:15:55.075270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:15:55.075270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:55.075270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:55.075708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:55.075708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:15:55.075708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:55.075708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:55.075708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:55.075708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:15:55.075708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:55.075708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:15:55.075708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:15:55.075708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:55.075708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:15:55.075708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:55.075708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:15:55.075708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:15:55.076063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:55.076063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:15:55.076063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:55.076063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:15:55.076063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:55.076063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:15:55.076063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:15:55.076063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:15:55.076063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:15:55.076063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:15:55.076063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:55.076063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:15:55.076063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:15:55.076063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:15:55.076522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:15:55.076522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:15:55.076522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:55.076522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:55.076522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:15:55.076522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:15:55.076522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:15:55.076522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:15:55.076522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:15:55.076522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:15:55.076522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:15:55.076522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:55.076522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:55.076522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:55.076522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:15:55.076522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:55.076522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:55.076522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:55.076522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:15:55.077121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:55.077121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:15:55.077121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:15:55.077121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:55.077121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:15:55.077121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:55.077121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:15:55.077121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:15:55.077121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:55.077121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:15:55.077121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:55.077121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:15:55.078708 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:15:55.078791 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:15:55.078875 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:15:55.078954 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:15:55.079187 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:15:55.079392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:55.079761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:15:55.079761 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:55.079761 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:15:55.079761 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:15:55.079761 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:55.079761 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:15:55.079761 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:55.079761 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:15:55.080532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:55.080532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:15:55.080532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:15:55.080532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:15:55.080532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:55.080532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:15:55.080532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:55.080532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:15:55.080532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:55.080532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:15:55.080532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:55.080532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:15:55.080532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:55.080532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:15:55.080532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:15:55.080976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:55.080976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:15:55.080976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:55.080976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:15:55.080976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:15:55.080976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:55.080976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:15:55.080976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:55.080976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:15:55.081373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:55.081373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:15:55.081373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:55.081373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:15:55.081373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:15:55.081373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:15:55.081373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:15:55.081373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:15:55.081373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:15:55.081373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:55.081373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:55.081797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:55.081797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:15:55.081797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:55.081797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:55.081797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:55.081797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:15:55.081797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:55.081797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:15:55.081797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:15:55.081797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:55.081797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:15:55.081797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:55.081797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:15:55.081797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:15:55.082183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:55.082183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:15:55.082183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:55.082183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:15:55.082533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:55.082533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:15:55.082533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:15:55.082533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:15:55.082533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:15:55.082533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:15:55.082533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:55.082533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:15:55.082533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:15:55.082533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:15:55.082533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:15:55.082533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:15:55.082533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:55.082533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:55.082533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:15:55.082533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:15:55.082533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:15:55.083031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:15:55.083031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:15:55.083031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:15:55.083031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:55.083031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:55.083031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:55.083031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:15:55.083031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:55.083031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:55.083031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:55.083031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:15:55.083031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:55.083031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:15:55.083031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:15:55.083031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:55.083031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:15:55.083031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:55.083031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:15:55.083031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:15:55.083595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:55.083595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:15:55.083595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:55.083595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:15:55.091112 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:15:55.091304 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:15:55.091382 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:15:55.091567 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:15:55.091790 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:15:55.092188 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:15:55.092319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:55.092780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:15:55.092780 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:55.092780 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:15:55.092780 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:15:55.092780 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:55.092780 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:15:55.092780 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:55.092780 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:15:55.093663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:55.093663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:15:55.093663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:55.093663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:15:55.093663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:15:55.093663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:15:55.093663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:15:55.093663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:55.093663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:55.093663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:55.093663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:15:55.093663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:55.094107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:55.094107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:55.094107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:15:55.094107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:55.094107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:15:55.094107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:15:55.094107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:55.094107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:15:55.094107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:55.094107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:15:55.094107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:15:55.094107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:55.094107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:15:55.094512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:55.094512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:15:55.096525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:55.096525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:15:55.096525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:15:55.096525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:15:55.096525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:15:55.096525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:15:55.096525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:55.096525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:15:55.096525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:15:55.096525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:15:55.096525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:15:55.096525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:15:55.096525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:55.096525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:55.096525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:15:55.096525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:15:55.097218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:15:55.097218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:15:55.097218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:55.097218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:55.097218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:55.097218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:15:55.097218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:55.097218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:55.097218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:55.097218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:15:55.097218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:55.097218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:15:55.097218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:15:55.097218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:55.097218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:15:55.097218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:55.097218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:15:55.097218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:15:55.097218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:55.097785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:15:55.097785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:55.097785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:15:59.829210 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:15:59.829704 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:15:59.829704 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:15:59.830648 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:15:59.868200 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:15:59.868736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:59.869155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:15:59.869155 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:59.869155 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:15:59.869155 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:15:59.869155 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:59.869155 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:15:59.869155 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:59.869155 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:15:59.869999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:59.869999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:15:59.869999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:59.869999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:15:59.869999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:15:59.869999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:15:59.869999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:15:59.869999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:59.869999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:59.869999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:59.869999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:15:59.869999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:59.870367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:15:59.870367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:59.870367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:15:59.870367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:59.870367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:15:59.870367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:15:59.870367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:59.870367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:15:59.870367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:59.870367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:15:59.870367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:15:59.870367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:59.870367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:15:59.870700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:59.870700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:15:59.872451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:59.872451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:15:59.872451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:15:59.872451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:15:59.872451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:15:59.872451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:15:59.872451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:59.872451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:15:59.872451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:15:59.872451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:15:59.872451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:15:59.872451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:15:59.872451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:15:59.872451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:15:59.872451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:15:59.872451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:15:59.872838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:15:59.872838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:15:59.872838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:59.872838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:59.872838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:15:59.872838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:15:59.872838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:15:59.872838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:15:59.872838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:15:59.872838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:15:59.872838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:15:59.872838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:15:59.872838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:15:59.872838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:15:59.872838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:15:59.872838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:15:59.872838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:15:59.872838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:15:59.872838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:15:59.873242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:15:59.873242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:15:59.873242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:16:00.065606 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:16:00.066174 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:16:00.066361 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:16:00.067525 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:16:00.067841 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:16:00.068079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:00.068899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:16:00.068899 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:00.068899 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:16:00.068899 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:16:00.068899 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:00.068899 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:16:00.068899 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:00.068899 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:16:00.070482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:00.070482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:16:00.070482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:00.070482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:16:00.070482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:16:00.070482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:16:00.070482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:16:00.070482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:00.070482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:00.070482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:00.070482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:16:00.070482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:00.070800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:00.070800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:00.070800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:16:00.070800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:00.070800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:16:00.070800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:16:00.070800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:00.070800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:16:00.070800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:00.070800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:16:00.070800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:16:00.070800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:00.070800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:16:00.071054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:00.071054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:16:00.072055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:00.072055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:16:00.072055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:16:00.072055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:16:00.072055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:16:00.072055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:16:00.072055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:00.072055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:16:00.072055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:16:00.072055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:16:00.072055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:16:00.072055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:16:00.072055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:00.072055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:00.072055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:16:00.072055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:16:00.072388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:16:00.072388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:16:00.072388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:00.072388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:00.072388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:00.072388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:16:00.072388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:00.072388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:00.072388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:00.072388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:16:00.072388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:00.072388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:16:00.072388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:16:00.072388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:00.072388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:16:00.072388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:00.072388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:16:00.072388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:16:00.072388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:00.072802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:16:00.072802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:00.072802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:16:00.072802 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:16:00.072802 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:16:00.072802 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:16:00.072957 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:16:00.073265 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:16:00.073536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:00.074111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:16:00.074111 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:00.074111 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:16:00.074111 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:16:00.074111 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:00.074111 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:16:00.074111 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:00.074111 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:16:00.074931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:00.074931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:16:00.074931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:16:00.074931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:16:00.074931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:00.074931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:16:00.074931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:00.074931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:16:00.074931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:00.074931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:16:00.074931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:00.074931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:16:00.074931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:00.074931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:16:00.074931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:16:00.075240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:00.075240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:16:00.075240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:00.075240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:16:00.075240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:16:00.075240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:00.075240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:16:00.075240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:00.075240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:16:00.076313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:00.076313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:16:00.076313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:00.076313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:16:00.076313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:16:00.076313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:16:00.076313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:16:00.076313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:16:00.076313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:16:00.076313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:16:00.076313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:00.076313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:00.076630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:00.076630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:16:00.076630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:00.076630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:00.076630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:00.076630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:16:00.076630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:00.076630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:16:00.076630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:16:00.076630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:00.076630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:16:00.076630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:00.076630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:16:00.076630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:16:00.076897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:00.076897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:16:00.076897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:00.076897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:16:00.076897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:00.076897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:16:00.076897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:16:00.076897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:16:00.076897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:16:00.076897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:16:00.076897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:00.076897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:16:00.076897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:16:00.076897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:16:00.077179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:16:00.077179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:16:00.077179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:00.077179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:00.077179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:16:00.077179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:16:00.077179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:16:00.077179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:16:00.077179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:16:00.077179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:16:00.077179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:16:00.077179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:00.077179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:00.077179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:00.077179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:16:00.077179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:00.077179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:00.077179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:00.077179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:16:00.077573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:00.077573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:16:00.077573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:16:00.077573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:00.077573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:16:00.077573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:00.077573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:16:00.077573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:16:00.077573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:00.077573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:16:00.077573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:00.077573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:16:00.079038 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:16:00.079318 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:16:00.079445 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:16:00.079578 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:16:00.079946 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:16:00.080246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:00.080852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:16:00.080852 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:00.080852 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:16:00.080852 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:16:00.080852 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:00.080852 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:16:00.080852 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:00.080852 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:16:00.081639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:00.081639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:16:00.081639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:16:00.081639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:16:00.081639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:00.081639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:16:00.081639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:00.081639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:16:00.081639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:00.081639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:16:00.081639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:00.081639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:16:00.081639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:00.081639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:16:00.081639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:16:00.081940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:00.081940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:16:00.081940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:00.081940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:16:00.081940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:16:00.081940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:00.081940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:16:00.081940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:00.081940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:16:00.082647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:00.082647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:16:00.082647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:00.082647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:16:00.082647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:16:00.082647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:16:00.082647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:16:00.082647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:16:00.082647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:16:00.082647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:00.082647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:00.082894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:00.082894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:16:00.082894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:00.082894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:00.082894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:00.082894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:16:00.082894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:00.082894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:16:00.082894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:16:00.082894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:00.082894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:16:00.082894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:00.082894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:16:00.082894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:16:00.083554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:00.083554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:16:00.083554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:00.083554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:16:00.083900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:00.083900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:16:00.083900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:16:00.083900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:16:00.083900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:16:00.083900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:16:00.083900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:00.083900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:16:00.083900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:16:00.083900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:16:00.083900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:16:00.083900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:16:00.083900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:00.083900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:00.083900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:16:00.083900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:16:00.083900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:16:00.084242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:16:00.084242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:16:00.084242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:16:00.084242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:00.084242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:00.084242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:00.084242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:16:00.084242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:00.084242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:00.084242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:00.084242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:16:00.084242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:00.084242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:16:00.084242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:16:00.084242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:00.084242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:16:00.084242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:00.084242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:16:00.084242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:16:00.084643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:00.084643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:16:00.084643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:00.084643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:16:00.091598 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:16:00.091891 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:16:00.092011 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:16:00.092164 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:16:00.092630 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:16:00.092958 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:16:00.093150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:00.093813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:16:00.093813 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:00.093813 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:16:00.093813 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:16:00.093813 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:00.093813 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:16:00.093813 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:00.093813 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:16:00.094669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:00.094669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:16:00.094669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:00.094669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:16:00.094669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:16:00.094669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:16:00.094669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:16:00.094669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:00.094669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:00.094669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:00.094669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:16:00.094669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:00.094946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:00.094946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:00.094946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:16:00.094946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:00.094946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:16:00.094946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:16:00.094946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:00.094946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:16:00.094946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:00.094946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:16:00.094946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:16:00.094946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:00.094946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:16:00.095174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:00.095174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:16:00.095737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:00.095737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:16:00.095737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:16:00.095737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:16:00.095737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:16:00.095737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:16:00.095737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:00.095737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:16:00.095737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:16:00.095737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:16:00.095737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:16:00.095737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:16:00.095737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:00.095737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:00.095737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:16:00.095737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:16:00.096039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:16:00.096039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:16:00.096039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:00.096039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:00.096039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:00.096039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:16:00.096039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:00.096039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:00.096039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:00.096039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:16:00.096039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:00.096039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:16:00.096039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:16:00.096039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:00.096039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:16:00.096039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:00.096039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:16:00.096039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:16:00.096039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:00.096376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:16:00.096376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:00.096376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:16:04.829715 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:16:04.830563 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:16:04.830563 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:16:04.831395 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:16:04.876124 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:16:04.876306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:04.876946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:16:04.876946 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:04.876946 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:16:04.876946 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:16:04.876946 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:04.876946 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:16:04.876946 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:04.876946 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:16:04.878007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.047 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:04.878007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:16:04.878007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:04.878007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:16:04.878007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:16:04.878007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:16:04.878007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:16:04.878007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:04.878007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:04.878007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:04.878007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:16:04.878007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:04.878637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:04.878637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:04.878637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:16:04.878637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:04.878637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:16:04.878637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:16:04.878637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:04.878637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:16:04.878637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:04.878637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:16:04.878637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:16:04.878637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:04.878637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:16:04.879225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:04.879225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:16:04.879225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:04.879225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:16:04.879225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:16:04.879225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:16:04.879225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:16:04.879225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:16:04.879225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:04.879225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:16:04.879225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:16:04.879225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:16:04.879225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:16:04.879225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:16:04.879225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:04.880024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:04.880024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:16:04.880024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:16:04.880024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:16:04.880024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:16:04.880024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:04.880024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:04.880024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:04.880024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:16:04.880024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:04.880024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:04.880024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:04.880024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:16:04.880024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:04.880024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:16:04.880024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:16:04.880024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:04.880024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:16:04.880024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:04.880882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:16:04.880882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:16:04.880882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:04.880882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:16:04.880882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:04.880882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:16:05.066231 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:16:05.066619 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:16:05.066740 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:16:05.067257 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:16:05.067380 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:16:05.067598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:05.068042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:16:05.068042 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:05.068042 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:16:05.068042 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:16:05.068042 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:05.068042 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:16:05.068042 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:05.068042 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:16:05.069111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:05.069111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:16:05.069111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:05.069111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:16:05.069111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:16:05.069111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:16:05.069111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:16:05.069111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:05.069111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:05.069111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:05.069111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:16:05.069111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:05.070268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:05.070268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:05.070268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:16:05.070268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:05.070268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:16:05.070268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:16:05.070268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:05.070268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:16:05.070268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:05.070268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:16:05.070268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:16:05.070268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:05.070268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:16:05.071719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:05.071719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:16:05.071719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:05.071719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:16:05.071719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:16:05.071719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:16:05.071719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:16:05.071719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:16:05.071719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:05.071719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:16:05.071719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:16:05.071719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:16:05.071719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:16:05.071719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:16:05.071719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:05.072645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:05.072645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:16:05.072645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:16:05.072645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:16:05.072645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:16:05.072645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:05.072645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:05.072645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:05.072645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:16:05.072645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:05.072645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:05.072645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:05.072645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:16:05.072645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:05.072645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:16:05.072645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:16:05.072645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:05.072645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:16:05.072645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:05.073151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:16:05.073151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:16:05.073151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:05.073151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:16:05.073151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:05.073151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:16:05.073151 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:16:05.073151 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:16:05.073151 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:16:05.073151 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:16:05.073574 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:16:05.073574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:05.074178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:16:05.074178 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:05.074178 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:16:05.074178 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:16:05.074178 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:05.074178 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:16:05.074178 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:05.074178 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:16:05.074783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:05.074783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:16:05.074783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:16:05.074783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:16:05.074783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:05.074783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:16:05.074783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:05.074783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:16:05.074783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:05.074783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:16:05.074783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:05.074783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:16:05.074783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:05.074783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:16:05.074783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:16:05.075424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:05.075424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:16:05.075424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:05.075424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:16:05.075424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:16:05.075424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:05.075424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:16:05.075424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:05.075424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:16:05.076506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:05.076506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:16:05.076506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:05.076506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:16:05.076506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:16:05.076506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:16:05.076506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:16:05.076506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:16:05.076506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:16:05.076506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:16:05.076506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:05.076506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:05.076976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:05.076976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:16:05.076976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:05.076976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:05.076976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:05.076976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:16:05.076976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:05.076976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:16:05.076976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:16:05.076976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:05.076976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:16:05.076976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:05.076976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:16:05.076976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:16:05.077491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:05.077491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:16:05.077491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:05.077491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:16:05.077491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:05.077491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:16:05.077491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:16:05.077491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:16:05.077491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:16:05.077491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:16:05.077491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:05.077491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:16:05.077491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:16:05.077491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:16:05.078008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:16:05.078008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:16:05.078008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:05.078008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:05.078008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:16:05.078008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:16:05.078008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:16:05.078008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:16:05.078008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:16:05.078008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:16:05.078008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:16:05.078008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:05.078008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:05.078008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:05.078008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:16:05.078008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:05.078008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:05.078008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:05.078008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:16:05.079654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:05.079654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:16:05.079654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:16:05.079654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:05.079654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:16:05.079654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:05.079654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:16:05.079654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:16:05.079654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:05.079654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:16:05.079654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:05.079654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:16:05.079654 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:16:05.079654 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:16:05.080457 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:16:05.080457 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:16:05.080457 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:16:05.080457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:05.080769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:16:05.080769 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:05.080769 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:16:05.080769 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:16:05.080769 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:05.080769 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:16:05.080769 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:05.080769 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:16:05.081815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:05.081815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:16:05.081815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:16:05.081815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:16:05.081815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:05.081815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:16:05.081815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:05.081815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:16:05.081815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:05.081815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:16:05.081815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:05.081815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:16:05.081815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:05.081815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:16:05.081815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:16:05.082698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:05.082698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:16:05.082698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:05.082698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:16:05.082698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:16:05.082698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:05.082698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:16:05.082698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:05.082698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:16:05.083104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:05.083104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:16:05.083104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:05.083104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:16:05.083104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:16:05.083104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:16:05.083104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:16:05.083104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:16:05.083104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:16:05.083104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:05.083104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:05.083503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:05.083503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:16:05.083503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:05.083503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:05.083503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:05.083503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:16:05.083503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:05.083503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:16:05.083503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:16:05.083503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:05.083503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:16:05.083503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:05.083503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:16:05.083503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:16:05.083804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:05.083804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:16:05.083804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:05.083804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:16:05.085032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:05.085032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:16:05.085032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:16:05.085032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:16:05.085032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:16:05.085032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:16:05.085032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:05.085032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:16:05.085032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:16:05.085032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:16:05.085032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:16:05.085032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:16:05.085032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:05.085032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:05.085032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:16:05.085032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:16:05.085032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:16:05.085377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:16:05.085377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:16:05.085377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:16:05.085377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:05.085377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:05.085377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:05.085377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:16:05.085377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:05.085377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:05.085377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:05.085377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:16:05.085377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:05.085377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:16:05.085377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:16:05.085377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:05.085377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:16:05.085377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:05.085377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:16:05.085377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:16:05.085757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:05.085757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:16:05.085757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:05.085757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:16:05.091910 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:16:05.092193 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:16:05.092193 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:16:05.092398 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:16:05.092512 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:16:05.092724 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:16:05.092724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:05.092999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:16:05.092999 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:05.092999 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:16:05.092999 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:16:05.092999 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:05.092999 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:16:05.092999 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:05.092999 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:16:05.093575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:05.093575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:16:05.093575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:05.093575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:16:05.093575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:16:05.093575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:16:05.093575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:16:05.093575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:05.093575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:05.093575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:05.093575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:16:05.093575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:05.094235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:05.094235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:05.094235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:16:05.094235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:05.094235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:16:05.094235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:16:05.094235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:05.094235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:16:05.094235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:05.094235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:16:05.094235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:16:05.094235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:05.094235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:16:05.094605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:05.094605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:16:05.094605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:05.094605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:16:05.094605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:16:05.094605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:16:05.094605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:16:05.094605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:16:05.094605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:05.094605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:16:05.094605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:16:05.094605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:16:05.094605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:16:05.094605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:16:05.094605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:05.095073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:05.095073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:16:05.095073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:16:05.095073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:16:05.095073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:16:05.095073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:05.095073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:05.095073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:05.095073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:16:05.095073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:05.095073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:05.095073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:05.095073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:16:05.095073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:05.095073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:16:05.095073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:16:05.095073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:05.095073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:16:05.095073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:05.095636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:16:05.095636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:16:05.095636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:05.095636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:16:05.095636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:05.095636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:16:09.830258 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:16:09.830762 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:16:09.830762 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:16:09.831333 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:16:09.860212 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:16:09.860505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:09.861927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:16:09.861927 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:09.861927 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:16:09.861927 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:16:09.861927 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:09.861927 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:16:09.861927 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:09.861927 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:16:09.861927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.030 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:09.861927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:16:09.861927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:09.861927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:16:09.861927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:16:09.861927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:16:09.861927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:16:09.862755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:09.862755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:09.862755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:09.862755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:16:09.862755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:09.862755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:09.862755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:09.862755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:16:09.862755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:09.862755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:16:09.862755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:16:09.862755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:09.862755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:16:09.862755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:09.863321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:16:09.863321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:16:09.863321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:09.863321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:16:09.863321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:09.863321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:16:09.863321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:09.863321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:16:09.863321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:16:09.863321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:16:09.863321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:16:09.863321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:16:09.863321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:09.863321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:16:09.863952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:16:09.863952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:16:09.863952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:16:09.863952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:16:09.863952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:09.863952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:09.863952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:16:09.863952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:16:09.863952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:16:09.863952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:16:09.863952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:09.863952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:09.863952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:09.863952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:16:09.863952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:09.863952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:09.863952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:09.863952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:16:09.863952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:09.863952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:16:09.863952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:16:09.864898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:09.864898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:16:09.864898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:09.864898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:16:09.864898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:16:09.864898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:09.864898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:16:09.864898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:09.864898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:16:10.066877 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:16:10.067492 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:16:10.067720 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:16:10.069654 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:16:10.069939 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:16:10.070109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:10.070842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:16:10.070842 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:10.070842 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:16:10.070842 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:16:10.070842 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:10.070842 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:16:10.070842 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:10.070842 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:16:10.072342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:10.072342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:16:10.072342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:10.072342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:16:10.072342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:16:10.072342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:16:10.072342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:16:10.072342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:10.072342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:10.072342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:10.072342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:16:10.072342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:10.072728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:10.072728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:10.072728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:16:10.072728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:10.072728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:16:10.072728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:16:10.072728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:10.072728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:16:10.072728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:10.072728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:16:10.072728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:16:10.072728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:10.072728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:16:10.073062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:10.073062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:16:10.074978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:10.074978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:16:10.074978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:16:10.074978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:16:10.074978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:16:10.074978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:16:10.074978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:10.074978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:16:10.074978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:16:10.074978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:16:10.074978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:16:10.074978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:16:10.074978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:10.074978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:10.074978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:16:10.074978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:16:10.075486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:16:10.075486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:16:10.075486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:10.075486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:10.075486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:10.075486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:16:10.075486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:10.075486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:10.075486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:10.075486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:16:10.075486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:10.075486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:16:10.075486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:16:10.075486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:10.075486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:16:10.075486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:10.075486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:16:10.075486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:16:10.075486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:10.076029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:16:10.076029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:10.076029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:16:10.076029 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:16:10.076029 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:16:10.076029 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:16:10.076029 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:16:10.076525 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:16:10.076686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:10.077297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:16:10.077297 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:10.077297 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:16:10.077297 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:16:10.077297 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:10.077297 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:16:10.077297 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:10.077297 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:16:10.079626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:10.079626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:16:10.079626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:16:10.079626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:16:10.079626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:10.079626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:16:10.079626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:10.079626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:16:10.079626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:10.079626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:16:10.079626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:10.079626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:16:10.079626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:10.079626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:16:10.079626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:16:10.080176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:10.080176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:16:10.080176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:10.080176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:16:10.080176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:16:10.080176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:10.080176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:16:10.080176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:10.080176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:16:10.080176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:10.080176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:16:10.080176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:10.080176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:16:10.080176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:16:10.080657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:16:10.080657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:16:10.080657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:16:10.080657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:16:10.080657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:16:10.080657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:10.080657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:10.080657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:10.080657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:16:10.080657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:10.080657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:10.080657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:10.080657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:16:10.081030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:10.081030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:16:10.081030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:16:10.081030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:10.081030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:16:10.081030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:10.081030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:16:10.081030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:16:10.081030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:10.081030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:16:10.081030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:10.081030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:16:10.081448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:10.081448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:16:10.081448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:16:10.081448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:16:10.081448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:16:10.081448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:16:10.081448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:10.081448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:16:10.081448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:16:10.081448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:16:10.081448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:16:10.081448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:16:10.081448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:10.081448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:10.081448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:16:10.081448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:16:10.081868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:16:10.081868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:16:10.081868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:16:10.081868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:16:10.081868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:16:10.081868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:10.081868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:10.081868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:10.081868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:16:10.081868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:10.081868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:10.081868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:10.081868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:16:10.081868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:10.081868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:16:10.081868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:16:10.081868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:10.081868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:16:10.082365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:10.082365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:16:10.082365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:16:10.082365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:10.082365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:16:10.082365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:10.082365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:16:10.082365 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:16:10.082365 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:16:10.082365 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:16:10.084332 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:16:10.084332 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:16:10.084332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:10.084332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:16:10.084332 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:10.084332 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:16:10.084332 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:16:10.084332 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:10.084332 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:16:10.084332 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:10.084332 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:16:10.084951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:10.084951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:16:10.084951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:16:10.084951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:16:10.084951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:10.084951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:16:10.084951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:10.084951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:16:10.084951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:10.084951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:16:10.084951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:10.084951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:16:10.084951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:10.084951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:16:10.084951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:16:10.085921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:10.085921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:16:10.085921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:10.085921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:16:10.085921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:16:10.085921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:10.085921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:16:10.085921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:10.085921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:16:10.085921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:10.085921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:16:10.085921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:10.085921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:16:10.085921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:16:10.086770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:16:10.086770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:16:10.086770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:16:10.086770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:16:10.086770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:10.086770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:10.086770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:10.086770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:16:10.086770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:10.086770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:10.086770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:10.086770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:16:10.086770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:10.087555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:16:10.087555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:16:10.087555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:10.087555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:16:10.087555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:10.087555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:16:10.087555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:16:10.087555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:10.087555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:16:10.087555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:10.087555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:16:10.087555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:10.088346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:16:10.088346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:16:10.088346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:16:10.088346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:16:10.088346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:16:10.088346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:10.088346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:16:10.088346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:16:10.088346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:16:10.088346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:16:10.088346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:16:10.088346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:10.088346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:10.088346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:16:10.088346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:16:10.088346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:16:10.088346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:16:10.088346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:16:10.088346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:16:10.089297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:10.089297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:10.089297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:10.089297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:16:10.089297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:10.089297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:10.089297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:10.089297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:16:10.089297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:10.089297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:16:10.089297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:16:10.089297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:10.089297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:16:10.089297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:10.089297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:16:10.089297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:16:10.089297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:10.089297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:16:10.089297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:10.089297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:16:10.092253 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:16:10.092618 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:16:10.092800 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:16:10.092933 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:16:10.093399 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:16:10.093768 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:16:10.093990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:10.094760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:16:10.094760 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:10.094760 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:16:10.094760 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:16:10.094760 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:10.094760 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:16:10.094760 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:10.094760 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:16:10.095678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:10.095678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:16:10.095678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:10.095678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:16:10.095678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:16:10.095678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:16:10.095678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:16:10.095678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:10.095678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:10.095678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:10.095678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:16:10.095678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:10.095989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:10.095989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:10.095989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:16:10.095989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:10.095989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:16:10.095989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:16:10.095989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:10.095989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:16:10.095989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:10.095989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:16:10.095989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:16:10.095989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:10.095989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:16:10.096212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:10.096212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:16:10.096636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:10.096636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:16:10.096636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:16:10.096636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:16:10.096636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:16:10.096636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:16:10.096636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:10.096636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:16:10.096636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:16:10.096636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:16:10.096636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:16:10.096636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:16:10.096636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:10.096636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:10.096636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:16:10.096636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:16:10.096939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:16:10.096939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:16:10.096939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:10.096939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:10.096939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:10.096939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:16:10.096939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:10.096939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:10.096939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:10.096939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:16:10.096939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:10.096939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:16:10.096939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:16:10.096939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:10.096939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:16:10.096939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:10.096939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:16:10.096939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:16:10.096939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:10.097260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:16:10.097260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:10.097260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:16:14.830748 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:16:14.831325 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:16:14.831325 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:16:14.832739 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:16:14.873963 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:16:14.874239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:14.874920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:16:14.874920 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:14.874920 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:16:14.874920 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:16:14.874920 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:14.874920 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:16:14.874920 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:14.874920 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:16:14.875732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:14.875732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:16:14.875732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:14.875732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:16:14.875732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:16:14.875732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:16:14.875732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:16:14.875732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:14.875732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:14.875732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:14.875732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:16:14.875732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:14.876109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:14.876109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:14.876109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:16:14.876109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:14.876109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:16:14.876109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:16:14.876109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:14.876109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:16:14.876109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:14.876109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:16:14.876109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:16:14.876109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:14.876109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:16:14.876492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:14.876492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:16:14.876492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:14.876492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:16:14.876492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:16:14.876492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:16:14.876492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:16:14.876492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:16:14.876492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:14.876492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:16:14.876492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:16:14.876492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:16:14.876492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:16:14.876492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:16:14.876492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:14.876915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:14.876915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:16:14.876915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:16:14.876915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:16:14.876915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:16:14.876915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:14.876915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:14.876915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:14.876915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:16:14.876915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:14.876915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:14.876915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:14.876915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:16:14.876915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:14.876915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:16:14.876915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:16:14.876915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:14.876915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:16:14.876915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:14.877435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:16:14.877435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:16:14.877435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:14.877435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:16:14.877435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:14.877435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:16:15.067525 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:16:15.068156 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:16:15.068357 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:16:15.069216 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:16:15.069398 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:16:15.069588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:15.070251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:16:15.070251 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:15.070251 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:16:15.070251 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:16:15.070251 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:15.070251 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:16:15.070251 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:15.070251 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:16:15.071788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:15.071788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:16:15.071788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:15.071788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:16:15.071788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:16:15.071788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:16:15.071788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:16:15.071788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:15.071788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:15.071788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:15.071788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:16:15.071788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:15.072143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:15.072143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:15.072143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:16:15.072143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:15.072143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:16:15.072143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:16:15.072143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:15.072143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:16:15.072143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:15.072143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:16:15.072143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:16:15.072143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:15.072143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:16:15.072478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:15.072478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:16:15.073649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:15.073649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:16:15.073649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:16:15.073649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:16:15.073649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:16:15.073649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:16:15.073649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:15.073649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:16:15.073649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:16:15.073649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:16:15.073649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:16:15.073649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:16:15.073649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:15.073649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:15.073649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:16:15.073649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:16:15.074166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:16:15.074166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:16:15.074166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:15.074166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:15.074166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:15.074166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:16:15.074166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:15.074166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:15.074166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:15.074166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:16:15.074166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:15.074166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:16:15.074166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:16:15.074166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:15.074166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:16:15.074166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:15.074166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:16:15.074166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:16:15.074166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:15.074660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:16:15.074660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:15.074660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:16:15.075967 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:16:15.076339 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:16:15.076599 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:16:15.076779 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:16:15.077105 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:16:15.077343 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:15.077887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:16:15.077887 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:15.077887 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:16:15.077887 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:16:15.077887 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:15.077887 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:16:15.077887 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:15.077887 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:16:15.078920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:15.078920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:16:15.078920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:16:15.078920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:16:15.078920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:15.078920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:16:15.078920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:15.078920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:16:15.078920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:15.078920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:16:15.078920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:15.078920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:16:15.078920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:15.078920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:16:15.078920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:16:15.079592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:15.079592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:16:15.079592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:15.079592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:16:15.079592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:16:15.079592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:15.079592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:16:15.079592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:15.079592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:16:15.079909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:15.079909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:16:15.079909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:15.079909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:16:15.079909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:16:15.079909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:16:15.079909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:16:15.079909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:16:15.079909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:16:15.079909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:16:15.079909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:15.079909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:15.080426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:15.080426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:16:15.080426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:15.080426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:15.080426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:15.080426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:16:15.080426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:15.080426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:16:15.080426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:16:15.080426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:15.080426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:16:15.080426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:15.080426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:16:15.080426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:16:15.080957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:15.080957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:16:15.080957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:15.080957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:16:15.081094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:15.081094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:16:15.081094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:16:15.081094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:16:15.081094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:16:15.081094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:16:15.081094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:15.081094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:16:15.081094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:16:15.081094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:16:15.081094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:16:15.081094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:16:15.081094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:15.081094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:15.081094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:16:15.081094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:16:15.081443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:16:15.081443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:16:15.081443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:16:15.081443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:16:15.081443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:16:15.081443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:15.081443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:15.081443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:15.081443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:16:15.081443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:15.081443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:15.081443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:15.081443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:16:15.081443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:15.081443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:16:15.081443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:16:15.081443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:15.081443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:16:15.081779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:15.081779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:16:15.081779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:16:15.081779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:15.081779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:16:15.081779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:15.081779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:16:15.081901 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:16:15.082119 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:16:15.082225 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:16:15.082358 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:16:15.082646 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:16:15.082883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:15.083313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:16:15.083313 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:15.083313 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:16:15.083313 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:16:15.083313 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:15.083313 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:16:15.083313 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:15.083313 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:16:15.084326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:15.084326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:16:15.084326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:16:15.084326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:16:15.084326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:15.084326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:16:15.084326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:15.084326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:16:15.084326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:15.084326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:16:15.084326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:15.084326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:16:15.084326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:15.084326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:16:15.084326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:16:15.084667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:15.084667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:16:15.084667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:15.084667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:16:15.084667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:16:15.084667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:15.084667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:16:15.084667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:15.084667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:16:15.084895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:15.084895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:16:15.084895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:15.084895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:16:15.084895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:16:15.084895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:16:15.084895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:16:15.084895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:16:15.084895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:16:15.084895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:15.084895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:15.085142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:15.085142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:16:15.085142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:15.085142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:15.085142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:15.085142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:16:15.085142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:15.085142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:16:15.085142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:16:15.085142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:15.085142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:16:15.085142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:15.085142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:16:15.085142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:16:15.085420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:15.085420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:16:15.085420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:15.085420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:16:15.085949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:15.085949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:16:15.085949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:16:15.085949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:16:15.085949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:16:15.085949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:16:15.085949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:15.085949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:16:15.085949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:16:15.085949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:16:15.085949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:16:15.085949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:16:15.085949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:15.085949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:15.085949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:16:15.085949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:16:15.085949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:16:15.086319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:16:15.086319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:16:15.086319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:16:15.086319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:15.086319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:15.086319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:15.086319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:16:15.086319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:15.086319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:15.086319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:15.086319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:16:15.086319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:15.086319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:16:15.086319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:16:15.086319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:15.086319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:16:15.086319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:15.086319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:16:15.086319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:16:15.086691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:15.086691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:16:15.086691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:15.086691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:16:15.092657 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:16:15.092892 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:16:15.093008 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:16:15.093174 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:16:15.093504 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:16:15.093731 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:16:15.093887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:15.094331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:16:15.094331 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:15.094331 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:16:15.094331 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:16:15.094331 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:15.094331 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:16:15.094331 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:15.094331 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:16:15.095115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:15.095115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:16:15.095115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:15.095115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:16:15.095115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:16:15.095115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:16:15.095115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:16:15.095115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:15.095115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:15.095115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:15.095115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:16:15.095115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:15.095360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:15.095360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:15.095360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:16:15.095360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:15.095360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:16:15.095360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:16:15.095360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:15.095360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:16:15.095360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:15.095360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:16:15.095360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:16:15.095360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:15.095360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:16:15.095620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:15.095620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:16:15.096057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:15.096057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:16:15.096057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:16:15.096057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:16:15.096057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:16:15.096057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:16:15.096057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:15.096057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:16:15.096057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:16:15.096057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:16:15.096057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:16:15.096057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:16:15.096057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:15.096057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:15.096057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:16:15.096057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:16:15.096451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:16:15.096451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:16:15.096451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:15.096451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:15.096451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:15.096451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:16:15.096451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:15.096451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:15.096451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:15.096451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:16:15.096451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:15.096451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:16:15.096451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:16:15.096451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:15.096451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:16:15.096451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:15.096451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:16:15.096451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:16:15.096451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:15.097170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:16:15.097170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:15.097170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:16:19.831323 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:16:19.831829 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:16:19.831952 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:16:19.833201 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:16:19.874158 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:16:19.874381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:19.875013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:16:19.875013 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:19.875013 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:16:19.875013 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:16:19.875013 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:19.875013 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:16:19.875013 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:19.875013 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:16:19.875867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:19.875867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:16:19.875867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:19.875867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:16:19.875867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:16:19.875867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:16:19.875867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:16:19.875867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:19.875867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:19.875867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:19.875867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:16:19.875867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:19.876121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:19.876121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:19.876121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:16:19.876121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:19.876121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:16:19.876121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:16:19.876121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:19.876121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:16:19.876121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:19.876121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:16:19.876121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:16:19.876121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:19.876121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:16:19.876371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:19.876371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:16:19.876690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:19.876690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:16:19.876690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:16:19.876690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:16:19.876690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:16:19.876690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:16:19.876690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:19.876690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:16:19.876690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:16:19.876690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:16:19.876690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:16:19.876690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:16:19.876690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:19.876690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:19.876690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:16:19.876690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:16:19.876995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:16:19.876995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:16:19.876995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:19.876995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:19.876995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:19.876995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:16:19.876995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:19.876995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:19.876995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:19.876995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:16:19.876995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:19.876995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:16:19.876995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:16:19.876995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:19.876995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:16:19.876995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:19.876995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:16:19.876995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:16:19.876995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:19.877339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:16:19.877339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:19.877339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:16:20.068356 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:16:20.068999 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:16:20.069176 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:16:20.070310 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:16:20.071626 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:16:20.071626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:20.071913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:16:20.071913 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:20.071913 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:16:20.071913 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:16:20.071913 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:20.071913 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:16:20.071913 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:20.071913 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:16:20.073490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:20.073490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:16:20.073490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:20.073490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:16:20.073490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:16:20.073490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:16:20.073490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:16:20.073490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:20.073490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:20.073490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:20.073490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:16:20.073490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:20.073879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:20.073879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:20.073879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:16:20.073879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:20.073879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:16:20.073879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:16:20.073879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:20.073879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:16:20.073879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:20.073879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:16:20.073879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:16:20.073879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:20.073879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:16:20.074246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:20.074246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:16:20.074798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:20.074798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:16:20.074798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:16:20.074798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:16:20.074798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:16:20.074798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:16:20.074798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:20.074798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:16:20.074798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:16:20.074798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:16:20.074798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:16:20.074798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:16:20.074798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:20.074798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:20.074798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:16:20.074798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:16:20.075275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:16:20.075275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:16:20.075275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:20.075275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:20.075275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:20.075275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:16:20.075275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:20.075275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:20.075275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:20.075275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:16:20.075275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:20.075275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:16:20.075275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:16:20.075275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:20.075275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:16:20.075275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:20.075275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:16:20.075275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:16:20.075275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:20.076305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:16:20.076305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:20.076305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:16:20.076305 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:16:20.076605 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:16:20.076718 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:16:20.076870 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:16:20.077195 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:16:20.077432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:20.077984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:16:20.077984 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:20.077984 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:16:20.077984 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:16:20.077984 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:20.077984 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:16:20.077984 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:20.077984 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:16:20.078781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:20.078781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:16:20.078781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:16:20.078781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:16:20.078781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:20.078781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:16:20.078781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:20.078781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:16:20.078781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:20.078781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:16:20.078781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:20.078781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:16:20.078781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:20.078781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:16:20.078781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:16:20.079230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:20.079230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:16:20.079230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:20.079230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:16:20.079230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:16:20.079230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:20.079230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:16:20.079230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:20.079230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:16:20.079704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:20.079704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:16:20.079704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:20.079704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:16:20.079704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:16:20.079704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:16:20.079704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:16:20.079704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:16:20.079704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:16:20.079704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:16:20.079704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:20.079704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:20.080108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:20.080108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:16:20.080108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:20.080108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:20.080108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:20.080108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:16:20.080108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:20.080108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:16:20.080108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:16:20.080108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:20.080108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:16:20.080108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:20.080108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:16:20.080108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:16:20.080526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:20.080526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:16:20.080526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:20.080526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:16:20.080653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:20.080653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:16:20.080653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:16:20.080653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:16:20.080653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:16:20.080653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:16:20.080653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:20.080653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:16:20.080653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:16:20.080653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:16:20.080653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:16:20.080653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:16:20.080653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:20.080653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:20.080653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:16:20.080653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:16:20.081134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:16:20.081134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:16:20.081134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:16:20.081134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:16:20.081134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:16:20.081134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:20.081134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:20.081134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:20.081134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:16:20.081134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:20.081134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:20.081134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:20.081134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:16:20.081134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:20.081134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:16:20.081134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:16:20.081134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:20.081134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:16:20.081965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:20.081965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:16:20.081965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:16:20.081965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:20.081965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:16:20.081965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:20.081965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:16:20.082388 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:16:20.085447 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:16:20.085447 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:16:20.085447 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:16:20.085447 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:16:20.085447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:20.085447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:16:20.085447 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:20.085447 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:16:20.085447 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:16:20.085802 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:20.085802 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:16:20.085802 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:20.085802 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:16:20.085802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:20.085802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:16:20.085802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:16:20.085802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:16:20.085802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:20.085802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:16:20.085802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:20.085802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:16:20.085802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:20.085802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:16:20.085802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:20.085802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:16:20.086252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:20.086252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:16:20.086252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:16:20.086252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:20.086252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:16:20.086252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:20.086252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:16:20.086252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:16:20.086252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:20.086252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:16:20.086252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:20.086252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:16:20.086252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:20.086252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:16:20.086680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:20.086680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:16:20.086680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:16:20.086680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:16:20.086680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:16:20.086680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:16:20.086680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:16:20.086680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:20.086680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:20.086680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:20.086680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:16:20.086680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:20.086680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:20.087040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:20.087040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:16:20.087040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:20.087040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:16:20.087040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:16:20.087040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:20.087040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:16:20.087040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:20.087040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:16:20.087040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:16:20.087040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:20.087040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:16:20.087040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:20.087387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:16:20.087387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:20.087387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:16:20.087387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:16:20.087387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:16:20.087387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:16:20.087387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:16:20.087387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:20.087387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:16:20.087387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:16:20.087387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:16:20.087387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:16:20.087387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:16:20.087387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:20.087387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:20.087387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:16:20.087387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:16:20.087924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:16:20.087924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:16:20.087924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:16:20.087924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:16:20.087924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:20.087924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:20.087924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:20.087924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:16:20.087924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:20.087924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:20.087924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:20.087924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:16:20.087924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:20.087924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:16:20.087924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:16:20.087924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:20.087924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:16:20.087924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:20.087924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:16:20.088470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:16:20.088470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:20.088470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:16:20.088470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:20.088470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:16:20.093034 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:16:20.093528 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:16:20.093816 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:16:20.094130 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:16:20.094543 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:16:20.094789 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:16:20.094859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:20.095419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:16:20.095419 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:20.095419 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:16:20.095419 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:16:20.095419 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:20.095419 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:16:20.095419 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:20.095419 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:16:20.096939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:20.096939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:16:20.096939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:20.096939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:16:20.096939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:16:20.096939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:16:20.096939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:16:20.096939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:20.096939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:20.096939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:20.096939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:16:20.096939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:20.097219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:20.097219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:20.097219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:16:20.097219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:20.097219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:16:20.097219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:16:20.097219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:20.097219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:16:20.097219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:20.097219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:16:20.097219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:16:20.097219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:20.097219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:16:20.097491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:20.097491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:16:20.099470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:20.099470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:16:20.099470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:16:20.099470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:16:20.099470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:16:20.099470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:16:20.099470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:20.099470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:16:20.099470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:16:20.099470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:16:20.099470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:16:20.099470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:16:20.099470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:20.099470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:20.099470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:16:20.099470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:16:20.099822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:16:20.099822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:16:20.099822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:20.099822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:20.099822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:20.099822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:16:20.099822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:20.099822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:20.099822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:20.099822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:16:20.099822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:20.099822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:16:20.099822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:16:20.099822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:20.099822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:16:20.099822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:20.099822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:16:20.099822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:16:20.099822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:20.100193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:16:20.100193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:20.100193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:16:24.831944 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:16:24.832492 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:16:24.832706 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:16:24.833975 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:16:24.840964 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG dbcounter [-] [105581] Writing DB stats neutron:SELECT=239 {{(pid=105581) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 14:16:24.869584 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:16:24.869584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:24.869584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:16:24.869584 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:24.869584 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:16:24.869584 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:16:24.869584 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:24.869584 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:16:24.869584 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:24.869584 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:16:24.870170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:24.870170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:16:24.870170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:24.870170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:16:24.870170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:16:24.870170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:16:24.870170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:16:24.870170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:24.870170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:24.870170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:24.870170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:16:24.870170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:24.870609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:24.870609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:24.870609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:16:24.870609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:24.870609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:16:24.870609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:16:24.870609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:24.870609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:16:24.870609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:24.870609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:16:24.870609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:16:24.870609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:24.870609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:16:24.870944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:24.870944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:16:24.870944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:24.870944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:16:24.870944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:16:24.870944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:16:24.870944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:16:24.870944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:16:24.870944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:24.870944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:16:24.870944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:16:24.870944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:16:24.870944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:16:24.870944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:16:24.870944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:24.871367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:24.871367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:16:24.871367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:16:24.871367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:16:24.871367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:16:24.871367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:24.871367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:24.871367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:24.871367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:16:24.871367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:24.871367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:24.871367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:24.871367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:16:24.871367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:24.871367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:16:24.871367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:16:24.871367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:24.871367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:16:24.871367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:24.871882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:16:24.871882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:16:24.871882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:24.871882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:16:24.871882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:24.871882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:16:25.069105 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:16:25.069720 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:16:25.069818 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:16:25.070707 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:16:25.071110 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:16:25.071400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:25.072342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:16:25.072342 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:25.072342 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:16:25.072342 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:16:25.072342 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:25.072342 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:16:25.072342 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:25.072342 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:16:25.073834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:25.073834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:16:25.073834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:25.073834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:16:25.073834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:16:25.073834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:16:25.073834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:16:25.073834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:25.073834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:25.073834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:25.073834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:16:25.073834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:25.074135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:25.074135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:25.074135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:16:25.074135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:25.074135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:16:25.074135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:16:25.074135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:25.074135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:16:25.074135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:25.074135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:16:25.074135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:16:25.074135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:25.074135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:16:25.074415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:25.074415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:16:25.075600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:25.075600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:16:25.075600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:16:25.075600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:16:25.075600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:16:25.075600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:16:25.075600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:25.075600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:16:25.075600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:16:25.075600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:16:25.075600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:16:25.075600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:16:25.075600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:25.075600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:25.075600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:16:25.075600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:16:25.075969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:16:25.075969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:16:25.075969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:25.075969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:25.075969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:25.075969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:16:25.075969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:25.075969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:25.075969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:25.075969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:16:25.075969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:25.075969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:16:25.075969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:16:25.075969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:25.075969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:16:25.075969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:25.075969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:16:25.075969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:16:25.075969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:25.076340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:16:25.076340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:25.076340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:16:25.076639 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:16:25.077042 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:16:25.077223 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:16:25.077484 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:16:25.077842 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:16:25.078055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:25.078675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:16:25.078675 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:25.078675 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:16:25.078675 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:16:25.078675 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:25.078675 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:16:25.078675 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:25.078675 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:16:25.079926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:25.079926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:16:25.079926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:16:25.079926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:16:25.079926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:25.079926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:16:25.079926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:25.079926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:16:25.079926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:25.079926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:16:25.079926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:25.079926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:16:25.079926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:25.079926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:16:25.079926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:16:25.080390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:25.080390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:16:25.080390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:25.080390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:16:25.080390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:16:25.080390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:25.080390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:16:25.080390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:25.080390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:16:25.081200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:25.081200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:16:25.081200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:25.081200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:16:25.081200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:16:25.081200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:16:25.081200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:16:25.081200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:16:25.081200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:16:25.081200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:16:25.081200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:25.081200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:25.081666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:25.081666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:16:25.081666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:25.081666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:25.081666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:25.081666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:16:25.081666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:25.081666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:16:25.081666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:16:25.081666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:25.081666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:16:25.081666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:25.081666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:16:25.081666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:16:25.082087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:25.082087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:16:25.082087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:25.082087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:16:25.082496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:25.082496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:16:25.082496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:16:25.082496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:16:25.082496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:16:25.082496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:16:25.082496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:25.082496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:16:25.082496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:16:25.082496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:16:25.082496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:16:25.082496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:16:25.082496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:25.082496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:25.082496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:16:25.082496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:16:25.082961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:16:25.082961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:16:25.082961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:16:25.082961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:16:25.082961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:16:25.082961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:25.082961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:25.082961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:25.082961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:16:25.082961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:25.082961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:25.082961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:25.082961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:16:25.082961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:25.082961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:16:25.082961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:16:25.082961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:25.082961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:16:25.083443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:25.083443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:16:25.083443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:16:25.083443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:25.083443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:16:25.083443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:25.083443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:16:25.083443 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:16:25.083443 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:16:25.083443 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:16:25.083751 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:16:25.083751 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:16:25.083852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:25.084340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:16:25.084340 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:25.084340 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:16:25.084340 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:16:25.084340 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:25.084340 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:16:25.084340 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:25.084340 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:16:25.086909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:25.086909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:16:25.086909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:16:25.086909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:16:25.086909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:25.086909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:16:25.086909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:25.086909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:16:25.086909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:25.086909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:16:25.086909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:25.086909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:16:25.086909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:25.086909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:16:25.086909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:16:25.087364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:25.087364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:16:25.087364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:25.087364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:16:25.087364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:16:25.087364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:25.087364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:16:25.087364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:25.087364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:16:25.087364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:25.087364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:16:25.087364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:25.087364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:16:25.087364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:16:25.087751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:16:25.087751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:16:25.087751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:16:25.087751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:16:25.087751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:25.087751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:25.087751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:25.087751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:16:25.087751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:25.087751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:25.087751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:25.087751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:16:25.087751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:25.088104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:16:25.088104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:16:25.088104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:25.088104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:16:25.088104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:25.088104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:16:25.088104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:16:25.088104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:25.088104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:16:25.088104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:25.088104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:16:25.088104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:25.088461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:16:25.088461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:16:25.088461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:16:25.088461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:16:25.088461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:16:25.088461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:25.088461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:16:25.088461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:16:25.088461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:16:25.088461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:16:25.088461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:16:25.088461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:25.088461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:25.088461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:16:25.088461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:16:25.088461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:16:25.088461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:16:25.088461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:16:25.088461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:16:25.089021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:25.089021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:25.089021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:25.089021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:16:25.089021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:25.089021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:25.089021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:25.089021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:16:25.089021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:25.089021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:16:25.089021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:16:25.089021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:25.089021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:16:25.089021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:25.089021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:16:25.089021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:16:25.089021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:25.089021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:16:25.089021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:25.089021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:16:25.093984 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:16:25.094373 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:16:25.094544 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:16:25.094704 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:16:25.095249 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:16:25.095616 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:16:25.095850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:25.096658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:16:25.096658 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:25.096658 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:16:25.096658 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:16:25.096658 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:25.096658 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:16:25.096658 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:25.096658 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:16:25.097664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:25.097664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:16:25.097664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:25.097664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:16:25.097664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:16:25.097664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:16:25.097664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:16:25.097664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:25.097664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:25.097664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:25.097664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:16:25.097664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:25.098004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:25.098004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:25.098004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:16:25.098004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:25.098004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:16:25.098004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:16:25.098004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:25.098004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:16:25.098004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:25.098004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:16:25.098004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:16:25.098004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:25.098004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:16:25.098310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:25.098310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:16:25.098759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:25.098759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:16:25.098759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:16:25.098759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:16:25.098759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:16:25.098759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:16:25.098759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:25.098759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:16:25.098759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:16:25.098759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:16:25.098759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:16:25.098759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:16:25.098759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:25.098759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:25.098759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:16:25.098759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:16:25.099180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:16:25.099180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:16:25.099180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:25.099180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:25.099180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:25.099180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:16:25.099180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:25.099180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:25.099180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:25.099180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:16:25.099180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:25.099180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:16:25.099180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:16:25.099180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:25.099180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:16:25.099180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:25.099180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:16:25.099180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:16:25.099180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:25.099618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:16:25.099618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:25.099618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:16:29.832454 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:16:29.833044 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:16:29.833233 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:16:29.834544 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:16:29.867832 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:16:29.868001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:29.868659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:16:29.868659 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:29.868659 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:16:29.868659 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:16:29.868659 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:29.868659 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:16:29.868659 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:29.868659 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:16:29.869492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:29.869492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:16:29.869492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:29.869492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:16:29.869492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:16:29.869492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:16:29.869492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:16:29.869492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:29.869492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:29.869492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:29.869492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:16:29.869492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:29.869922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:29.869922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:29.869922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:16:29.869922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:29.869922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:16:29.869922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:16:29.869922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:29.869922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:16:29.869922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:29.869922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:16:29.869922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:16:29.869922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:29.869922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:16:29.870367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:29.870367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:16:29.870367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:29.870367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:16:29.870367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:16:29.870367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:16:29.870367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:16:29.870367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:16:29.870367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:29.870367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:16:29.870367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:16:29.870367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:16:29.870367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:16:29.870367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:16:29.870367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:29.870875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:29.870875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:16:29.870875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:16:29.870875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:16:29.870875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:16:29.870875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:29.870875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:29.870875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:29.870875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:16:29.870875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:29.870875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:29.870875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:29.870875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:16:29.870875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:29.870875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:16:29.870875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:16:29.870875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:29.870875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:16:29.870875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:29.871450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:16:29.871450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:16:29.871450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:29.871450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:16:29.871450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:29.871450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:16:30.069800 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:16:30.070225 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:16:30.070295 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:16:30.071219 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:16:30.071537 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:16:30.071793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:30.072785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:16:30.072785 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:30.072785 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:16:30.072785 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:16:30.072785 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:30.072785 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:16:30.072785 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:30.072785 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:16:30.073535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:30.073535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:16:30.073535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:30.073535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:16:30.073535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:16:30.073535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:16:30.073535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:16:30.073535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:30.073535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:30.073535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:30.073535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:16:30.073535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:30.073913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:30.073913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:30.073913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:16:30.073913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:30.073913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:16:30.073913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:16:30.073913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:30.073913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:16:30.073913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:30.073913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:16:30.073913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:16:30.073913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:30.073913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:16:30.074295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:30.074295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:16:30.074295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:30.074295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:16:30.074295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:16:30.074295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:16:30.074295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:16:30.074295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:16:30.074295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:30.074295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:16:30.074295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:16:30.074295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:16:30.074295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:16:30.074295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:16:30.074295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:30.075281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:30.075281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:16:30.075281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:16:30.075281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:16:30.075281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:16:30.075281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:30.075281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:30.075281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:30.075281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:16:30.075281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:30.075281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:30.075281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:30.075281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:16:30.075281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:30.075281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:16:30.075281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:16:30.075281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:30.075281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:16:30.075281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:30.075909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:16:30.075909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:16:30.075909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:30.075909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:16:30.075909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:30.075909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:16:30.077648 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:16:30.077997 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:16:30.078072 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:16:30.078155 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:16:30.078710 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:16:30.079014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:30.079734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:16:30.079734 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:30.079734 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:16:30.079734 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:16:30.079734 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:30.079734 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:16:30.079734 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:30.079734 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:16:30.080265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:30.080265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:16:30.080265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:16:30.080265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:16:30.080265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:30.080265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:16:30.080265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:30.080265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:16:30.080265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:30.080265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:16:30.080265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:30.080265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:16:30.080265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:30.080265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:16:30.080265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:16:30.080619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:30.080619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:16:30.080619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:30.080619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:16:30.080619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:16:30.080619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:30.080619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:16:30.080619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:30.080619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:16:30.080884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:30.080884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:16:30.080884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:30.080884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:16:30.080884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:16:30.080884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:16:30.080884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:16:30.080884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:16:30.080884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:16:30.080884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:16:30.080884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:30.080884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:30.081325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:30.081325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:16:30.081325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:30.081325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:30.081325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:30.081325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:16:30.081325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:30.081325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:16:30.081325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:16:30.081325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:30.081325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:16:30.081325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:30.081325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:16:30.081325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:16:30.081818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:30.081818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:16:30.081818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:30.081818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:16:30.081818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:30.081818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:16:30.081818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:16:30.081818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:16:30.081818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:16:30.081818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:16:30.081818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:30.081818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:16:30.081818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:16:30.081818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:16:30.082293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:16:30.082293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:16:30.082293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:30.082293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:30.082293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:16:30.082293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:16:30.082293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:16:30.082293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:16:30.082293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:16:30.082293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:16:30.082293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:16:30.082293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:30.082293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:30.082293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:30.082293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:16:30.082293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:30.082293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:30.082293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:30.082293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:16:30.082766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:30.082766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:16:30.082766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:16:30.082766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:30.082766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:16:30.082766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:30.082766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:16:30.082766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:16:30.082766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:30.082766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:16:30.082766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:30.082766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:16:30.083167 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:16:30.083645 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:16:30.083957 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:16:30.084148 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:16:30.084497 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:16:30.084799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:30.085390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:16:30.085390 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:30.085390 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:16:30.085390 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:16:30.085390 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:30.085390 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:16:30.085390 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:30.085390 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:16:30.085939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:30.085939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:16:30.085939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:16:30.085939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:16:30.085939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:30.085939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:16:30.085939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:30.085939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:16:30.085939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:30.085939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:16:30.085939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:30.085939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:16:30.085939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:30.085939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:16:30.085939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:16:30.086440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:30.086440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:16:30.086440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:30.086440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:16:30.086440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:16:30.086440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:30.086440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:16:30.086440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:30.086440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:16:30.087181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:30.087181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:16:30.087181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:30.087181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:16:30.087181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:16:30.087181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:16:30.087181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:16:30.087181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:16:30.087181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:16:30.087181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:30.087181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:30.087553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:30.087553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:16:30.087553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:30.087553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:30.087553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:30.087553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:16:30.087553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:30.087553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:16:30.087553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:16:30.087553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:30.087553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:16:30.087553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:30.087553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:16:30.087553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:16:30.087968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:30.087968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:16:30.087968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:30.087968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:16:30.087968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:30.087968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:16:30.087968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:16:30.087968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:16:30.087968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:16:30.087968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:16:30.087968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:30.087968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:16:30.087968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:16:30.087968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:16:30.088443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:16:30.088443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:16:30.088443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:30.088443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:30.088443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:16:30.088443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:16:30.088443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:16:30.088443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:16:30.088443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:16:30.088443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:16:30.088443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:30.088443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:30.088443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:30.088443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:16:30.088443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:30.088443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:30.088443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:30.088443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:16:30.088443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:30.089017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:16:30.089017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:16:30.089017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:30.089017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:16:30.089017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:30.089017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:16:30.089017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:16:30.089017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:30.089017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:16:30.089017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:30.089017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:16:30.094453 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:16:30.095109 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:16:30.095318 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:16:30.095649 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:16:30.096873 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:16:30.097177 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:16:30.097335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:30.098008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:16:30.098008 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:30.098008 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:16:30.098008 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:16:30.098008 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:30.098008 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:16:30.098008 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:30.098008 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:16:30.099518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:30.099518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:16:30.099518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:30.099518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:16:30.099518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:16:30.099518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:16:30.099518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:16:30.099518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:30.099518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:30.099518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:30.099518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:16:30.099518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:30.099896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:30.099896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:30.099896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:16:30.099896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:30.099896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:16:30.099896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:16:30.099896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:30.099896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:16:30.099896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:30.099896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:16:30.099896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:16:30.099896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:30.099896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:16:30.100271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:30.100271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:16:30.101560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:30.101560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:16:30.101560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:16:30.101560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:16:30.101560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:16:30.101560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:16:30.101560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:30.101560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:16:30.101560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:16:30.101560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:16:30.101560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:16:30.101560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:16:30.101560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:30.101560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:30.101560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:16:30.101560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:16:30.102003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:16:30.102003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:16:30.102003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:30.102003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:30.102003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:30.102003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:16:30.102003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:30.102003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:30.102003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:30.102003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:16:30.102003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:30.102003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:16:30.102003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:16:30.102003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:30.102003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:16:30.102003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:30.102003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:16:30.102003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:16:30.102003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:30.102582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:16:30.102582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:30.102582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:16:34.833039 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:16:34.833567 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:16:34.833802 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:16:34.835039 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:16:34.872678 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:16:34.873709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:34.873709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:16:34.873709 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:34.873709 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:16:34.873709 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:16:34.873709 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:34.873709 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:16:34.873709 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:34.873709 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:16:34.874494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:34.874494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:16:34.874494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:34.874494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:16:34.874494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:16:34.874494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:16:34.874494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:16:34.874494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:34.874494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:34.874494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:34.874494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:16:34.874494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:34.874978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:34.874978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:34.874978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:16:34.874978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:34.874978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:16:34.874978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:16:34.874978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:34.874978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:16:34.874978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:34.874978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:16:34.874978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:16:34.874978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:34.874978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:16:34.875389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:34.875389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:16:34.878441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:34.878441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:16:34.878441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:16:34.878441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:16:34.878441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:16:34.878441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:16:34.878441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:34.878441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:16:34.878441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:16:34.878441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:16:34.878441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:16:34.878441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:16:34.878441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:34.878441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:34.878441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:16:34.878441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:16:34.879012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:16:34.879012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:16:34.879012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:34.879012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:34.879012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:34.879012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:16:34.879012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:34.879012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:34.879012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:34.879012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:16:34.879012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:34.879012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:16:34.879012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:16:34.879012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:34.879012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:16:34.879012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:34.879012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:16:34.879012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:16:34.879012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:34.879628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:16:34.879628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:34.879628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:16:35.071082 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:16:35.071082 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:16:35.071433 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:16:35.072118 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:16:35.072316 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:16:35.072436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:35.073689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:16:35.073689 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:35.073689 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:16:35.073689 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:16:35.073689 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:35.073689 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:16:35.073689 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:35.073689 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:16:35.074330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:35.074330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:16:35.074330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:35.074330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:16:35.074330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:16:35.074330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:16:35.074330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:16:35.074330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:35.074330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:35.074330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:35.074330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:16:35.074330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:35.074877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:35.074877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:35.074877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:16:35.074877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:35.074877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:16:35.074877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:16:35.074877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:35.074877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:16:35.074877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:35.074877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:16:35.074877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:16:35.074877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:35.074877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:16:35.075350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:35.075350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:16:35.075350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:35.075350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:16:35.075350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:16:35.075350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:16:35.075350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:16:35.075350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:16:35.075350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:35.075350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:16:35.075350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:16:35.075350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:16:35.075350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:16:35.075350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:16:35.075350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:35.076015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:35.076015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:16:35.076015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:16:35.076015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:16:35.076015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:16:35.076015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:35.076015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:35.076015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:35.076015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:16:35.076015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:35.076015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:35.076015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:35.076015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:16:35.076015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:35.076015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:16:35.076015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:16:35.076015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:35.076015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:16:35.076015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:35.076754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:16:35.076754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:16:35.076754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:35.076754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:16:35.076754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:35.076754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:16:35.078121 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:16:35.079327 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:16:35.079327 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:16:35.079327 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:16:35.079327 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:16:35.079327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:35.079327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:16:35.079327 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:35.079327 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:16:35.079327 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:16:35.079772 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:35.079772 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:16:35.079772 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:35.079772 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:16:35.079772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:35.079772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:16:35.079772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:16:35.079772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:16:35.079772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:35.079772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:16:35.079772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:35.079772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:16:35.079772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:35.079772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:16:35.079772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:35.079772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:16:35.080276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:35.080276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:16:35.080276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:16:35.080276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:35.080276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:16:35.080276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:35.080276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:16:35.080276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:16:35.080276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:35.080276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:16:35.080276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:35.080276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:16:35.080276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:35.080276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:16:35.080689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:35.080689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:16:35.080689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:16:35.080689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:16:35.080689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:16:35.080689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:16:35.080689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:16:35.080689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:16:35.080689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:35.080689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:35.080689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:35.080689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:16:35.080689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:35.081036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:35.081036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:35.081036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:16:35.081036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:35.081036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:16:35.081036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:16:35.081036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:35.081036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:16:35.081036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:35.081036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:16:35.081036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:16:35.081036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:35.081036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:16:35.081358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:35.081358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:16:35.081358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:35.081358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:16:35.081358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:16:35.081358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:16:35.081358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:16:35.081358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:16:35.081358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:35.081358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:16:35.081358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:16:35.081358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:16:35.081358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:16:35.081358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:16:35.081358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:35.081771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:35.081771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:16:35.081771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:16:35.081771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:16:35.081771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:16:35.081771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:16:35.081771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:16:35.081771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:16:35.081771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:35.081771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:35.081771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:35.081771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:16:35.081771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:35.081771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:35.081771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:35.081771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:16:35.081771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:35.081771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:16:35.081771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:16:35.082324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:35.082324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:16:35.082324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:35.082324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:16:35.082324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:16:35.082324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:35.082324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:16:35.082324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:35.082324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:16:35.085258 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:16:35.085462 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:16:35.085573 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:16:35.085694 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:16:35.085950 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:16:35.086228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:35.086579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:16:35.086579 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:35.086579 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:16:35.086579 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:16:35.086579 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:35.086579 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:16:35.086579 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:35.086579 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:16:35.087350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:35.087350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:16:35.087350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:16:35.087350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:16:35.087350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:35.087350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:16:35.087350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:35.087350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:16:35.087350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:35.087350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:16:35.087350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:35.087350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:16:35.087350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:35.087350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:16:35.087350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:16:35.088148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:35.088148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:16:35.088148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:35.088148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:16:35.088148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:16:35.088148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:35.088148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:16:35.088148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:35.088148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:16:35.088148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:35.088148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:16:35.088148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:35.088148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:16:35.088148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:16:35.088812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:16:35.088812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:16:35.088812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:16:35.088812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:16:35.088812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:35.088812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:35.088812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:35.088812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:16:35.088812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:35.088812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:35.088812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:35.088812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:16:35.088812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:35.089426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:16:35.089426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:16:35.089426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:35.089426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:16:35.089426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:35.089426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:16:35.089426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:16:35.089426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:35.089426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:16:35.089426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:35.089426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:16:35.089426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:35.089974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:16:35.089974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:16:35.089974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:16:35.089974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:16:35.089974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:16:35.089974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:35.089974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:16:35.089974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:16:35.089974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:16:35.089974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:16:35.089974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:16:35.089974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:35.089974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:35.089974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:16:35.089974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:16:35.089974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:16:35.089974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:16:35.089974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:16:35.089974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:16:35.092444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:35.092444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:35.092444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:35.092444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:16:35.092444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:35.092444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:35.092444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:35.092444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:16:35.092444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:35.092444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:16:35.092444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:16:35.092444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:35.092444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:16:35.092444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:35.092444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:16:35.092444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:16:35.092444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:35.092444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:16:35.092444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:35.092444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:16:35.094888 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:16:35.095098 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:16:35.095204 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:16:35.095333 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:16:35.095606 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:16:35.095816 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:16:35.095950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:35.096383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:16:35.096383 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:35.096383 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:16:35.096383 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:16:35.096383 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:35.096383 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:16:35.096383 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:35.096383 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:16:35.097167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:35.097167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:16:35.097167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:35.097167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:16:35.097167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:16:35.097167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:16:35.097167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:16:35.097167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:35.097167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:35.097167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:35.097167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:16:35.097167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:35.097462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:35.097462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:35.097462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:16:35.097462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:35.097462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:16:35.097462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:16:35.097462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:35.097462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:16:35.097462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:35.097462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:16:35.097462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:16:35.097462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:35.097462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:16:35.097718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:35.097718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:16:35.098199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:35.098199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:16:35.098199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:16:35.098199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:16:35.098199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:16:35.098199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:16:35.098199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:35.098199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:16:35.098199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:16:35.098199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:16:35.098199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:16:35.098199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:16:35.098199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:35.098199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:35.098199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:16:35.098199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:16:35.098578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:16:35.098578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:16:35.098578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:35.098578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:35.098578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:35.098578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:16:35.098578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:35.098578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:35.098578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:35.098578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:16:35.098578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:35.098578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:16:35.098578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:16:35.098578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:35.098578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:16:35.098578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:35.098578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:16:35.098578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:16:35.098578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:35.098953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:16:35.098953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:35.098953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:16:39.834218 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:16:39.835150 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:16:39.835150 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:16:39.835660 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:16:39.873801 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:16:39.874101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:39.874706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:16:39.874706 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:39.874706 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:16:39.874706 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:16:39.874706 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:39.874706 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:16:39.874706 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:39.874706 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:16:39.875958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:39.875958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:16:39.875958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:39.875958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:16:39.875958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:16:39.875958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:16:39.875958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:16:39.875958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:39.875958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:39.875958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:39.875958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:16:39.875958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:39.876418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:39.876418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:39.876418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:16:39.876418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:39.876418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:16:39.876418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:16:39.876418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:39.876418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:16:39.876418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:39.876418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:16:39.876418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:16:39.876418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:39.876418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:16:39.876852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:39.876852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:16:39.877240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:39.877240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:16:39.877240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:16:39.877240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:16:39.877240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:16:39.877240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:16:39.877240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:39.877240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:16:39.877240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:16:39.877240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:16:39.877240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:16:39.877240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:16:39.877240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:39.877240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:39.877240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:16:39.877240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:16:39.877820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:16:39.877820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:16:39.877820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:39.877820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:39.877820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:39.877820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:16:39.877820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:39.877820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:39.877820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:39.877820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:16:39.877820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:39.877820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:16:39.877820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:16:39.877820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:39.877820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:16:39.877820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:39.877820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:16:39.877820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:16:39.877820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:39.878534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:16:39.878534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:39.878534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:16:40.071115 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:16:40.071655 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:16:40.071843 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:16:40.072713 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:16:40.073016 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:16:40.073259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:40.074054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:16:40.074054 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:40.074054 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:16:40.074054 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:16:40.074054 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:40.074054 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:16:40.074054 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:40.074054 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:16:40.075491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:40.075491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:16:40.075491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:40.075491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:16:40.075491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:16:40.075491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:16:40.075491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:16:40.075491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:40.075491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:40.075491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:40.075491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:16:40.075491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:40.076097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:40.076097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:40.076097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:16:40.076097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:40.076097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:16:40.076097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:16:40.076097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:40.076097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:16:40.076097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:40.076097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:16:40.076097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:16:40.076097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:40.076097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:16:40.076683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:40.076683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:16:40.077273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:40.077273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:16:40.077273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:16:40.077273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:16:40.077273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:16:40.077273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:16:40.077273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:40.077273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:16:40.077273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:16:40.077273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:16:40.077273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:16:40.077273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:16:40.077273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:40.077273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:40.077273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:16:40.077273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:16:40.078047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:16:40.078047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:16:40.078047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:40.078047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:40.078047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:40.078047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:16:40.078047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:40.078047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:40.078047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:40.078047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:16:40.078047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:40.078047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:16:40.078047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:16:40.078047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:40.078047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:16:40.078047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:40.078047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:16:40.078047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:16:40.078047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:40.078887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:16:40.078887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:40.078887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:16:40.078887 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:16:40.078887 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:16:40.078887 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:16:40.078887 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:16:40.079277 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:16:40.079277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:40.079685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:16:40.079685 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:40.079685 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:16:40.079685 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:16:40.079685 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:40.079685 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:16:40.079685 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:40.079685 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:16:40.080354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:40.080354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:16:40.080354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:16:40.080354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:16:40.080354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:40.080354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:16:40.080354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:40.080354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:16:40.080354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:40.080354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:16:40.080354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:40.080354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:16:40.080354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:40.080354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:16:40.080354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:16:40.081110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:40.081110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:16:40.081110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:40.081110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:16:40.081110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:16:40.081110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:40.081110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:16:40.081110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:40.081110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:16:40.081357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:40.081357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:16:40.081357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:40.081357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:16:40.081357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:16:40.081357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:16:40.081357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:16:40.081357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:16:40.081357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:16:40.081357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:16:40.081357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:40.081357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:40.081728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:40.081728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:16:40.081728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:40.081728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:40.081728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:40.081728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:16:40.081728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:40.081728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:16:40.081728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:16:40.081728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:40.081728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:16:40.081728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:40.081728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:16:40.081728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:16:40.082117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:40.082117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:16:40.082117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:40.082117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:16:40.082233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:40.082233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:16:40.082233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:16:40.082233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:16:40.082233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:16:40.082233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:16:40.082233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:40.082233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:16:40.082233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:16:40.082233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:16:40.082233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:16:40.082233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:16:40.082233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:40.082233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:40.082233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:16:40.082233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:16:40.082675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:16:40.082675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:16:40.082675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:16:40.082675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:16:40.082675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:16:40.082675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:40.082675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:40.082675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:40.082675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:16:40.082675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:40.082675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:40.082675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:40.082675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:16:40.082675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:40.082675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:16:40.082675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:16:40.082675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:40.082675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:16:40.083165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:40.083165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:16:40.083165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:16:40.083165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:40.083165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:16:40.083165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:40.083165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:16:40.085517 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:16:40.085665 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:16:40.085773 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:16:40.085894 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:16:40.086112 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:16:40.086333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:40.086814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:16:40.086814 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:40.086814 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:16:40.086814 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:16:40.086814 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:40.086814 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:16:40.086814 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:40.086814 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:16:40.087539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:40.087539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:16:40.087539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:16:40.087539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:16:40.087539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:40.087539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:16:40.087539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:40.087539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:16:40.087539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:40.087539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:16:40.087539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:40.087539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:16:40.087539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:40.087539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:16:40.087539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:16:40.087940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:40.087940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:16:40.087940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:40.087940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:16:40.087940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:16:40.087940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:40.087940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:16:40.087940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:40.087940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:16:40.088382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:40.088382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:16:40.088382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:40.088382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:16:40.088382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:16:40.088382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:16:40.088382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:16:40.088382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:16:40.088382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:16:40.088382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:40.088382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:40.088761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:40.088761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:16:40.088761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:40.088761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:40.088761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:40.088761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:16:40.088761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:40.088761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:16:40.088761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:16:40.088761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:40.088761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:16:40.088761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:40.088761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:16:40.088761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:16:40.089131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:40.089131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:16:40.089131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:40.089131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:16:40.089507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:40.089507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:16:40.089507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:16:40.089507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:16:40.089507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:16:40.089507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:16:40.089507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:40.089507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:16:40.089507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:16:40.089507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:16:40.089507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:16:40.089507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:16:40.089507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:40.089507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:40.089507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:16:40.089507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:16:40.089507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:16:40.090042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:16:40.090042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:16:40.090042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:16:40.090042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:40.090042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:40.090042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:40.090042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:16:40.090042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:40.090042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:40.090042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:40.090042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:16:40.090042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:40.090042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:16:40.090042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:16:40.090042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:40.090042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:16:40.090042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:40.090042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:16:40.090042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:16:40.090658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:40.090658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:16:40.090658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:40.090658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:16:40.095154 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:16:40.095347 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:16:40.095474 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:16:40.095618 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:16:40.095854 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:16:40.096059 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:16:40.096197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:40.096696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:16:40.096696 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:40.096696 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:16:40.096696 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:16:40.096696 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:40.096696 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:16:40.096696 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:40.096696 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:16:40.097579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:40.097579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:16:40.097579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:40.097579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:16:40.097579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:16:40.097579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:16:40.097579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:16:40.097579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:40.097579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:40.097579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:40.097579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:16:40.097579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:40.097947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:40.097947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:40.097947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:16:40.097947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:40.097947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:16:40.097947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:16:40.097947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:40.097947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:16:40.097947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:40.097947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:16:40.097947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:16:40.097947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:40.097947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:16:40.098295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:40.098295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:16:40.098510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:40.098510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:16:40.098510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:16:40.098510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:16:40.098510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:16:40.098510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:16:40.098510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:40.098510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:16:40.098510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:16:40.098510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:16:40.098510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:16:40.098510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:16:40.098510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:40.098510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:40.098510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:16:40.098510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:16:40.098950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:16:40.098950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:16:40.098950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:40.098950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:40.098950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:40.098950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:16:40.098950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:40.098950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:40.098950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:40.098950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:16:40.098950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:40.098950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:16:40.098950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:16:40.098950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:40.098950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:16:40.098950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:40.098950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:16:40.098950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:16:40.098950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:40.099432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:16:40.099432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:40.099432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:16:44.836042 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:16:44.836607 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:16:44.836607 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:16:44.837003 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:16:44.877533 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:16:44.877757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:44.878444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:16:44.878444 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:44.878444 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:16:44.878444 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:16:44.878444 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:44.878444 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:16:44.878444 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:44.878444 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:16:44.879549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:44.879549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:16:44.879549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:44.879549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:16:44.879549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:16:44.879549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:16:44.879549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:16:44.879549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:44.879549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:44.879549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:44.879549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:16:44.879549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:44.879876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:44.879876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:44.879876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:16:44.879876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:44.879876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:16:44.879876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:16:44.879876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:44.879876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:16:44.879876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:44.879876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:16:44.879876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:16:44.879876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:44.879876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:16:44.880206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:44.880206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:16:44.880735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:44.880735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:16:44.880735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:16:44.880735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:16:44.880735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:16:44.880735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:16:44.880735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:44.880735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:16:44.880735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:16:44.880735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:16:44.880735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:16:44.880735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:16:44.880735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:44.880735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:44.880735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:16:44.880735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:16:44.881139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:16:44.881139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:16:44.881139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:44.881139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:44.881139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:44.881139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:16:44.881139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:44.881139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:44.881139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:44.881139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:16:44.881139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:44.881139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:16:44.881139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:16:44.881139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:44.881139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:16:44.881139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:44.881139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:16:44.881139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:16:44.881139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:44.881609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:16:44.881609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:44.881609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:16:45.071805 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:16:45.072437 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:16:45.072651 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:16:45.073645 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:16:45.073844 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:16:45.073969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:45.074655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:16:45.074655 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:45.074655 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:16:45.074655 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:16:45.074655 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:45.074655 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:16:45.074655 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:45.074655 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:16:45.076200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:45.076200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:16:45.076200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:45.076200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:16:45.076200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:16:45.076200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:16:45.076200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:16:45.076200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:45.076200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:45.076200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:45.076200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:16:45.076200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:45.077166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:45.077166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:45.077166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:16:45.077166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:45.077166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:16:45.077166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:16:45.077166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:45.077166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:16:45.077166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:45.077166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:16:45.077166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:16:45.077166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:45.077166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:16:45.078002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:45.078002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:16:45.078002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:45.078002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:16:45.078002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:16:45.078002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:16:45.078002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:16:45.078002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:16:45.078002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:45.078002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:16:45.078002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:16:45.078002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:16:45.078002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:16:45.078002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:16:45.078002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:45.078796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:45.078796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:16:45.078796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:16:45.078796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:16:45.078796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:16:45.078796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:45.078796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:45.078796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:45.078796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:16:45.078796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:45.078796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:45.078796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:45.078796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:16:45.078796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:45.078796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:16:45.078796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:16:45.078796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:45.078796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:16:45.078796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:45.079769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:16:45.079769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:16:45.079769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:45.079769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:16:45.079769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:45.079769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:16:45.079769 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:16:45.079769 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:16:45.079769 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:16:45.079769 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:16:45.080277 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:16:45.080277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:45.080788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:16:45.080788 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:45.080788 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:16:45.080788 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:16:45.080788 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:45.080788 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:16:45.080788 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:45.080788 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:16:45.082248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:45.082248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:16:45.082248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:16:45.082248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:16:45.082248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:45.082248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:16:45.082248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:45.082248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:16:45.082248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:45.082248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:16:45.082248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:45.082248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:16:45.082248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:45.082248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:16:45.082248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:16:45.082720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:45.082720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:16:45.082720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:45.082720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:16:45.082720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:16:45.082720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:45.082720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:16:45.082720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:45.082720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:16:45.083816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:45.083816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:16:45.083816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:45.083816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:16:45.083816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:16:45.083816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:16:45.083816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:16:45.083816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:16:45.083816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:16:45.083816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:16:45.083816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:45.083816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:45.084191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:45.084191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:16:45.084191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:45.084191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:45.084191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:45.084191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:16:45.084191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:45.084191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:16:45.084191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:16:45.084191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:45.084191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:16:45.084191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:45.084191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:16:45.084191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:16:45.084669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:45.084669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:16:45.084669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:45.084669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:16:45.085653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:45.085653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:16:45.085653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:16:45.085653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:16:45.085653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:16:45.085653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:16:45.085653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:45.085653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:16:45.085653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:16:45.085653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:16:45.085653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:16:45.085653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:16:45.085653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:45.085653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:45.085653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:16:45.085653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:16:45.086143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:16:45.086143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:16:45.086143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:16:45.086143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:16:45.086143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:16:45.086143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:45.086143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:45.086143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:45.086143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:16:45.086143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:45.086143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:45.086143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:45.086143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:16:45.086143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:45.086143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:16:45.086143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:16:45.086143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:45.086143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:16:45.086653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:45.086653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:16:45.086653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:16:45.086653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:45.086653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:16:45.086653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:45.086653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:16:45.086653 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:16:45.086653 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:16:45.086653 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:16:45.086938 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:16:45.087123 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:16:45.087290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:45.087887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:16:45.087887 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:45.087887 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:16:45.087887 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:16:45.087887 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:45.087887 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:16:45.087887 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:45.087887 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:16:45.089232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:45.089232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:16:45.089232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:16:45.089232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:16:45.089232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:45.089232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:16:45.089232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:45.089232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:16:45.089232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:45.089232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:16:45.089232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:45.089232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:16:45.089232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:45.089232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:16:45.089232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:16:45.089694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:45.089694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:16:45.089694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:45.089694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:16:45.089694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:16:45.089694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:45.089694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:16:45.089694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:45.089694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:16:45.090279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:45.090279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:16:45.090279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:45.090279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:16:45.090279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:16:45.090279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:16:45.090279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:16:45.090279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:16:45.090279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:16:45.090279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:45.090279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:45.090654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:45.090654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:16:45.090654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:45.090654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:45.090654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:45.090654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:16:45.090654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:45.090654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:16:45.090654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:16:45.090654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:45.090654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:16:45.090654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:45.090654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:16:45.090654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:16:45.091053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:45.091053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:16:45.091053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:45.091053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:16:45.091053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:45.091053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:16:45.091053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:16:45.091053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:16:45.091053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:16:45.091053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:16:45.091053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:45.091053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:16:45.091053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:16:45.091053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:16:45.091472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:16:45.091472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:16:45.091472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:45.091472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:45.091472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:16:45.091472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:16:45.091472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:16:45.091472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:16:45.091472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:16:45.091472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:16:45.091472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:45.091472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:45.091472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:45.091472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:16:45.091472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:45.091472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:45.091472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:45.091472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:16:45.091472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:45.091996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:16:45.091996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:16:45.091996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:45.091996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:16:45.091996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:45.091996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:16:45.091996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:16:45.091996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:45.091996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:16:45.091996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:45.091996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:16:45.095530 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:16:45.095769 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:16:45.095852 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:16:45.095960 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:16:45.096266 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:16:45.096514 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:16:45.096655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:45.097150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:16:45.097150 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:45.097150 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:16:45.097150 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:16:45.097150 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:45.097150 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:16:45.097150 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:45.097150 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:16:45.098487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:45.098487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:16:45.098487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:45.098487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:16:45.098487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:16:45.098487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:16:45.098487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:16:45.098487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:45.098487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:45.098487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:45.098487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:16:45.098487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:45.098736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:45.098736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:45.098736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:16:45.098736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:45.098736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:16:45.098736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:16:45.098736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:45.098736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:16:45.098736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:45.098736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:16:45.098736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:16:45.098736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:45.098736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:16:45.098966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:45.098966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:16:45.100090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:45.100090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:16:45.100090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:16:45.100090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:16:45.100090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:16:45.100090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:16:45.100090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:45.100090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:16:45.100090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:16:45.100090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:16:45.100090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:16:45.100090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:16:45.100090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:45.100090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:45.100090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:16:45.100090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:16:45.100476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:16:45.100476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:16:45.100476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:45.100476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:45.100476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:45.100476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:16:45.100476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:45.100476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:45.100476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:45.100476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:16:45.100476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:45.100476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:16:45.100476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:16:45.100476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:45.100476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:16:45.100476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:45.100476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:16:45.100476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:16:45.100476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:45.100856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:16:45.100856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:45.100856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:16:49.836961 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:16:49.837814 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:16:49.837814 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:16:49.838566 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:16:49.880533 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:16:49.880913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:49.881882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:16:49.881882 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:49.881882 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:16:49.881882 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:16:49.881882 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:49.881882 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:16:49.881882 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:49.881882 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:16:49.883009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:49.883009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:16:49.883009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:49.883009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:16:49.883009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:16:49.883009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:16:49.883009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:16:49.883009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:49.883009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:49.883009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:49.883009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:16:49.883009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:49.883696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:49.883696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:49.883696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:16:49.883696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:49.883696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:16:49.883696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:16:49.883696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:49.883696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:16:49.883696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:49.883696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:16:49.883696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:16:49.883696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:49.883696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:16:49.884302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:49.884302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:16:49.884302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:49.884302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:16:49.884302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:16:49.884302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:16:49.884302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:16:49.884302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:16:49.884302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:49.884302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:16:49.884302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:16:49.884302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:16:49.884302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:16:49.884302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:16:49.884302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:49.885029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:49.885029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:16:49.885029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:16:49.885029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:16:49.885029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:16:49.885029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:49.885029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:49.885029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:49.885029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:16:49.885029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:49.885029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:49.885029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:49.885029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:16:49.885029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:49.885029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:16:49.885029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:16:49.885029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:49.885029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:16:49.885029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:49.885902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:16:49.885902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:16:49.885902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:49.885902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:16:49.885902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:49.885902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:16:50.072453 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:16:50.073062 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:16:50.073187 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:16:50.074118 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:16:50.074858 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:16:50.075852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:50.077071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:16:50.077071 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:50.077071 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:16:50.077071 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:16:50.077071 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:50.077071 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:16:50.077071 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:50.077071 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:16:50.078140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:50.078140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:16:50.078140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:50.078140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:16:50.078140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:16:50.078140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:16:50.078140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:16:50.078140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:50.078140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:50.078140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:50.078140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:16:50.078140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:50.078622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:50.078622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:50.078622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:16:50.078622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:50.078622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:16:50.078622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:16:50.078622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:50.078622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:16:50.078622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:50.078622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:16:50.078622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:16:50.078622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:50.078622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:16:50.079001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:50.079001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:16:50.079284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:50.079284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:16:50.079284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:16:50.079284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:16:50.079284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:16:50.079284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:16:50.079284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:50.079284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:16:50.079284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:16:50.079284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:16:50.079284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:16:50.079284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:16:50.079284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:50.079284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:50.079284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:16:50.079284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:16:50.079888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:16:50.079888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:16:50.079888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:50.079888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:50.079888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:50.079888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:16:50.079888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:50.079888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:50.079888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:50.079888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:16:50.079888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:50.079888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:16:50.079888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:16:50.079888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:50.079888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:16:50.079888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:50.079888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:16:50.079888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:16:50.079888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:50.080561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:16:50.080561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:50.080561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:16:50.080561 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:16:50.080561 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:16:50.080561 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:16:50.080561 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:16:50.080561 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:16:50.080908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:50.081386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:16:50.081386 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:50.081386 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:16:50.081386 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:16:50.081386 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:50.081386 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:16:50.081386 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:50.081386 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:16:50.082212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:50.082212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:16:50.082212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:16:50.082212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:16:50.082212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:50.082212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:16:50.082212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:50.082212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:16:50.082212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:50.082212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:16:50.082212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:50.082212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:16:50.082212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:50.082212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:16:50.082212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:16:50.082689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:50.082689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:16:50.082689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:50.082689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:16:50.082689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:16:50.082689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:50.082689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:16:50.082689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:50.082689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:16:50.083176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:50.083176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:16:50.083176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:50.083176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:16:50.083176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:16:50.083176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:16:50.083176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:16:50.083176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:16:50.083176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:16:50.083176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:16:50.083176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:50.083176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:50.083560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:50.083560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:16:50.083560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:50.083560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:50.083560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:50.083560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:16:50.083560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:50.083560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:16:50.083560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:16:50.083560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:50.083560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:16:50.083560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:50.083560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:16:50.083560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:16:50.083934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:50.083934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:16:50.083934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:50.083934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:16:50.084193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:50.084193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:16:50.084193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:16:50.084193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:16:50.084193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:16:50.084193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:16:50.084193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:50.084193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:16:50.084193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:16:50.084193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:16:50.084193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:16:50.084193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:16:50.084193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:50.084193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:50.084193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:16:50.084193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:16:50.084671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:16:50.084671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:16:50.084671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:16:50.084671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:16:50.084671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:16:50.084671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:50.084671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:50.084671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:50.084671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:16:50.084671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:50.084671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:50.084671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:50.084671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:16:50.084671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:50.084671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:16:50.084671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:16:50.084671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:50.084671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:16:50.085168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:50.085168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:16:50.085168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:16:50.085168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:50.085168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:16:50.085168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:50.085168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:16:50.086642 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:16:50.087134 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:16:50.087234 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:16:50.087424 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:16:50.087982 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:16:50.088475 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:50.089303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:16:50.089303 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:50.089303 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:16:50.089303 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:16:50.089303 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:50.089303 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:16:50.089303 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:50.089303 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:16:50.090144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:50.090144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:16:50.090144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:16:50.090144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:16:50.090144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:50.090144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:16:50.090144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:50.090144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:16:50.090144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:50.090144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:16:50.090144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:50.090144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:16:50.090144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:50.090144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:16:50.090144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:16:50.090628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:50.090628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:16:50.090628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:50.090628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:16:50.090628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:16:50.090628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:50.090628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:16:50.090628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:50.090628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:16:50.091048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:50.091048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:16:50.091048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:50.091048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:16:50.091048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:16:50.091048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:16:50.091048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:16:50.091048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:16:50.091048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:16:50.091048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:50.091048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:50.091350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:50.091350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:16:50.091350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:50.091350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:50.091350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:50.091350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:16:50.091350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:50.091350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:16:50.091350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:16:50.091350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:50.091350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:16:50.091350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:50.091350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:16:50.091350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:16:50.091800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:50.091800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:16:50.091800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:50.091800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:16:50.092212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:50.092212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:16:50.092212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:16:50.092212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:16:50.092212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:16:50.092212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:16:50.092212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:50.092212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:16:50.092212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:16:50.092212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:16:50.092212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:16:50.092212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:16:50.092212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:50.092212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:50.092212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:16:50.092212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:16:50.092212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:16:50.092675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:16:50.092675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:16:50.092675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:16:50.092675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:50.092675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:50.092675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:50.092675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:16:50.092675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:50.092675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:50.092675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:50.092675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:16:50.092675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:50.092675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:16:50.092675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:16:50.092675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:50.092675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:16:50.092675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:50.092675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:16:50.092675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:16:50.093228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:50.093228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:16:50.093228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:50.093228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:16:50.095985 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:16:50.096287 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:16:50.096413 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:16:50.096619 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:16:50.097076 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:16:50.097516 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:16:50.097743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:50.098619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:16:50.098619 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:50.098619 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:16:50.098619 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:16:50.098619 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:50.098619 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:16:50.098619 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:50.098619 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:16:50.099618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:50.099618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:16:50.099618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:50.099618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:16:50.099618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:16:50.099618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:16:50.099618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:16:50.099618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:50.099618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:50.099618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:50.099618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:16:50.099618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:50.099967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:50.099967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:50.099967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:16:50.099967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:50.099967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:16:50.099967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:16:50.099967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:50.099967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:16:50.099967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:50.099967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:16:50.099967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:16:50.099967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:50.099967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:16:50.100314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:50.100314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:16:50.100658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:50.100658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:16:50.100658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:16:50.100658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:16:50.100658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:16:50.100658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:16:50.100658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:50.100658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:16:50.100658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:16:50.100658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:16:50.100658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:16:50.100658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:16:50.100658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:50.100658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:50.100658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:16:50.100658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:16:50.101080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:16:50.101080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:16:50.101080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:50.101080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:50.101080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:50.101080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:16:50.101080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:50.101080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:50.101080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:50.101080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:16:50.101080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:50.101080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:16:50.101080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:16:50.101080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:50.101080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:16:50.101080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:50.101080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:16:50.101080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:16:50.101080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:50.101605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:16:50.101605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:50.101605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:16:54.837566 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:16:54.838168 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:16:54.838359 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:16:54.839663 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:16:54.867357 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:16:54.867691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:54.868278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:16:54.868278 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:54.868278 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:16:54.868278 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:16:54.868278 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:54.868278 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:16:54.868278 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:54.868278 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:16:54.869025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.030 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:54.869025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:16:54.869025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:54.869025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:16:54.869025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:16:54.869025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:16:54.869025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:16:54.869025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:54.869025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:54.869025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:54.869025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:16:54.869025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:54.869474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:54.869474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:54.869474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:16:54.869474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:54.869474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:16:54.869474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:16:54.869474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:54.869474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:16:54.869474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:54.869474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:16:54.869474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:16:54.869474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:54.869474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:16:54.869870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:54.869870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:16:54.869870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:54.869870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:16:54.869870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:16:54.869870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:16:54.869870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:16:54.869870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:16:54.869870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:54.869870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:16:54.869870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:16:54.869870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:16:54.869870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:16:54.869870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:16:54.869870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:54.870309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:54.870309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:16:54.870309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:16:54.870309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:16:54.870309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:16:54.870309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:54.870309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:54.870309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:54.870309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:16:54.870309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:54.870309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:54.870309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:54.870309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:16:54.870309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:54.870309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:16:54.870309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:16:54.870309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:54.870309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:16:54.870309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:54.871039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:16:54.871039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:16:54.871039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:54.871039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:16:54.871039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:54.871039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:16:55.073066 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:16:55.073641 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:16:55.073847 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:16:55.074678 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:16:55.074845 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:16:55.074947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:55.075621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:16:55.075621 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:55.075621 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:16:55.075621 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:16:55.075621 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:55.075621 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:16:55.075621 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:55.075621 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:16:55.078241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:55.078241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:16:55.078241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:55.078241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:16:55.078241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:16:55.078241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:16:55.078241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:16:55.078241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:55.078241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:55.078241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:55.078241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:16:55.078241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:55.078937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:55.078937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:55.078937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:16:55.078937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:55.078937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:16:55.078937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:16:55.078937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:55.078937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:16:55.078937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:55.078937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:16:55.078937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:16:55.078937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:55.078937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:16:55.079535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:55.079535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:16:55.080549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:55.080549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:16:55.080549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:16:55.080549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:16:55.080549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:16:55.080549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:16:55.080549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:55.080549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:16:55.080549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:16:55.080549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:16:55.080549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:16:55.080549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:16:55.080549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:55.080549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:55.080549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:16:55.080549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:16:55.081272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:16:55.081272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:16:55.081272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:55.081272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:55.081272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:55.081272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:16:55.081272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:55.081272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:55.081272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:55.081272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:16:55.081272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:55.081272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:16:55.081272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:16:55.081272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:55.081272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:16:55.081272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:55.081272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:16:55.081272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:16:55.081272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:55.082150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:16:55.082150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:55.082150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:16:55.082150 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:16:55.082150 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:16:55.082150 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:16:55.082150 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:16:55.082150 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:16:55.082588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:55.082797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:16:55.082797 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:55.082797 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:16:55.082797 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:16:55.082797 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:55.082797 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:16:55.082797 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:55.082797 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:16:55.083360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:55.083360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:16:55.083360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:16:55.083360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:16:55.083360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:55.083360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:16:55.083360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:55.083360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:16:55.083360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:55.083360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:16:55.083360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:55.083360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:16:55.083360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:55.083360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:16:55.083360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:16:55.083865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:55.083865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:16:55.083865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:55.083865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:16:55.083865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:16:55.083865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:55.083865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:16:55.083865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:55.083865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:16:55.084157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:55.084157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:16:55.084157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:55.084157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:16:55.084157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:16:55.084157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:16:55.084157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:16:55.084157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:16:55.084157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:16:55.084157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:16:55.084157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:55.084157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:55.084546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:55.084546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:16:55.084546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:55.084546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:55.084546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:55.084546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:16:55.084546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:55.084546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:16:55.084546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:16:55.084546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:55.084546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:16:55.084546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:55.084546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:16:55.084546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:16:55.084912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:55.084912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:16:55.084912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:55.084912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:16:55.085170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:55.085170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:16:55.085170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:16:55.085170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:16:55.085170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:16:55.085170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:16:55.085170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:55.085170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:16:55.085170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:16:55.085170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:16:55.085170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:16:55.085170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:16:55.085170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:55.085170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:55.085170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:16:55.085170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:16:55.085663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:16:55.085663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:16:55.085663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:16:55.085663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:16:55.085663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:16:55.085663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:55.085663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:55.085663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:55.085663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:16:55.085663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:55.085663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:55.085663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:55.085663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:16:55.085663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:55.085663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:16:55.085663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:16:55.085663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:55.085663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:16:55.086170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:55.086170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:16:55.086170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:16:55.086170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:55.086170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:16:55.086170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:55.086170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:16:55.086997 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:16:55.087188 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:16:55.087268 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:16:55.087350 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:16:55.087681 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:16:55.087797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:55.088224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:16:55.088224 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:55.088224 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:16:55.088224 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:16:55.088224 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:55.088224 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:16:55.088224 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:55.088224 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:16:55.088805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:55.088805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:16:55.088805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:16:55.088805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:16:55.088805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:55.088805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:16:55.088805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:55.088805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:16:55.088805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:55.088805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:16:55.088805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:55.088805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:16:55.088805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:55.088805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:16:55.088805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:16:55.089248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:55.089248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:16:55.089248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:55.089248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:16:55.089248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:16:55.089248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:55.089248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:16:55.089248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:55.089248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:16:55.089577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:55.089577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:16:55.089577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:55.089577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:16:55.089577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:16:55.089577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:16:55.089577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:16:55.089577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:16:55.089577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:16:55.089577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:55.089577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:55.090179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:55.090179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:16:55.090179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:55.090179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:55.090179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:55.090179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:16:55.090179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:55.090179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:16:55.090179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:16:55.090179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:55.090179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:16:55.090179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:55.090179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:16:55.090179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:16:55.090697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:55.090697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:16:55.090697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:55.090697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:16:55.090826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:55.090826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:16:55.090826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:16:55.090826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:16:55.090826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:16:55.090826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:16:55.090826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:55.090826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:16:55.090826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:16:55.090826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:16:55.090826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:16:55.090826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:16:55.090826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:55.090826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:55.090826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:16:55.090826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:16:55.090826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:16:55.091300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:16:55.091300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:16:55.091300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:16:55.091300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:55.091300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:55.091300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:55.091300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:16:55.091300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:55.091300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:55.091300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:55.091300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:16:55.091300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:55.091300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:16:55.091300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:16:55.091300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:55.091300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:16:55.091300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:55.091300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:16:55.091300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:16:55.091859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:55.091859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:16:55.091859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:55.091859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:16:55.096357 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:16:55.096714 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:16:55.096827 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:16:55.096977 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:16:55.097290 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:16:55.097571 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:16:55.097710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:55.098232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:16:55.098232 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:55.098232 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:16:55.098232 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:16:55.098232 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:55.098232 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:16:55.098232 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:55.098232 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:16:55.099126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:55.099126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:16:55.099126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:55.099126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:16:55.099126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:16:55.099126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:16:55.099126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:16:55.099126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:55.099126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:55.099126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:55.099126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:16:55.099126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:55.099491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:55.099491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:55.099491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:16:55.099491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:55.099491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:16:55.099491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:16:55.099491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:55.099491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:16:55.099491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:55.099491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:16:55.099491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:16:55.099491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:55.099491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:16:55.099822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:55.099822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:16:55.100221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:55.100221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:16:55.100221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:16:55.100221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:16:55.100221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:16:55.100221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:16:55.100221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:55.100221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:16:55.100221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:16:55.100221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:16:55.100221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:16:55.100221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:16:55.100221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:55.100221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:55.100221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:16:55.100221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:16:55.100832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:16:55.100832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:16:55.100832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:55.100832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:55.100832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:55.100832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:16:55.100832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:55.100832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:55.100832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:55.100832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:16:55.100832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:55.100832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:16:55.100832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:16:55.100832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:55.100832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:16:55.100832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:55.100832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:16:55.100832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:16:55.100832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:55.101761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:16:55.101761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:55.101761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:16:59.838250 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:16:59.838858 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:16:59.838980 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:16:59.839823 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:16:59.888671 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:16:59.888863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:59.890017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:16:59.890017 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:59.890017 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:16:59.890017 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:16:59.890017 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:59.890017 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:16:59.890017 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:59.890017 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:16:59.890853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.051 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:59.890853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:16:59.890853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:59.890853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:16:59.890853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:16:59.890853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:16:59.890853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:16:59.890853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:59.890853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:59.890853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:59.890853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:16:59.890853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:59.891271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:16:59.891271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:59.891271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:16:59.891271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:59.891271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:16:59.891271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:16:59.891271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:59.891271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:16:59.891271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:59.891271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:16:59.891271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:16:59.891271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:59.891271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:16:59.891655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:59.891655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:16:59.892811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:59.892811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:16:59.892811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:16:59.892811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:16:59.892811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:16:59.892811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:16:59.892811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:59.892811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:16:59.892811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:16:59.892811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:16:59.892811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:16:59.892811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:16:59.892811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:16:59.892811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:16:59.892811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:16:59.892811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:16:59.893289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:16:59.893289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:16:59.893289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:59.893289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:59.893289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:16:59.893289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:16:59.893289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:16:59.893289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:16:59.893289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:16:59.893289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:16:59.893289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:16:59.893289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:16:59.893289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:16:59.893289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:16:59.893289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:16:59.893289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:16:59.893289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:16:59.893289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:16:59.893289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:16:59.893791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:16:59.893791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:16:59.893791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:17:00.073759 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:17:00.074264 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:17:00.074392 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:17:00.076068 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:17:00.076694 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:17:00.076694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:00.077331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:17:00.077331 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:00.077331 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:17:00.077331 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:17:00.077331 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:00.077331 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:17:00.077331 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:00.077331 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:17:00.078615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:00.078615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:17:00.078615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:00.078615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:17:00.078615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:17:00.078615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:17:00.078615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:17:00.078615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:00.078615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:00.078615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:00.078615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:17:00.078615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:00.078994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:00.078994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:00.078994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:17:00.078994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:00.078994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:17:00.078994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:17:00.078994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:00.078994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:17:00.078994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:00.078994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:17:00.078994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:17:00.078994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:00.078994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:17:00.079355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:00.079355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:17:00.080247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:00.080247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:17:00.080247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:17:00.080247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:17:00.080247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:17:00.080247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:17:00.080247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:00.080247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:17:00.080247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:17:00.080247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:17:00.080247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:17:00.080247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:17:00.080247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:00.080247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:00.080247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:17:00.080247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:17:00.080715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:17:00.080715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:17:00.080715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:00.080715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:00.080715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:00.080715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:17:00.080715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:00.080715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:00.080715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:00.080715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:17:00.080715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:00.080715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:17:00.080715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:17:00.080715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:00.080715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:17:00.080715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:00.080715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:17:00.080715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:17:00.080715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:00.081144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:17:00.081144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:00.081144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:17:00.081667 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:17:00.081727 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:17:00.081883 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:17:00.082060 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:17:00.082578 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:17:00.082890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:00.083677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:17:00.083677 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:00.083677 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:17:00.083677 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:17:00.083677 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:00.083677 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:17:00.083677 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:00.083677 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:17:00.085458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:00.085458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:17:00.085458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:17:00.085458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:17:00.085458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:00.085458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:17:00.085458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:00.085458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:17:00.085458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:00.085458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:17:00.085458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:00.085458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:17:00.085458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:00.085458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:17:00.085458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:17:00.085780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:00.085780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:17:00.085780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:00.085780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:17:00.085780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:17:00.085780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:00.085780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:17:00.085780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:00.085780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:17:00.085780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:00.085780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:17:00.085780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:00.085780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:17:00.085780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:17:00.086063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:17:00.086063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:17:00.086063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:17:00.086063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:17:00.086063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:17:00.086063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:00.086063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:00.086063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:00.086063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:17:00.086063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:00.086063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:00.086063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:00.086063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:17:00.086295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:00.086295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:17:00.086295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:17:00.086295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:00.086295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:17:00.086295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:00.086295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:17:00.086295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:17:00.086295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:00.086295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:17:00.086295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:00.086295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:17:00.087238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:00.087238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:17:00.087238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:17:00.087238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:17:00.087238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:17:00.087238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:17:00.087238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:00.087238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:17:00.087238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:17:00.087238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:17:00.087238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:17:00.087238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:17:00.087238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:00.087238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:00.087238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:17:00.087238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:17:00.087642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:17:00.087642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:17:00.087642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:17:00.087642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:17:00.087642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:17:00.087642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:00.087642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:00.087642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:00.087642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:17:00.087642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:00.087642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:00.087642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:00.087642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:17:00.087642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:00.087642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:17:00.087642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:17:00.087642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:00.087642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:17:00.087983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:00.087983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:17:00.087983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:17:00.087983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:00.087983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:17:00.087983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:00.087983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:17:00.087983 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:17:00.087983 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:17:00.087983 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:17:00.088191 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:17:00.089011 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:17:00.089011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:00.089233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:17:00.089233 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:00.089233 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:17:00.089233 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:17:00.089233 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:00.089233 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:17:00.089233 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:00.089233 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:17:00.089887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:00.089887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:17:00.089887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:17:00.089887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:17:00.089887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:00.089887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:17:00.089887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:00.089887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:17:00.089887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:00.089887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:17:00.089887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:00.089887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:17:00.089887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:00.089887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:17:00.089887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:17:00.090395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:00.090395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:17:00.090395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:00.090395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:17:00.090395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:17:00.090395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:00.090395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:17:00.090395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:00.090395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:17:00.090681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:00.090681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:17:00.090681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:00.090681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:17:00.090681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:17:00.090681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:17:00.090681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:17:00.090681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:17:00.090681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:17:00.090681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:00.090681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:00.090996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:00.090996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:17:00.090996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:00.090996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:00.090996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:00.090996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:17:00.090996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:00.090996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:17:00.090996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:17:00.090996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:00.090996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:17:00.090996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:00.090996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:17:00.090996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:17:00.091386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:00.091386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:17:00.091386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:00.091386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:17:00.093760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:00.093760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:17:00.093760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:17:00.093760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:17:00.093760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:17:00.093760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:17:00.093760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:00.093760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:17:00.093760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:17:00.093760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:17:00.093760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:17:00.093760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:17:00.093760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:00.093760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:00.093760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:17:00.093760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:17:00.093760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:17:00.094488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:17:00.094488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:17:00.094488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:17:00.094488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:00.094488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:00.094488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:00.094488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:17:00.094488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:00.094488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:00.094488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:00.094488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:17:00.094488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:00.094488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:17:00.094488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:17:00.094488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:00.094488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:17:00.094488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:00.094488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:17:00.094488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:17:00.095122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:00.095122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:17:00.095122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:00.095122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:17:00.096838 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:17:00.097194 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:17:00.097358 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:17:00.097579 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:17:00.098121 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:17:00.098508 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:17:00.098726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:00.099420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:17:00.099420 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:00.099420 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:17:00.099420 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:17:00.099420 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:00.099420 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:17:00.099420 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:00.099420 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:17:00.100657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:00.100657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:17:00.100657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:00.100657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:17:00.100657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:17:00.100657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:17:00.100657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:17:00.100657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:00.100657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:00.100657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:00.100657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:17:00.100657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:00.101106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:00.101106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:00.101106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:17:00.101106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:00.101106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:17:00.101106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:17:00.101106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:00.101106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:17:00.101106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:00.101106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:17:00.101106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:17:00.101106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:00.101106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:17:00.101483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:00.101483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:17:00.101670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:00.101670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:17:00.101670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:17:00.101670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:17:00.101670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:17:00.101670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:17:00.101670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:00.101670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:17:00.101670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:17:00.101670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:17:00.101670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:17:00.101670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:17:00.101670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:00.101670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:00.101670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:17:00.101670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:17:00.102113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:17:00.102113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:17:00.102113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:00.102113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:00.102113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:00.102113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:17:00.102113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:00.102113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:00.102113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:00.102113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:17:00.102113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:00.102113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:17:00.102113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:17:00.102113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:00.102113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:17:00.102113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:00.102113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:17:00.102113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:17:00.102113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:00.102641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:17:00.102641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:00.102641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:17:04.839077 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:17:04.839785 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:17:04.839834 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:17:04.840656 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:17:04.870783 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:17:04.871564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:04.871813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:17:04.871813 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:04.871813 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:17:04.871813 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:17:04.871813 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:04.871813 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:17:04.871813 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:04.871813 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:17:04.872792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:04.872792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:17:04.872792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:04.872792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:17:04.872792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:17:04.872792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:17:04.872792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:17:04.872792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:04.872792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:04.872792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:04.872792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:17:04.872792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:04.873560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:04.873560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:04.873560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:17:04.873560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:04.873560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:17:04.873560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:17:04.873560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:04.873560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:17:04.873560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:04.873560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:17:04.873560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:17:04.873560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:04.873560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:17:04.874187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:04.874187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:17:04.874187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:04.874187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:17:04.874187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:17:04.874187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:17:04.874187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:17:04.874187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:17:04.874187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:04.874187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:17:04.874187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:17:04.874187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:17:04.874187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:17:04.874187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:17:04.874187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:04.875119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:04.875119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:17:04.875119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:17:04.875119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:17:04.875119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:17:04.875119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:04.875119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:04.875119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:04.875119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:17:04.875119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:04.875119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:04.875119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:04.875119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:17:04.875119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:04.875119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:17:04.875119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:17:04.875119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:04.875119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:17:04.875119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:04.876030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:17:04.876030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:17:04.876030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:04.876030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:17:04.876030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:04.876030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:17:05.074495 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:17:05.075084 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:17:05.075258 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:17:05.076101 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:17:05.076305 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:17:05.076466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:05.077084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:17:05.077084 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:05.077084 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:17:05.077084 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:17:05.077084 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:05.077084 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:17:05.077084 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:05.077084 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:17:05.078037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:05.078037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:17:05.078037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:05.078037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:17:05.078037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:17:05.078037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:17:05.078037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:17:05.078037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:05.078037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:05.078037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:05.078037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:17:05.078037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:05.078711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:05.078711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:05.078711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:17:05.078711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:05.078711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:17:05.078711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:17:05.078711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:05.078711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:17:05.078711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:05.078711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:17:05.078711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:17:05.078711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:05.078711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:17:05.080054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:05.080054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:17:05.080054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:05.080054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:17:05.080054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:17:05.080054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:17:05.080054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:17:05.080054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:17:05.080054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:05.080054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:17:05.080054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:17:05.080054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:17:05.080054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:17:05.080054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:17:05.080054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:05.080882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:05.080882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:17:05.080882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:17:05.080882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:17:05.080882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:17:05.080882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:05.080882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:05.080882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:05.080882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:17:05.080882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:05.080882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:05.080882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:05.080882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:17:05.080882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:05.080882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:17:05.080882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:17:05.080882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:05.080882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:17:05.080882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:05.081379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:17:05.081379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:17:05.081379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:05.081379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:17:05.081379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:05.081379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:17:05.081981 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:17:05.082081 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:17:05.082252 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:17:05.082424 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:17:05.082860 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:17:05.083112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:05.083771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:17:05.083771 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:05.083771 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:17:05.083771 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:17:05.083771 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:05.083771 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:17:05.083771 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:05.083771 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:17:05.084819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:05.084819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:17:05.084819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:17:05.084819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:17:05.084819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:05.084819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:17:05.084819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:05.084819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:17:05.084819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:05.084819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:17:05.084819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:05.084819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:17:05.084819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:05.084819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:17:05.084819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:17:05.085295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:05.085295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:17:05.085295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:05.085295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:17:05.085295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:17:05.085295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:05.085295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:17:05.085295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:05.085295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:17:05.085968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:05.085968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:17:05.085968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:05.085968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:17:05.085968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:17:05.085968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:17:05.085968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:17:05.085968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:17:05.085968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:17:05.085968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:17:05.085968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:05.085968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:05.086337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:05.086337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:17:05.086337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:05.086337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:05.086337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:05.086337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:17:05.086337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:05.086337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:17:05.086337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:17:05.086337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:05.086337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:17:05.086337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:05.086337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:17:05.086337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:17:05.086748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:05.086748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:17:05.086748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:05.086748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:17:05.087426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:05.087426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:17:05.087426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:17:05.087426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:17:05.087426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:17:05.087426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:17:05.087426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:05.087426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:17:05.087426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:17:05.087426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:17:05.087426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:17:05.087426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:17:05.087426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:05.087426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:05.087426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:17:05.087426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:17:05.087881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:17:05.087881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:17:05.087881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:17:05.087881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:17:05.087881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:17:05.087881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:05.087881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:05.087881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:05.087881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:17:05.087881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:05.087881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:05.087881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:05.087881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:17:05.087881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:05.087881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:17:05.087881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:17:05.087881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:05.087881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:17:05.088424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:05.088424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:17:05.088424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:17:05.088424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:05.088424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:17:05.088424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:05.088424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:17:05.088424 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:17:05.088424 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:17:05.088424 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:17:05.088853 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:17:05.089532 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:17:05.089532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:05.090012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:17:05.090012 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:05.090012 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:17:05.090012 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:17:05.090012 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:05.090012 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:17:05.090012 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:05.090012 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:17:05.090872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:05.090872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:17:05.090872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:17:05.090872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:17:05.090872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:05.090872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:17:05.090872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:05.090872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:17:05.090872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:05.090872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:17:05.090872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:05.090872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:17:05.090872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:05.090872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:17:05.090872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:17:05.091323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:05.091323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:17:05.091323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:05.091323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:17:05.091323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:17:05.091323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:05.091323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:17:05.091323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:05.091323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:17:05.091783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:05.091783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:17:05.091783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:05.091783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:17:05.091783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:17:05.091783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:17:05.091783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:17:05.091783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:17:05.091783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:17:05.091783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:05.091783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:05.092127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:05.092127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:17:05.092127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:05.092127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:05.092127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:05.092127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:17:05.092127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:05.092127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:17:05.092127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:17:05.092127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:05.092127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:17:05.092127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:05.092127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:17:05.092127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:17:05.092565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:05.092565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:17:05.092565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:05.092565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:17:05.092961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:05.092961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:17:05.092961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:17:05.092961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:17:05.092961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:17:05.092961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:17:05.092961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:05.092961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:17:05.092961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:17:05.092961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:17:05.092961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:17:05.092961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:17:05.092961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:05.092961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:05.092961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:17:05.092961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:17:05.092961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:17:05.093485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:17:05.093485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:17:05.093485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:17:05.093485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:05.093485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:05.093485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:05.093485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:17:05.093485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:05.093485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:05.093485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:05.093485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:17:05.093485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:05.093485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:17:05.093485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:17:05.093485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:05.093485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:17:05.093485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:05.093485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:17:05.093485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:17:05.093981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:05.093981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:17:05.093981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:05.093981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:17:05.097221 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:17:05.097798 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:17:05.097798 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:17:05.097798 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:17:05.098073 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:17:05.098354 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:17:05.098577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:05.099031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:17:05.099031 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:05.099031 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:17:05.099031 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:17:05.099031 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:05.099031 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:17:05.099031 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:05.099031 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:17:05.099985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:05.099985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:17:05.099985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:05.099985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:17:05.099985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:17:05.099985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:17:05.099985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:17:05.099985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:05.099985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:05.099985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:05.099985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:17:05.099985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:05.100387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:05.100387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:05.100387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:17:05.100387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:05.100387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:17:05.100387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:17:05.100387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:05.100387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:17:05.100387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:05.100387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:17:05.100387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:17:05.100387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:05.100387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:17:05.100775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:05.100775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:17:05.101012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:05.101012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:17:05.101012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:17:05.101012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:17:05.101012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:17:05.101012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:17:05.101012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:05.101012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:17:05.101012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:17:05.101012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:17:05.101012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:17:05.101012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:17:05.101012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:05.101012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:05.101012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:17:05.101012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:17:05.101611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:17:05.101611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:17:05.101611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:05.101611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:05.101611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:05.101611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:17:05.101611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:05.101611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:05.101611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:05.101611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:17:05.101611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:05.101611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:17:05.101611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:17:05.101611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:05.101611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:17:05.101611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:05.101611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:17:05.101611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:17:05.101611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:05.102123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:17:05.102123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:05.102123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:17:09.840140 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:17:09.840991 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:17:09.840991 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:17:09.841811 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:17:09.874165 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:17:09.874379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:09.874776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:17:09.874776 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:09.874776 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:17:09.874776 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:17:09.874776 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:09.874776 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:17:09.874776 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:09.874776 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:17:09.875481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:09.875481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:17:09.875481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:09.875481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:17:09.875481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:17:09.875481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:17:09.875481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:17:09.875481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:09.875481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:09.875481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:09.875481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:17:09.875481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:09.876053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:09.876053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:09.876053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:17:09.876053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:09.876053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:17:09.876053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:17:09.876053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:09.876053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:17:09.876053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:09.876053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:17:09.876053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:17:09.876053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:09.876053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:17:09.876608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:09.876608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:17:09.876608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:09.876608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:17:09.876608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:17:09.876608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:17:09.876608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:17:09.876608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:17:09.876608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:09.876608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:17:09.876608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:17:09.876608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:17:09.876608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:17:09.876608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:17:09.876608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:09.877243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:09.877243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:17:09.877243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:17:09.877243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:17:09.877243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:17:09.877243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:09.877243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:09.877243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:09.877243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:17:09.877243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:09.877243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:09.877243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:09.877243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:17:09.877243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:09.877243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:17:09.877243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:17:09.877243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:09.877243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:17:09.877243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:09.878053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:17:09.878053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:17:09.878053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:09.878053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:17:09.878053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:09.878053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:17:10.075135 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:17:10.075474 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:17:10.075609 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:17:10.076481 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:17:10.076663 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:17:10.076815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:10.077354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:17:10.077354 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:10.077354 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:17:10.077354 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:17:10.077354 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:10.077354 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:17:10.077354 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:10.077354 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:17:10.078445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:10.078445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:17:10.078445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:10.078445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:17:10.078445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:17:10.078445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:17:10.078445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:17:10.078445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:10.078445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:10.078445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:10.078445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:17:10.078445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:10.078790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:10.078790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:10.078790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:17:10.078790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:10.078790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:17:10.078790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:17:10.078790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:10.078790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:17:10.078790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:10.078790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:17:10.078790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:17:10.078790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:10.078790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:17:10.079105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:10.079105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:17:10.079332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:10.079332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:17:10.079332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:17:10.079332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:17:10.079332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:17:10.079332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:17:10.079332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:10.079332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:17:10.079332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:17:10.079332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:17:10.079332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:17:10.079332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:17:10.079332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:10.079332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:10.079332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:17:10.079332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:17:10.079668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:17:10.079668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:17:10.079668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:10.079668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:10.079668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:10.079668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:17:10.079668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:10.079668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:10.079668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:10.079668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:17:10.079668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:10.079668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:17:10.079668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:17:10.079668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:10.079668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:17:10.079668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:10.079668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:17:10.079668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:17:10.079668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:10.080009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:17:10.080009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:10.080009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:17:10.081980 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:17:10.082239 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:17:10.082317 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:17:10.082424 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:17:10.082608 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:17:10.082752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:10.083064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:17:10.083064 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:10.083064 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:17:10.083064 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:17:10.083064 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:10.083064 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:17:10.083064 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:10.083064 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:17:10.083602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:10.083602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:17:10.083602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:17:10.083602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:17:10.083602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:10.083602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:17:10.083602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:10.083602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:17:10.083602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:10.083602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:17:10.083602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:10.083602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:17:10.083602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:10.083602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:17:10.083602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:17:10.083906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:10.083906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:17:10.083906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:10.083906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:17:10.083906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:17:10.083906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:10.083906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:17:10.083906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:10.083906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:17:10.084232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:10.084232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:17:10.084232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:10.084232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:17:10.084232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:17:10.084232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:17:10.084232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:17:10.084232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:17:10.084232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:17:10.084232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:17:10.084232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:10.084232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:10.084533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:10.084533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:17:10.084533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:10.084533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:10.084533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:10.084533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:17:10.084533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:10.084533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:17:10.084533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:17:10.084533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:10.084533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:17:10.084533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:10.084533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:17:10.084533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:17:10.084790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:10.084790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:17:10.084790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:10.084790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:17:10.084992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:10.084992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:17:10.084992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:17:10.084992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:17:10.084992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:17:10.084992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:17:10.084992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:10.084992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:17:10.084992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:17:10.084992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:17:10.084992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:17:10.084992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:17:10.084992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:10.084992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:10.084992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:17:10.084992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:17:10.085294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:17:10.085294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:17:10.085294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:17:10.085294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:17:10.085294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:17:10.085294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:10.085294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:10.085294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:10.085294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:17:10.085294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:10.085294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:10.085294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:10.085294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:17:10.085294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:10.085294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:17:10.085294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:17:10.085294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:10.085294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:17:10.085628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:10.085628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:17:10.085628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:17:10.085628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:10.085628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:17:10.085628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:10.085628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:17:10.089435 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:17:10.089435 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:17:10.089435 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:17:10.089435 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:17:10.089435 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:17:10.089435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:10.089673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:17:10.089673 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:10.089673 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:17:10.089673 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:17:10.089673 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:10.089673 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:17:10.089673 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:10.089673 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:17:10.090786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:10.090786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:17:10.090786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:17:10.090786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:17:10.090786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:10.090786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:17:10.090786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:10.090786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:17:10.090786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:10.090786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:17:10.090786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:10.090786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:17:10.090786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:10.090786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:17:10.090786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:17:10.091092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:10.091092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:17:10.091092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:10.091092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:17:10.091092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:17:10.091092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:10.091092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:17:10.091092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:10.091092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:17:10.091724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:10.091724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:17:10.091724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:10.091724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:17:10.091724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:17:10.091724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:17:10.091724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:17:10.091724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:17:10.091724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:17:10.091724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:10.091724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:10.091973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:10.091973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:17:10.091973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:10.091973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:10.091973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:10.091973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:17:10.091973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:10.091973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:17:10.091973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:17:10.091973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:10.091973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:17:10.091973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:10.091973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:17:10.091973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:17:10.092230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:10.092230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:17:10.092230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:10.092230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:17:10.092959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:10.092959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:17:10.092959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:17:10.092959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:17:10.092959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:17:10.092959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:17:10.092959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:10.092959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:17:10.092959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:17:10.092959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:17:10.092959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:17:10.092959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:17:10.092959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:10.092959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:10.092959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:17:10.092959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:17:10.092959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:17:10.093639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:17:10.093639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:17:10.093639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:17:10.093639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:10.093639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:10.093639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:10.093639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:17:10.093639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:10.093639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:10.093639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:10.093639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:17:10.093639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:10.093639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:17:10.093639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:17:10.093639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:10.093639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:17:10.093639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:10.093639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:17:10.093639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:17:10.094150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:10.094150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:17:10.094150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:10.094150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:17:10.097613 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:17:10.097927 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:17:10.098186 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:17:10.098416 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:17:10.099902 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:17:10.100214 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:17:10.100437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:10.101314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:17:10.101314 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:10.101314 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:17:10.101314 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:17:10.101314 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:10.101314 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:17:10.101314 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:10.101314 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:17:10.102145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:10.102145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:17:10.102145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:10.102145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:17:10.102145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:17:10.102145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:17:10.102145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:17:10.102145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:10.102145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:10.102145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:10.102145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:17:10.102145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:10.102555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:10.102555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:10.102555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:17:10.102555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:10.102555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:17:10.102555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:17:10.102555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:10.102555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:17:10.102555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:10.102555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:17:10.102555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:17:10.102555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:10.102555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:17:10.102799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:10.102799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:17:10.102947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:10.102947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:17:10.102947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:17:10.102947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:17:10.102947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:17:10.102947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:17:10.102947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:10.102947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:17:10.102947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:17:10.102947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:17:10.102947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:17:10.102947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:17:10.102947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:10.102947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:10.102947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:17:10.102947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:17:10.103554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:17:10.103554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:17:10.103554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:10.103554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:10.103554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:10.103554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:17:10.103554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:10.103554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:10.103554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:10.103554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:17:10.103554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:10.103554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:17:10.103554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:17:10.103554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:10.103554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:17:10.103554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:10.103554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:17:10.103554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:17:10.103554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:10.104038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:17:10.104038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:10.104038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:17:14.840792 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:17:14.842186 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:17:14.842186 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:17:14.842186 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:17:14.874777 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:17:14.875073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:14.875975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:17:14.875975 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:14.875975 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:17:14.875975 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:17:14.875975 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:14.875975 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:17:14.875975 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:14.875975 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:17:14.877524 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:14.877524 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:17:14.877524 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:14.877524 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:17:14.877524 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:17:14.877524 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:17:14.877524 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:17:14.877524 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:14.877524 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:14.877524 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:14.877524 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:17:14.877524 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:14.878469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:14.878469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:14.878469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:17:14.878469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:14.878469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:17:14.878469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:17:14.878469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:14.878469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:17:14.878469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:14.878469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:17:14.878469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:17:14.878469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:14.878469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:17:14.879271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:14.879271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:17:14.879271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:14.879271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:17:14.879271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:17:14.879271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:17:14.879271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:17:14.879271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:17:14.879271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:14.879271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:17:14.879271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:17:14.879271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:17:14.879271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:17:14.879271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:17:14.879271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:14.880277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:14.880277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:17:14.880277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:17:14.880277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:17:14.880277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:17:14.880277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:14.880277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:14.880277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:14.880277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:17:14.880277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:14.880277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:14.880277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:14.880277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:17:14.880277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:14.880277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:17:14.880277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:17:14.880277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:14.880277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:17:14.880277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:14.881298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:17:14.881298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:17:14.881298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:14.881298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:17:14.881298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:14.881298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:17:15.075649 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:17:15.076240 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:17:15.076523 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:17:15.077937 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:17:15.078370 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:17:15.078730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:15.080249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:17:15.080249 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:15.080249 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:17:15.080249 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:17:15.080249 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:15.080249 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:17:15.080249 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:15.080249 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:17:15.082000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:15.082000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:17:15.082000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:15.082000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:17:15.082000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:17:15.082000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:17:15.082000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:17:15.082000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:15.082000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:15.082000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:15.082000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:17:15.082000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:15.082305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:15.082305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:15.082305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:17:15.082305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:15.082305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:17:15.082305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:17:15.082305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:15.082305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:17:15.082305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:15.082305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:17:15.082305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:17:15.082305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:15.082305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:17:15.082727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:15.082727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:17:15.084018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:15.084018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:17:15.084018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:17:15.084018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:17:15.084018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:17:15.084018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:17:15.084018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:15.084018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:17:15.084018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:17:15.084018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:17:15.084018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:17:15.084018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:17:15.084018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:15.084018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:15.084018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:17:15.084018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:17:15.084395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:17:15.084395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:17:15.084395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:15.084395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:15.084395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:15.084395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:17:15.084395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:15.084395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:15.084395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:15.084395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:17:15.084395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:15.084395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:17:15.084395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:17:15.084395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:15.084395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:17:15.084395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:15.084395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:17:15.084395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:17:15.084395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:15.084776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:17:15.084776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:15.084776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:17:15.084776 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:17:15.084776 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:17:15.084897 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:17:15.085364 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:17:15.085711 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:17:15.085952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:15.087322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:17:15.087322 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:15.087322 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:17:15.087322 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:17:15.087322 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:15.087322 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:17:15.087322 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:15.087322 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:17:15.087948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:15.087948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:17:15.087948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:17:15.087948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:17:15.087948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:15.087948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:17:15.087948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:15.087948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:17:15.087948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:15.087948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:17:15.087948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:15.087948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:17:15.087948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:15.087948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:17:15.087948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:17:15.088323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:15.088323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:17:15.088323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:15.088323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:17:15.088323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:17:15.088323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:15.088323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:17:15.088323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:15.088323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:17:15.089206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:15.089206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:17:15.089206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:15.089206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:17:15.089206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:17:15.089206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:17:15.089206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:17:15.089206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:17:15.089206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:17:15.089206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:17:15.089206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:15.089206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:15.089504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:15.089504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:17:15.089504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:15.089504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:15.089504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:15.089504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:17:15.089504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:15.089504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:17:15.089504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:17:15.089504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:15.089504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:17:15.089504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:15.089504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:17:15.089504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:17:15.089762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:15.089762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:17:15.089762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:15.089762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:17:15.090224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:15.090224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:17:15.090224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:17:15.090224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:17:15.090224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:17:15.090224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:17:15.090224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:15.090224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:17:15.090224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:17:15.090224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:17:15.090224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:17:15.090224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:17:15.090224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:15.090224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:15.090224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:17:15.090224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:17:15.090645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:17:15.090645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:17:15.090645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:17:15.090645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:17:15.090645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:17:15.090645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:15.090645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:15.090645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:15.090645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:17:15.090645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:15.090645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:15.090645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:15.090645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:17:15.090645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:15.090645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:17:15.090645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:17:15.090645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:15.090645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:17:15.091025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:15.091025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:17:15.091025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:17:15.091025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:15.091025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:17:15.091025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:15.091025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:17:15.091025 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:17:15.091191 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:17:15.091261 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:17:15.091354 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:17:15.091853 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:17:15.092131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:15.092777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:17:15.092777 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:15.092777 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:17:15.092777 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:17:15.092777 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:15.092777 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:17:15.092777 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:15.092777 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:17:15.093331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:15.093331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:17:15.093331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:17:15.093331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:17:15.093331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:15.093331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:17:15.093331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:15.093331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:17:15.093331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:15.093331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:17:15.093331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:15.093331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:17:15.093331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:15.093331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:17:15.093331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:17:15.093668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:15.093668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:17:15.093668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:15.093668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:17:15.093668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:17:15.093668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:15.093668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:17:15.093668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:15.093668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:17:15.094323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:15.094323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:17:15.094323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:15.094323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:17:15.094323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:17:15.094323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:17:15.094323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:17:15.094323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:17:15.094323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:17:15.094323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:15.094323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:15.094597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:15.094597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:17:15.094597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:15.094597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:15.094597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:15.094597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:17:15.094597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:15.094597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:17:15.094597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:17:15.094597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:15.094597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:17:15.094597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:15.094597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:17:15.094597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:17:15.094858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:15.094858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:17:15.094858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:15.094858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:17:15.094858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:15.094858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:17:15.094858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:17:15.094858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:17:15.094858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:17:15.094858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:17:15.094858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:15.094858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:17:15.094858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:17:15.094858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:17:15.095120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:17:15.095120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:17:15.095120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:15.095120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:15.095120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:17:15.095120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:17:15.095120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:17:15.095120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:17:15.095120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:17:15.095120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:17:15.095120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:15.095120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:15.095120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:15.095120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:17:15.095120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:15.095120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:15.095120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:15.095120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:17:15.095120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:15.095661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:17:15.095661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:17:15.095661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:15.095661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:17:15.095661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:15.095661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:17:15.095661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:17:15.095661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:15.095661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:17:15.095661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:15.095661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:17:15.097997 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:17:15.098253 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:17:15.098334 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:17:15.098520 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:17:15.099526 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:17:15.099928 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:17:15.100155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:15.101110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:17:15.101110 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:15.101110 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:17:15.101110 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:17:15.101110 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:15.101110 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:17:15.101110 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:15.101110 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:17:15.101967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:15.101967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:17:15.101967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:15.101967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:17:15.101967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:17:15.101967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:17:15.101967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:17:15.101967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:15.101967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:15.101967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:15.101967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:17:15.101967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:15.102396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:15.102396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:15.102396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:17:15.102396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:15.102396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:17:15.102396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:17:15.102396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:15.102396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:17:15.102396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:15.102396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:17:15.102396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:17:15.102396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:15.102396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:17:15.102850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:15.102850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:17:15.102922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:15.102922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:17:15.102922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:17:15.102922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:17:15.102922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:17:15.102922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:17:15.102922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:15.102922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:17:15.102922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:17:15.102922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:17:15.102922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:17:15.102922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:17:15.102922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:15.102922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:15.102922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:17:15.102922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:17:15.103427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:17:15.103427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:17:15.103427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:15.103427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:15.103427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:15.103427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:17:15.103427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:15.103427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:15.103427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:15.103427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:17:15.103427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:15.103427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:17:15.103427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:17:15.103427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:15.103427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:17:15.103427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:15.103427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:17:15.103427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:17:15.103427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:15.103970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:17:15.103970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:15.103970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:17:19.841215 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:17:19.841979 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:17:19.841979 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:17:19.842354 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:17:19.874797 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:17:19.875226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:19.875848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:17:19.875848 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:19.875848 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:17:19.875848 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:17:19.875848 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:19.875848 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:17:19.875848 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:19.875848 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:17:19.877923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:19.877923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:17:19.877923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:19.877923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:17:19.877923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:17:19.877923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:17:19.877923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:17:19.877923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:19.877923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:19.877923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:19.877923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:17:19.877923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:19.878384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:19.878384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:19.878384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:17:19.878384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:19.878384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:17:19.878384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:17:19.878384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:19.878384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:17:19.878384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:19.878384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:17:19.878384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:17:19.878384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:19.878384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:17:19.878776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:19.878776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:17:19.878776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:19.878776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:17:19.878776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:17:19.878776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:17:19.878776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:17:19.878776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:17:19.878776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:19.878776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:17:19.878776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:17:19.878776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:17:19.878776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:17:19.878776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:17:19.878776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:19.879216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:19.879216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:17:19.879216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:17:19.879216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:17:19.879216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:17:19.879216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:19.879216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:19.879216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:19.879216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:17:19.879216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:19.879216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:19.879216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:19.879216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:17:19.879216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:19.879216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:17:19.879216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:17:19.879216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:19.879216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:17:19.879216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:19.879772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:17:19.879772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:17:19.879772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:19.879772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:17:19.879772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:19.879772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:17:20.076161 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:17:20.076594 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:17:20.076794 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:17:20.077779 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:17:20.078009 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:17:20.078131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:20.078883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:17:20.078883 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:20.078883 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:17:20.078883 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:17:20.078883 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:20.078883 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:17:20.078883 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:20.078883 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:17:20.080522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:20.080522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:17:20.080522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:20.080522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:17:20.080522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:17:20.080522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:17:20.080522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:17:20.080522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:20.080522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:20.080522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:20.080522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:17:20.080522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:20.080897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:20.080897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:20.080897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:17:20.080897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:20.080897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:17:20.080897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:17:20.080897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:20.080897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:17:20.080897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:20.080897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:17:20.080897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:17:20.080897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:20.080897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:17:20.081246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:20.081246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:17:20.082555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:20.082555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:17:20.082555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:17:20.082555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:17:20.082555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:17:20.082555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:17:20.082555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:20.082555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:17:20.082555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:17:20.082555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:17:20.082555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:17:20.082555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:17:20.082555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:20.082555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:20.082555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:17:20.082555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:17:20.083704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:17:20.083704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:17:20.083704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:20.083704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:20.083704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:20.083704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:17:20.083704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:20.083704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:20.083704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:20.083704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:17:20.083704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:20.083704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:17:20.083704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:17:20.083704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:20.083704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:17:20.083704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:20.083704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:17:20.083704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:17:20.083704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:20.084201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:17:20.084201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:20.084201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:17:20.084719 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:17:20.085042 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:17:20.085224 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:17:20.085375 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:17:20.085792 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:17:20.086026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:20.086527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:17:20.086527 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:20.086527 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:17:20.086527 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:17:20.086527 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:20.086527 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:17:20.086527 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:20.086527 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:17:20.087491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:20.087491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:17:20.087491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:17:20.087491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:17:20.087491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:20.087491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:17:20.087491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:20.087491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:17:20.087491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:20.087491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:17:20.087491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:20.087491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:17:20.087491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:20.087491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:17:20.087491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:17:20.088086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:20.088086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:17:20.088086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:20.088086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:17:20.088086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:17:20.088086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:20.088086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:17:20.088086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:20.088086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:17:20.088977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:20.088977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:17:20.088977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:20.088977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:17:20.088977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:17:20.088977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:17:20.088977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:17:20.088977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:17:20.088977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:17:20.088977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:17:20.088977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:20.088977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:20.089734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:20.089734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:17:20.089734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:20.089734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:20.089734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:20.089734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:17:20.089734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:20.089734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:17:20.089734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:17:20.089734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:20.089734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:17:20.089734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:20.089734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:17:20.089734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:17:20.090066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:20.090066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:17:20.090066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:20.090066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:17:20.090762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:20.090762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:17:20.090762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:17:20.090762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:17:20.090762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:17:20.090762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:17:20.090762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:20.090762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:17:20.090762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:17:20.090762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:17:20.090762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:17:20.090762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:17:20.090762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:20.090762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:20.090762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:17:20.090762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:17:20.094692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:17:20.094692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:17:20.094692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:17:20.094692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:17:20.094692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:17:20.094692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:20.094692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:20.094692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:20.094692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:17:20.094692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:20.094692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:20.094692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:20.094692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:17:20.094692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:20.094692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:17:20.094692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:17:20.094692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:20.094692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:17:20.095018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:20.095018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:17:20.095018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:17:20.095018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:20.095018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:17:20.095018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:20.095018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:17:20.095018 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:17:20.095018 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:17:20.095018 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:17:20.095229 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:17:20.095229 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:17:20.095229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:20.095229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:17:20.095229 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:20.095229 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:17:20.095229 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:17:20.095229 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:20.095229 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:17:20.095229 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:20.095229 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:17:20.095229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:20.095816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:17:20.095816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:17:20.095816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:17:20.095816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:20.095816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:17:20.095816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:20.095816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:17:20.095816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:20.095816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:17:20.095816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:20.095816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:17:20.095816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:20.095816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:17:20.095816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:17:20.095816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:20.095816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:17:20.095816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:20.096569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:17:20.096569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:17:20.096569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:20.096569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:17:20.096569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:20.096569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:17:20.096569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:20.096569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:17:20.096569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:20.096569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:17:20.096569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:17:20.096569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:17:20.096569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:17:20.097137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:17:20.097137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:17:20.097137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:20.097137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:20.097137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:20.097137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:17:20.097137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:20.097137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:20.097137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:20.097137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:17:20.097137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:20.097137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:17:20.097137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:17:20.097366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:20.097366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:17:20.097366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:20.097366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:17:20.097366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:17:20.097366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:20.097366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:17:20.097366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:20.097366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:17:20.097366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:20.097366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:17:20.097366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:17:20.097613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:17:20.097613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:17:20.097613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:17:20.097613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:20.097613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:17:20.097613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:17:20.097613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:17:20.097613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:17:20.097613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:17:20.097613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:20.097613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:20.097613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:17:20.097613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:17:20.097613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:17:20.097613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:17:20.097613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:17:20.097613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:17:20.097613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:20.097613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:20.098135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:20.098135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:17:20.098135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:20.098135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:20.098135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:20.098135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:17:20.098135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:20.098135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:17:20.098135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:17:20.098135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:20.098135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:17:20.098135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:20.098135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:17:20.098135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:17:20.098135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:20.098135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:17:20.098135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:20.098135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:17:20.098465 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:17:20.098689 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:17:20.098831 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:17:20.099028 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:17:20.099383 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:17:20.099669 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:17:20.099816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:20.100372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:17:20.100372 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:20.100372 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:17:20.100372 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:17:20.100372 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:20.100372 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:17:20.100372 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:20.100372 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:17:20.101553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:20.101553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:17:20.101553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:20.101553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:17:20.101553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:17:20.101553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:17:20.101553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:17:20.101553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:20.101553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:20.101553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:20.101553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:17:20.101553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:20.101814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:20.101814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:20.101814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:17:20.101814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:20.101814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:17:20.101814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:17:20.101814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:20.101814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:17:20.101814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:20.101814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:17:20.101814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:17:20.101814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:20.101814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:17:20.102074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:20.102074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:17:20.103036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:20.103036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:17:20.103036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:17:20.103036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:17:20.103036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:17:20.103036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:17:20.103036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:20.103036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:17:20.103036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:17:20.103036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:17:20.103036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:17:20.103036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:17:20.103036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:20.103036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:20.103036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:17:20.103036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:17:20.103365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:17:20.103365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:17:20.103365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:20.103365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:20.103365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:20.103365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:17:20.103365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:20.103365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:20.103365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:20.103365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:17:20.103365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:20.103365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:17:20.103365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:17:20.103365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:20.103365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:17:20.103365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:20.103365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:17:20.103365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:17:20.103365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:20.103748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:17:20.103748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:20.103748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:17:24.841849 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:17:24.842485 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:17:24.842485 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:17:24.842811 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:17:24.851736 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG dbcounter [-] [105581] Writing DB stats neutron:SELECT=240 {{(pid=105581) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 14:17:24.874958 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:17:24.875141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:24.875796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:17:24.875796 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:24.875796 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:17:24.875796 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:17:24.875796 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:24.875796 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:17:24.875796 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:24.875796 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:17:24.876772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:24.876772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:17:24.876772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:24.876772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:17:24.876772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:17:24.876772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:17:24.876772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:17:24.876772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:24.876772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:24.876772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:24.876772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:17:24.876772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:24.877465 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:24.877465 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:24.877465 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:17:24.877465 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:24.877465 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:17:24.877465 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:17:24.877465 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:24.877465 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:17:24.877465 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:24.877465 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:17:24.877465 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:17:24.877465 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:24.877465 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:17:24.878151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:24.878151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:17:24.878151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:24.878151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:17:24.878151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:17:24.878151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:17:24.878151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:17:24.878151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:17:24.878151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:24.878151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:17:24.878151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:17:24.878151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:17:24.878151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:17:24.878151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:17:24.878151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:24.879073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:24.879073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:17:24.879073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:17:24.879073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:17:24.879073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:17:24.879073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:24.879073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:24.879073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:24.879073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:17:24.879073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:24.879073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:24.879073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:24.879073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:17:24.879073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:24.879073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:17:24.879073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:17:24.879073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:24.879073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:17:24.879073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:24.880126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:17:24.880126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:17:24.880126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:24.880126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:17:24.880126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:24.880126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:17:25.076961 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:17:25.085082 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:17:25.090378 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:17:25.090520 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:17:25.091839 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:17:25.092573 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:17:25.092948 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:17:25.093151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:25.093576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:17:25.093576 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:25.093576 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:17:25.093576 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:17:25.093576 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:25.093576 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:17:25.093576 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:25.093576 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:17:25.094354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:25.094354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:17:25.094354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:25.094354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:17:25.094354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:17:25.094354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:17:25.094354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:17:25.094354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:25.094354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:25.094354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:25.094354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:17:25.094354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:25.095190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:25.095190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:25.095190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:17:25.095190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:25.095190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:17:25.095190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:17:25.095190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:25.095190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:17:25.095190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:25.095190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:17:25.095190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:17:25.095190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:25.095190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:17:25.096176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:25.096176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:17:25.096176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:25.096176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:17:25.096176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:17:25.096176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:17:25.096176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:17:25.096176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:17:25.096176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:25.096176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:17:25.096176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:17:25.096176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:17:25.096176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:17:25.096176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:17:25.096176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:25.096959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:25.096959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:17:25.096959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:17:25.096959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:17:25.096959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:17:25.096959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:25.096959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:25.096959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:25.096959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:17:25.096959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:25.096959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:25.096959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:25.096959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:17:25.096959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:25.096959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:17:25.096959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:17:25.096959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:25.096959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:17:25.096959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:25.097464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:17:25.097464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:17:25.097464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:25.097464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:17:25.097464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:25.097464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:17:25.097464 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:17:25.097464 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:17:25.097464 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:17:25.097464 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:17:25.097749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:25.097749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:17:25.097749 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:25.097749 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:17:25.097749 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:17:25.097749 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:25.097749 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:17:25.097749 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:25.097749 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:17:25.097749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:25.097749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:17:25.097749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:17:25.097749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:17:25.097749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:25.097749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:17:25.097749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:25.097749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:17:25.098276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:25.098276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:17:25.098276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:25.098276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:17:25.098276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:25.098276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:17:25.098276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:17:25.098276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:25.098276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:17:25.098276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:25.098276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:17:25.098276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:17:25.098276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:25.098276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:17:25.098276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:25.098276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:17:25.100301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:25.100301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:17:25.100301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:25.100301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:17:25.100301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:17:25.100301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:17:25.100301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:17:25.100301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:17:25.100301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:17:25.100301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:17:25.100301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:25.100301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:25.100647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:25.100647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:17:25.100647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:25.100647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:25.100647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:25.100647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:17:25.100647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:25.100647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:17:25.100647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:17:25.100647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:25.100647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:17:25.100647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:25.100647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:17:25.100647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:17:25.100975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:25.100975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:17:25.100975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:25.100975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:17:25.100975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:25.100975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:17:25.100975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:17:25.100975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:17:25.100975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:17:25.100975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:17:25.100975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:25.100975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:17:25.100975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:17:25.100975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:17:25.101303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:17:25.101303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:17:25.101303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:25.101303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:25.101303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:17:25.101303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:17:25.101303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:17:25.101303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:17:25.101303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:17:25.101303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:17:25.101303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:17:25.101303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:25.101303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:25.101303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:25.101303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:17:25.101303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:25.101303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:25.101303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:25.101303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:17:25.101843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:25.101843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:17:25.101843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:17:25.101843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:25.101843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:17:25.101843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:25.101843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:17:25.101843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:17:25.101843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:25.101843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:17:25.101843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:25.101843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:17:25.101843 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:17:25.101843 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:17:25.102281 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:17:25.102281 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:17:25.102281 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:17:25.102281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:25.102281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:17:25.102281 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:25.102281 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:17:25.102281 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:17:25.102281 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:25.102281 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:17:25.102281 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:25.102281 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:17:25.102614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:25.102614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:17:25.102614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:17:25.102614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:17:25.102614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:25.102614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:17:25.102614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:25.102614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:17:25.102614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:25.102614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:17:25.102614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:25.102614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:17:25.102614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:25.102614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:17:25.102614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:17:25.102962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:25.102962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:17:25.102962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:25.102962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:17:25.102962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:17:25.102962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:25.102962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:17:25.102962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:25.102962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:17:25.102962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:25.102962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:17:25.102962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:25.102962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:17:25.102962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:17:25.105187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:17:25.105187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:17:25.105187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:17:25.105187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:17:25.105187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:25.105187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:25.105187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:25.105187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:17:25.105187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:25.105187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:25.105187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:25.105187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:17:25.105187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:25.106067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:17:25.106067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:17:25.106067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:25.106067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:17:25.106067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:25.106067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:17:25.106067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:17:25.106067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:25.106067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:17:25.106067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:25.106067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:17:25.106067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:25.106615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:17:25.106615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:17:25.106615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:17:25.106615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:17:25.106615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:17:25.106615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:25.106615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:17:25.106615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:17:25.106615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:17:25.106615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:17:25.106615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:17:25.106615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:25.106615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:25.106615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:17:25.106615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:17:25.106615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:17:25.106615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:17:25.106615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:17:25.106615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:17:25.107054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:25.107054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:25.107054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:25.107054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:17:25.107054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:25.107054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:25.107054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:25.107054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:17:25.107054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:25.107054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:17:25.107054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:17:25.107054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:25.107054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:17:25.107054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:25.107054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:17:25.107054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:17:25.107054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:25.107054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:17:25.107054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:25.107054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:17:25.107562 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:17:25.107562 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:17:25.107562 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:17:25.107562 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:17:25.107562 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:17:25.107562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:25.107741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:17:25.107741 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:25.107741 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:17:25.107741 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:17:25.107741 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:25.107741 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:17:25.107741 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:25.107741 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:17:25.107741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:25.107741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:17:25.107741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:25.107741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:17:25.107741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:17:25.107741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:17:25.107741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:17:25.108097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:25.108097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:25.108097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:25.108097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:17:25.108097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:25.108097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:25.108097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:25.108097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:17:25.108097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:25.108097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:17:25.108097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:17:25.108097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:25.108097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:17:25.108097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:25.108879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:17:25.108879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:17:25.108879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:25.108879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:17:25.108879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:25.108879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:17:25.108879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:25.108879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:17:25.108879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:17:25.108879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:17:25.108879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:17:25.108879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:17:25.108879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:25.108879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:17:25.110086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:17:25.110086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:17:25.110086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:17:25.110086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:17:25.110086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:25.110086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:25.110086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:17:25.110086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:17:25.110086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:17:25.110086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:17:25.110086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:25.110086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:25.110086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:25.110086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:17:25.110086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:25.110086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:25.110086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:25.110086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:17:25.110086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:25.110086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:17:25.110984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:17:25.110984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:25.110984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:17:25.110984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:25.110984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:17:25.110984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:17:25.110984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:25.110984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:17:25.110984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:25.110984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:17:29.842319 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:17:29.843370 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:17:29.843370 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:17:29.844456 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:17:29.875856 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:17:29.876286 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:29.877216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:17:29.877216 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:29.877216 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:17:29.877216 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:17:29.877216 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:29.877216 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:17:29.877216 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:29.877216 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:17:29.878056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:29.878056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:17:29.878056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:29.878056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:17:29.878056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:17:29.878056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:17:29.878056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:17:29.878056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:29.878056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:29.878056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:29.878056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:17:29.878056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:29.878497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:29.878497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:29.878497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:17:29.878497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:29.878497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:17:29.878497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:17:29.878497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:29.878497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:17:29.878497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:29.878497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:17:29.878497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:17:29.878497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:29.878497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:17:29.879029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:29.879029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:17:29.879029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:29.879029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:17:29.879029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:17:29.879029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:17:29.879029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:17:29.879029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:17:29.879029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:29.879029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:17:29.879029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:17:29.879029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:17:29.879029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:17:29.879029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:17:29.879029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:29.879808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:29.879808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:17:29.879808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:17:29.879808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:17:29.879808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:17:29.879808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:29.879808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:29.879808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:29.879808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:17:29.879808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:29.879808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:29.879808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:29.879808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:17:29.879808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:29.879808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:17:29.879808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:17:29.879808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:29.879808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:17:29.879808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:29.880814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:17:29.880814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:17:29.880814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:29.880814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:17:29.880814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:29.880814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:17:30.090990 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:17:30.091677 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:17:30.091954 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:17:30.094221 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:17:30.094221 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:17:30.094221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:30.094532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:17:30.094532 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:30.094532 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:17:30.094532 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:17:30.094532 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:30.094532 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:17:30.094532 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:30.094532 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:17:30.095845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:30.095845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:17:30.095845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:30.095845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:17:30.095845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:17:30.095845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:17:30.095845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:17:30.095845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:30.095845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:30.095845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:30.095845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:17:30.095845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:30.096667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:30.096667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:30.096667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:17:30.096667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:30.096667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:17:30.096667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:17:30.096667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:30.096667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:17:30.096667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:30.096667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:17:30.096667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:17:30.096667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:30.096667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:17:30.097073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:30.097073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:17:30.097073 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:17:30.097991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:30.097991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:17:30.097991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:17:30.097991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:17:30.097991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:17:30.097991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:17:30.097991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:30.097991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:17:30.097991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:17:30.097991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:17:30.097991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:17:30.097991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:17:30.097991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:30.097991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:30.097991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:17:30.097991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:17:30.098661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:17:30.098661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:17:30.098661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:30.098661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:30.098661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:30.098661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:17:30.098661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:30.098661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:30.098661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:30.098661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:17:30.098661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:30.098661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:17:30.098661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:17:30.098661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:30.098661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:17:30.098661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:30.098661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:17:30.098661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:17:30.098661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:30.099224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:17:30.099224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:30.099224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:17:30.099224 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:17:30.099224 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:17:30.099224 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:17:30.099224 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:17:30.099224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:30.099766 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:17:30.104285 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:17:30.140785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:17:30.140785 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:30.140785 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:17:30.140785 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:17:30.140785 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:30.140785 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:17:30.140785 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:30.140785 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:17:30.142161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:30.142161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:17:30.142161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:17:30.142161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:17:30.142161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:30.142161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:17:30.142161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:30.142161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:17:30.142161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:30.142161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:17:30.142161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:30.142161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:17:30.142161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:30.142161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:17:30.142161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:17:30.142518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:30.142518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:17:30.142518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:30.142518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:17:30.142518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:17:30.142518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:30.142518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:17:30.142518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:30.142518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:17:30.142518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:30.142518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:17:30.142518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:30.142518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:17:30.142518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:17:30.142860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:17:30.142860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:17:30.142860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:17:30.142860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:17:30.142860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:17:30.142860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:30.142860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:30.142860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:30.142860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:17:30.142860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:30.142860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:30.142860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:30.142860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:17:30.143138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:30.143138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:17:30.143138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:17:30.143138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:30.143138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:17:30.143138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:30.143138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:17:30.143138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:17:30.143138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:30.143138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:17:30.143138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:30.143138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:17:30.143395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:30.143395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:17:30.143395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:17:30.143395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:17:30.143395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:17:30.143395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:17:30.143395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:30.143395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:17:30.143395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:17:30.143395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:17:30.143395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:17:30.143395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:17:30.143395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:30.143395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:30.143395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:17:30.143395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:17:30.143758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:17:30.143758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:17:30.143758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:17:30.143758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:17:30.143758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:17:30.143758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:30.143758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:30.143758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:30.143758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:17:30.143758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:30.143758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:30.143758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:30.143758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:17:30.143758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:30.143758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:17:30.143758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:17:30.143758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:30.143758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:17:30.144135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:30.144135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:17:30.144135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:17:30.144135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:30.144135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:17:30.144135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:30.144135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:17:30.144135 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:17:30.144135 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:17:30.144135 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:17:30.144355 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:17:30.144355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:30.144355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:17:30.144355 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:30.144355 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:17:30.144355 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:17:30.144355 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:30.144355 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:17:30.144355 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:30.144355 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:17:30.144621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:30.144621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:17:30.144621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:17:30.144621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:17:30.144621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:30.144621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:17:30.144621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:30.144621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:17:30.144621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:30.144621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:17:30.144621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:30.144621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:17:30.144621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:30.144621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:17:30.144621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:17:30.144925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:30.144925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:17:30.144925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:30.144925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:17:30.144925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:17:30.144925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:30.144925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:17:30.144925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:30.144925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:17:30.145171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:30.145171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:17:30.145171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:30.145171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:17:30.145171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:17:30.145171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:17:30.145171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:17:30.145171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:17:30.145171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:17:30.145171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:30.145171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:30.145427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:30.145427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:17:30.145427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:30.145427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:30.145427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:30.145427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:17:30.145427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:30.145427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:17:30.145427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:17:30.145427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:30.145427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:17:30.145427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:30.145427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:17:30.145427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:17:30.145695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:30.145695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:17:30.145695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:30.145695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:17:30.145927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:30.145927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:17:30.145927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:17:30.145927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:17:30.145927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:17:30.145927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:17:30.145927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:30.145927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:17:30.145927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:17:30.145927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:17:30.145927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:17:30.145927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:17:30.145927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:30.145927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:30.145927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:17:30.145927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:17:30.145927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:17:30.146281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:17:30.146281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:17:30.146281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:17:30.146281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:30.146281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:30.146281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:30.146281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:17:30.146281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:30.146281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:30.146281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:30.146281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:17:30.146281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:30.146281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:17:30.146281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:17:30.146281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:30.146281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:17:30.146281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:30.146281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:17:30.146281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:17:30.146673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:30.146673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:17:30.146673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:30.146673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:17:30.146673 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:17:30.146673 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:17:30.146673 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:17:30.146673 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:17:30.146875 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:17:30.146875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:30.147004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:17:30.147004 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:30.147004 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:17:30.147004 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:17:30.147004 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:30.147004 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:17:30.147004 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:30.147004 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:17:30.147686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:30.147686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:17:30.147686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:30.147686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:17:30.147686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:17:30.147686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:17:30.147686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:17:30.147686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:30.147686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:30.147686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:30.147686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:17:30.147686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:30.147989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:30.147989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:30.147989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:17:30.147989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:30.147989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:17:30.147989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:17:30.147989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:30.147989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:17:30.147989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:30.147989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:17:30.147989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:17:30.147989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:30.147989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:17:30.148241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:30.148241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:17:30.148332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:30.148332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:17:30.148332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:17:30.148332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:17:30.148332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:17:30.148332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:17:30.148332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:30.148332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:17:30.148332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:17:30.148332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:17:30.148332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:17:30.148332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:17:30.148332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:30.148332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:30.148332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:17:30.148332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:17:30.148724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:17:30.148724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:17:30.148724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:30.148724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:30.148724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:30.148724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:17:30.148724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:30.148724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:30.148724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:30.148724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:17:30.148724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:30.148724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:17:30.148724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:17:30.148724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:30.148724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:17:30.148724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:30.148724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:17:30.148724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:17:30.148724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:30.149149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:17:30.149149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:30.149149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:17:34.843223 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:17:34.843842 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:17:34.843842 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:17:34.845367 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:17:34.876891 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:17:34.877149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:34.877970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:17:34.877970 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:34.877970 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:17:34.877970 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:17:34.877970 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:34.877970 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:17:34.877970 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:34.877970 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:17:34.878924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:34.878924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:17:34.878924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:34.878924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:17:34.878924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:17:34.878924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:17:34.878924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:17:34.878924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:34.878924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:34.878924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:34.878924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:17:34.878924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:34.879459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:34.879459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:34.879459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:17:34.879459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:34.879459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:17:34.879459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:17:34.879459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:34.879459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:17:34.879459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:34.879459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:17:34.879459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:17:34.879459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:34.879459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:17:34.879877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:34.879877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:17:34.880031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:34.880031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:17:34.880031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:17:34.880031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:17:34.880031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:17:34.880031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:17:34.880031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:34.880031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:17:34.880031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:17:34.880031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:17:34.880031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:17:34.880031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:17:34.880031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:34.880031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:34.880031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:17:34.880031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:17:34.880553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:17:34.880553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:17:34.880553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:34.880553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:34.880553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:34.880553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:17:34.880553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:34.880553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:34.880553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:34.880553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:17:34.880553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:34.880553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:17:34.880553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:17:34.880553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:34.880553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:17:34.880553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:34.880553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:17:34.880553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:17:34.880553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:34.881122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:17:34.881122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:34.881122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:17:35.091531 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:17:35.092257 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:17:35.092376 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:17:35.093549 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:17:35.093845 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:17:35.094097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:35.095020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:17:35.095020 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:35.095020 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:17:35.095020 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:17:35.095020 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:35.095020 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:17:35.095020 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:35.095020 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:17:35.096641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:35.096641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:17:35.096641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:35.096641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:17:35.096641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:17:35.096641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:17:35.096641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:17:35.096641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:35.096641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:35.096641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:35.096641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:17:35.096641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:35.097026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:35.097026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:35.097026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:17:35.097026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:35.097026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:17:35.097026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:17:35.097026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:35.097026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:17:35.097026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:35.097026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:17:35.097026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:17:35.097026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:35.097026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:17:35.097382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:35.097382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:17:35.098290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:35.098290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:17:35.098290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:17:35.098290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:17:35.098290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:17:35.098290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:17:35.098290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:35.098290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:17:35.098290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:17:35.098290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:17:35.098290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:17:35.098290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:17:35.098290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:35.098290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:35.098290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:17:35.098290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:17:35.098808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:17:35.098808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:17:35.098808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:35.098808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:35.098808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:35.098808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:17:35.098808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:35.098808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:35.098808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:35.098808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:17:35.098808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:35.098808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:17:35.098808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:17:35.098808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:35.098808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:17:35.098808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:35.098808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:17:35.098808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:17:35.098808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:35.099460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:17:35.099460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:35.099460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:17:35.099460 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:17:35.099460 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:17:35.099460 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:17:35.099460 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:17:35.101534 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:17:35.101534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:35.101534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:17:35.101534 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:35.101534 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:17:35.101534 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:17:35.101534 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:35.101534 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:17:35.101534 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:35.101534 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:17:35.101993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:35.101993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:17:35.101993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:17:35.101993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:17:35.101993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:35.101993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:17:35.101993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:35.101993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:17:35.101993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:35.101993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:17:35.101993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:35.101993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:17:35.101993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:35.101993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:17:35.101993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:17:35.102446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:35.102446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:17:35.102446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:35.102446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:17:35.102446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:17:35.102446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:35.102446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:17:35.102446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:35.102446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:17:35.102798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:35.102798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:17:35.102798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:35.102798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:17:35.102798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:17:35.102798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:17:35.102798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:17:35.102798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:17:35.102798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:17:35.102798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:17:35.102798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:35.102798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:35.103196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:35.103196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:17:35.103196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:35.103196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:35.103196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:35.103196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:17:35.103196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:35.103196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:17:35.103196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:17:35.103196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:35.103196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:17:35.103196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:35.103196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:17:35.103196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:17:35.103571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:35.103571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:17:35.103571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:35.103571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:17:35.103948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:35.103948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:17:35.103948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:17:35.103948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:17:35.103948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:17:35.103948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:17:35.103948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:35.103948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:17:35.103948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:17:35.103948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:17:35.103948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:17:35.103948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:17:35.103948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:35.103948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:35.103948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:17:35.103948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:17:35.104427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:17:35.104427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:17:35.104427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:17:35.104427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:17:35.104427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:17:35.104427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:35.104427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:35.104427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:35.104427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:17:35.104427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:35.104427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:35.104427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:35.104427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:17:35.104427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:35.104427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:17:35.104427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:17:35.104427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:35.104427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:17:35.104928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:35.104928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:17:35.104928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:17:35.104928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:35.104928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:17:35.104928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:35.104928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:17:35.143269 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:17:35.143884 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:17:35.143884 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:17:35.143996 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:17:35.144768 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:17:35.145208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:35.146156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:17:35.146156 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:35.146156 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:17:35.146156 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:17:35.146156 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:35.146156 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:17:35.146156 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:35.146156 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:17:35.147360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:35.147360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:17:35.147360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:17:35.147360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:17:35.147360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:35.147360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:17:35.147360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:35.147360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:17:35.147360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:35.147360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:17:35.147360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:35.147360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:17:35.147360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:35.147360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:17:35.147360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:17:35.147966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:35.147966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:17:35.147966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:35.147966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:17:35.147966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:17:35.147966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:35.147966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:17:35.147966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:35.147966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:17:35.147966 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:17:35.148724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:35.148724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:17:35.148724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:35.148724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:17:35.148724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:17:35.148724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:17:35.148724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:17:35.148724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:17:35.148724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:17:35.148724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:35.148724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:35.149208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:35.149208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:17:35.149208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:35.149208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:35.149208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:35.149208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:17:35.149208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:35.149208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:17:35.149208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:17:35.149208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:35.149208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:17:35.149208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:35.149208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:17:35.149208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:17:35.149600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:35.149600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:17:35.149600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:35.149600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:17:35.149839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:35.149839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:17:35.149839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:17:35.149839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:17:35.149839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:17:35.149839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:17:35.149839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:35.149839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:17:35.149839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:17:35.149839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:17:35.149839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:17:35.149839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:17:35.149839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:35.149839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:35.149839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:17:35.149839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:17:35.149839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:17:35.150331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:17:35.150331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:17:35.150331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:17:35.150331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:35.150331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:35.150331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:35.150331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:17:35.150331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:35.150331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:35.150331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:35.150331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:17:35.150331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:35.150331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:17:35.150331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:17:35.150331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:35.150331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:17:35.150331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:35.150331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:17:35.150331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:17:35.150885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:35.150885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:17:35.150885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:35.150885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:17:35.150885 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:17:35.150885 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:17:35.150885 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:17:35.151116 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:17:35.151630 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:17:35.151858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:35.152502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:17:35.152502 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:35.152502 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:17:35.152502 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:17:35.152502 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:35.152502 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:17:35.152502 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:35.152502 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:17:35.153339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:35.153339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:17:35.153339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:35.153339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:17:35.153339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:17:35.153339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:17:35.153339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:17:35.153339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:35.153339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:35.153339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:35.153339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:17:35.153339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:35.153758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:35.153758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:35.153758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:17:35.153758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:35.153758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:17:35.153758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:17:35.153758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:35.153758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:17:35.153758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:35.153758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:17:35.153758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:17:35.153758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:35.153758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:17:35.154113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:35.154113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:17:35.154422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:35.154422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:17:35.154422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:17:35.154422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:17:35.154422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:17:35.154422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:17:35.154422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:35.154422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:17:35.154422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:17:35.154422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:17:35.154422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:17:35.154422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:17:35.154422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:35.154422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:35.154422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:17:35.154422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:17:35.155135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:17:35.155135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:17:35.155135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:35.155135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:35.155135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:35.155135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:17:35.155135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:35.155135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:35.155135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:35.155135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:17:35.155135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:35.155135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:17:35.155135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:17:35.155135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:35.155135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:17:35.155135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:35.155135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:17:35.155135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:17:35.155135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:35.155697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:17:35.155697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:35.155697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:17:39.844029 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:17:39.844796 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:17:39.844796 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:17:39.845139 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:17:39.873398 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:17:39.873584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:39.874088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:17:39.874088 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:39.874088 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:17:39.874088 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:17:39.874088 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:39.874088 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:17:39.874088 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:39.874088 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:17:39.874787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.030 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:39.874787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:17:39.874787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:39.874787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:17:39.874787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:17:39.874787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:17:39.874787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:17:39.874787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:39.874787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:39.874787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:39.874787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:17:39.874787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:39.875433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:39.875433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:39.875433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:17:39.875433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:39.875433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:17:39.875433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:17:39.875433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:39.875433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:17:39.875433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:39.875433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:17:39.875433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:17:39.875433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:39.875433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:17:39.876078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:39.876078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:17:39.876078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:39.876078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:17:39.876078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:17:39.876078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:17:39.876078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:17:39.876078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:17:39.876078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:39.876078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:17:39.876078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:17:39.876078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:17:39.876078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:17:39.876078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:17:39.876078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:39.876816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:39.876816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:17:39.876816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:17:39.876816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:17:39.876816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:17:39.876816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:39.876816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:39.876816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:39.876816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:17:39.876816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:39.876816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:39.876816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:39.876816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:17:39.876816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:39.876816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:17:39.876816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:17:39.876816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:39.876816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:17:39.876816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:39.877752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:17:39.877752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:17:39.877752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:39.877752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:17:39.877752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:39.877752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:17:40.092145 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:17:40.093003 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:17:40.093003 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:17:40.093991 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:17:40.093991 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:17:40.093991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:40.095313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:17:40.095313 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:40.095313 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:17:40.095313 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:17:40.095313 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:40.095313 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:17:40.095313 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:40.095313 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:17:40.096713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:40.096713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:17:40.096713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:40.096713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:17:40.096713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:17:40.096713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:17:40.096713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:17:40.096713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:40.096713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:40.096713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:40.096713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:17:40.096713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:40.097378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:40.097378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:40.097378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:17:40.097378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:40.097378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:17:40.097378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:17:40.097378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:40.097378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:17:40.097378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:40.097378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:17:40.097378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:17:40.097378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:40.097378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:17:40.098567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:40.098567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:17:40.098878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:40.098878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:17:40.098878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:17:40.098878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:17:40.098878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:17:40.098878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:17:40.098878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:40.098878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:17:40.098878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:17:40.098878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:17:40.098878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:17:40.098878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:17:40.098878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:40.098878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:40.098878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:17:40.098878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:17:40.099643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:17:40.099643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:17:40.099643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:40.099643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:40.099643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:40.099643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:17:40.099643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:40.099643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:40.099643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:40.099643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:17:40.099643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:40.099643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:17:40.099643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:17:40.099643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:40.099643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:17:40.099643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:40.099643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:17:40.099643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:17:40.099643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:40.100608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:17:40.100608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:40.100608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:17:40.100608 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:17:40.100608 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:17:40.100608 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:17:40.100608 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:17:40.100608 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:17:40.101165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:40.101165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:17:40.101165 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:40.101165 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:17:40.101165 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:17:40.101165 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:40.101165 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:17:40.101165 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:40.101165 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:17:40.102103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:40.102103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:17:40.102103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:17:40.102103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:17:40.102103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:40.102103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:17:40.102103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:40.102103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:17:40.102103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:40.102103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:17:40.102103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:40.102103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:17:40.102103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:40.102103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:17:40.102103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:17:40.103209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:40.103209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:17:40.103209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:40.103209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:17:40.103209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:17:40.103209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:40.103209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:17:40.103209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:40.103209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:17:40.103209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:40.103209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:17:40.103209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:40.103209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:17:40.103209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:17:40.103949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:17:40.103949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:17:40.103949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:17:40.103949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:17:40.103949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:17:40.103949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:40.103949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:40.103949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:40.103949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:17:40.103949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:40.103949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:40.103949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:40.103949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:17:40.104627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:40.104627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:17:40.104627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:17:40.104627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:40.104627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:17:40.104627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:40.104627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:17:40.104627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:17:40.104627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:40.104627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:17:40.104627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:40.104627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:17:40.105489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:40.105489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:17:40.105489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:17:40.105489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:17:40.105489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:17:40.105489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:17:40.105489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:40.105489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:17:40.105489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:17:40.105489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:17:40.105489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:17:40.105489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:17:40.105489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:40.105489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:40.105489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:17:40.105489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:17:40.106259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:17:40.106259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:17:40.106259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:17:40.106259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:17:40.106259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:17:40.106259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:40.106259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:40.106259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:40.106259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:17:40.106259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:40.106259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:40.106259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:40.106259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:17:40.106259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:40.106259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:17:40.106259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:17:40.106259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:40.106259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:17:40.106638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:40.106638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:17:40.106638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:17:40.106638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:40.106638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:17:40.106638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:40.106638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:17:40.144832 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:17:40.145290 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:17:40.145547 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:17:40.145866 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:17:40.146227 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:17:40.146439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:40.146919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:17:40.146919 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:40.146919 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:17:40.146919 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:17:40.146919 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:40.146919 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:17:40.146919 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:40.146919 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:17:40.148049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:40.148049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:17:40.148049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:17:40.148049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:17:40.148049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:40.148049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:17:40.148049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:40.148049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:17:40.148049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:40.148049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:17:40.148049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:40.148049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:17:40.148049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:40.148049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:17:40.148049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:17:40.149102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:40.149102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:17:40.149102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:40.149102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:17:40.149102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:17:40.149102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:40.149102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:17:40.149102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:40.149102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:17:40.150300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:40.150300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:17:40.150300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:40.150300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:17:40.150300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:17:40.150300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:17:40.150300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:17:40.150300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:17:40.150300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:17:40.150300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:40.150300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:40.151250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:40.151250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:17:40.151250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:40.151250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:40.151250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:40.151250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:17:40.151250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:40.151250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:17:40.151250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:17:40.151250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:40.151250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:17:40.151250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:40.151250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:17:40.151250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:17:40.151942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:40.151942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:17:40.151942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:40.151942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:17:40.151942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:40.151942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:17:40.151942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:17:40.151942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:17:40.151942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:17:40.151942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:17:40.151942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:40.151942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:17:40.151942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:17:40.151942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:17:40.153712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:17:40.153712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:17:40.153712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:40.153712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:40.153712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:17:40.153712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:17:40.153712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:17:40.153712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:17:40.153712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:17:40.153712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:17:40.153712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:40.153712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:40.153712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:40.153712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:17:40.153712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:40.153712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:40.153712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:40.153712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:17:40.153712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:40.154193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:17:40.154193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:17:40.154193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:40.154193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:17:40.154193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:40.154193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:17:40.154193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:17:40.154193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:40.154193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:17:40.154193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:40.154193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:17:40.154193 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:17:40.154193 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:17:40.154590 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:17:40.154590 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:17:40.154590 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:17:40.154590 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:17:40.154590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:40.154590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:17:40.154590 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:40.154590 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:17:40.154590 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:17:40.154870 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:40.154870 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:17:40.154870 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:40.154870 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:17:40.154870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:40.154870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:17:40.154870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:40.154870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:17:40.154870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:17:40.154870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:17:40.154870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:17:40.154870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:40.154870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:40.154870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:40.155261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:17:40.155261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:40.155261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:40.155261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:40.155261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:17:40.155261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:40.155261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:17:40.155261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:17:40.155261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:40.155261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:17:40.155261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:40.155261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:17:40.155261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:17:40.155261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:40.155679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:17:40.155679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:40.155679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:17:40.155679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:40.155679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:17:40.155679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:17:40.155679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:17:40.155679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:17:40.155679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:17:40.155679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:40.155679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:17:40.155679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:17:40.155679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:17:40.155679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:17:40.155679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:17:40.155679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:40.156092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:40.156092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:17:40.156092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:17:40.156092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:17:40.156092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:17:40.156092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:40.156092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:40.156092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:40.156092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:17:40.156092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:40.156092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:40.156092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:40.156092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:17:40.156092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:40.156092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:17:40.156092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:17:40.156092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:40.156092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:17:40.156092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:40.156613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:17:40.156613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:17:40.156613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:40.156613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:17:40.156613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:40.156613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:17:44.844792 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:17:44.845879 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:17:44.845879 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:17:44.847063 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:17:44.883087 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:17:44.883358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:44.883998 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:17:44.883998 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:44.883998 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:17:44.883998 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:17:44.883998 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:44.883998 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:17:44.883998 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:44.883998 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:17:44.884813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:44.884813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:17:44.884813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:44.884813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:17:44.884813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:17:44.884813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:17:44.884813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:17:44.884813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:44.884813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:44.884813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:44.884813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:17:44.884813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:44.885459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:44.885459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:44.885459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:17:44.885459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:44.885459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:17:44.885459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:17:44.885459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:44.885459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:17:44.885459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:44.885459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:17:44.885459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:17:44.885459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:44.885459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:17:44.886377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:44.886377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:17:44.886377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:44.886377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:17:44.886377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:17:44.886377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:17:44.886377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:17:44.886377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:17:44.886377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:44.886377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:17:44.886377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:17:44.886377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:17:44.886377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:17:44.886377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:17:44.886377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:44.887439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:44.887439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:17:44.887439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:17:44.887439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:17:44.887439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:17:44.887439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:44.887439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:44.887439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:44.887439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:17:44.887439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:44.887439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:44.887439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:44.887439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:17:44.887439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:44.887439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:17:44.887439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:17:44.887439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:44.887439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:17:44.887439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:44.888924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:17:44.888924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:17:44.888924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:44.888924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:17:44.888924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:44.888924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:17:45.092953 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:17:45.093688 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:17:45.093896 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:17:45.095137 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:17:45.095618 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:17:45.095908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:45.096904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:17:45.096904 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:45.096904 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:17:45.096904 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:17:45.096904 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:45.096904 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:17:45.096904 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:45.096904 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:17:45.098495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:45.098495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:17:45.098495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:45.098495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:17:45.098495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:17:45.098495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:17:45.098495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:17:45.098495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:45.098495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:45.098495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:45.098495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:17:45.098495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:45.099117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:45.099117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:45.099117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:17:45.099117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:45.099117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:17:45.099117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:17:45.099117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:45.099117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:17:45.099117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:45.099117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:17:45.099117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:17:45.099117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:45.099117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:17:45.099768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:45.099768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:17:45.100599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:45.100599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:17:45.100599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:17:45.100599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:17:45.100599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:17:45.100599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:17:45.100599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:45.100599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:17:45.100599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:17:45.100599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:17:45.100599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:17:45.100599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:17:45.100599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:45.100599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:45.100599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:17:45.100599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:17:45.101064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:17:45.101064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:17:45.101064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:45.101064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:45.101064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:45.101064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:17:45.101064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:45.101064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:45.101064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:45.101064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:17:45.101064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:45.101064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:17:45.101064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:17:45.101064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:45.101064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:17:45.101064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:45.101064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:17:45.101064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:17:45.101064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:45.101570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:17:45.101570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:45.101570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:17:45.101570 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:17:45.101570 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:17:45.101570 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:17:45.101570 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:17:45.102120 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:17:45.102311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:45.102831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:17:45.102831 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:45.102831 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:17:45.102831 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:17:45.102831 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:45.102831 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:17:45.102831 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:45.102831 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:17:45.103590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:45.103590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:17:45.103590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:17:45.103590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:17:45.103590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:45.103590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:17:45.103590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:45.103590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:17:45.103590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:45.103590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:17:45.103590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:45.103590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:17:45.103590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:45.103590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:17:45.103590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:17:45.104061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:45.104061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:17:45.104061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:45.104061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:17:45.104061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:17:45.104061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:45.104061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:17:45.104061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:45.104061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:17:45.104418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:45.104418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:17:45.104418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:45.104418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:17:45.104418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:17:45.104418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:17:45.104418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:17:45.104418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:17:45.104418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:17:45.104418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:17:45.104418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:45.104418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:45.104817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:45.104817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:17:45.104817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:45.104817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:45.104817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:45.104817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:17:45.104817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:45.104817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:17:45.104817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:17:45.104817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:45.104817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:17:45.104817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:45.104817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:17:45.104817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:17:45.105187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:45.105187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:17:45.105187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:45.105187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:17:45.105492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:45.105492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:17:45.105492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:17:45.105492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:17:45.105492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:17:45.105492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:17:45.105492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:45.105492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:17:45.105492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:17:45.105492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:17:45.105492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:17:45.105492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:17:45.105492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:45.105492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:45.105492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:17:45.105492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:17:45.105944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:17:45.105944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:17:45.105944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:17:45.105944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:17:45.105944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:17:45.105944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:45.105944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:45.105944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:45.105944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:17:45.105944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:45.105944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:45.105944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:45.105944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:17:45.105944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:45.105944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:17:45.105944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:17:45.105944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:45.105944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:17:45.106429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:45.106429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:17:45.106429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:17:45.106429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:45.106429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:17:45.106429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:45.106429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:17:45.145452 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:17:45.146166 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:17:45.146364 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:17:45.146727 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:17:45.147177 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:17:45.147444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:45.148091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:17:45.148091 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:45.148091 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:17:45.148091 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:17:45.148091 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:45.148091 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:17:45.148091 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:45.148091 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:17:45.149650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:45.149650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:17:45.149650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:17:45.149650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:17:45.149650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:45.149650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:17:45.149650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:45.149650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:17:45.149650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:45.149650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:17:45.149650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:45.149650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:17:45.149650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:45.149650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:17:45.149650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:17:45.151072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:45.151072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:17:45.151072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:45.151072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:17:45.151072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:17:45.151072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:45.151072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:17:45.151072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:45.151072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:17:45.151558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:45.151558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:17:45.151558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:45.151558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:17:45.151558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:17:45.151558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:17:45.151558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:17:45.151558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:17:45.151558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:17:45.151558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:45.151558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:45.152109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:45.152109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:17:45.152109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:45.152109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:45.152109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:45.152109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:17:45.152109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:45.152109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:17:45.152109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:17:45.152109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:45.152109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:17:45.152109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:45.152109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:17:45.152109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:17:45.152882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:45.152882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:17:45.152882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:45.152882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:17:45.155621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:45.155621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:17:45.155621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:17:45.155621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:17:45.155621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:17:45.155621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:17:45.155621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:45.155621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:17:45.155621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:17:45.155621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:17:45.155621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:17:45.155621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:17:45.155621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:45.155621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:45.155621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:17:45.155621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:17:45.155621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:17:45.156675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:17:45.156675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:17:45.156675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:17:45.156675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:45.156675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:45.156675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:45.156675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:17:45.156675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:45.156675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:45.156675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:45.156675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:17:45.156675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:45.156675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:17:45.156675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:17:45.156675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:45.156675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:17:45.156675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:45.156675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:17:45.156675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:17:45.157541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:45.157541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:17:45.157541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:45.157541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:17:45.157541 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:17:45.157541 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:17:45.157541 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:17:45.157541 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:17:45.158072 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:17:45.158072 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:17:45.158072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:45.158072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:17:45.158072 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:45.158072 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:17:45.158072 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:17:45.158072 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:45.158072 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:17:45.158072 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:45.158072 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:17:45.158072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:45.158792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:17:45.158792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:45.158792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:17:45.158792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:17:45.158792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:17:45.158792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:17:45.158792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:45.158792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:45.158792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:45.158792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:17:45.158792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:45.158792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:45.158792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:45.158792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:17:45.159549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:45.159549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:17:45.159549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:17:45.159549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:45.159549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:17:45.159549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:45.159549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:17:45.159549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:17:45.159549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:45.159549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:17:45.159549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:45.159549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:17:45.160206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:45.160206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:17:45.160206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:17:45.160206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:17:45.160206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:17:45.160206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:17:45.160206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:45.160206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:17:45.160206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:17:45.160206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:17:45.160206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:17:45.160206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:17:45.160206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:45.160206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:45.160206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:17:45.160206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:17:45.160945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:17:45.160945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:17:45.160945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:45.160945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:45.160945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:45.160945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:17:45.160945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:45.160945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:45.160945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:45.160945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:17:45.160945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:45.160945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:17:45.160945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:17:45.160945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:45.160945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:17:45.160945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:45.160945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:17:45.160945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:17:45.160945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:45.161797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:17:45.161797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:45.161797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:17:49.845519 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:17:49.847695 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:17:49.847695 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:17:49.847695 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:17:49.882470 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:17:49.882741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:49.883735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:17:49.883735 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:49.883735 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:17:49.883735 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:17:49.883735 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:49.883735 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:17:49.883735 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:49.883735 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:17:49.884426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:49.884426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:17:49.884426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:49.884426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:17:49.884426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:17:49.884426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:17:49.884426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:17:49.884426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:49.884426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:49.884426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:49.884426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:17:49.884426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:49.884890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:49.884890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:49.884890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:17:49.884890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:49.884890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:17:49.884890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:17:49.884890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:49.884890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:17:49.884890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:49.884890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:17:49.884890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:17:49.884890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:49.884890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:17:49.885300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:49.885300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:17:49.885497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:49.885497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:17:49.885497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:17:49.885497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:17:49.885497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:17:49.885497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:17:49.885497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:49.885497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:17:49.885497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:17:49.885497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:17:49.885497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:17:49.885497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:17:49.885497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:49.885497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:49.885497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:17:49.885497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:17:49.886494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:17:49.886494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:17:49.886494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:49.886494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:49.886494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:49.886494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:17:49.886494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:49.886494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:49.886494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:49.886494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:17:49.886494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:49.886494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:17:49.886494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:17:49.886494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:49.886494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:17:49.886494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:49.886494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:17:49.886494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:17:49.886494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:49.888761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:17:49.888761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:49.888761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:17:50.094047 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:17:50.094735 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:17:50.094937 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:17:50.096111 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:17:50.096346 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:17:50.096663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:50.097567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:17:50.097567 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:50.097567 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:17:50.097567 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:17:50.097567 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:50.097567 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:17:50.097567 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:50.097567 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:17:50.099357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:50.099357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:17:50.099357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:50.099357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:17:50.099357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:17:50.099357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:17:50.099357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:17:50.099357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:50.099357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:50.099357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:50.099357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:17:50.099357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:50.100079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:50.100079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:50.100079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:17:50.100079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:50.100079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:17:50.100079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:17:50.100079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:50.100079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:17:50.100079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:50.100079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:17:50.100079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:17:50.100079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:50.100079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:17:50.100742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:50.100742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:17:50.100742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:50.100742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:17:50.100742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:17:50.100742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:17:50.100742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:17:50.100742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:17:50.100742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:50.100742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:17:50.100742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:17:50.100742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:17:50.100742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:17:50.100742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:17:50.100742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:50.101587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:50.101587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:17:50.101587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:17:50.101587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:17:50.101587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:17:50.101587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:50.101587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:50.101587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:50.101587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:17:50.101587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:50.101587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:50.101587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:50.101587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:17:50.101587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:50.101587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:17:50.101587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:17:50.101587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:50.101587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:17:50.101587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:50.102383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:17:50.102383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:17:50.102383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:50.102383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:17:50.102383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:50.102383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:17:50.102383 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:17:50.102383 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:17:50.102383 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:17:50.102383 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:17:50.102878 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:17:50.102878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:50.103347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:17:50.103347 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:50.103347 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:17:50.103347 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:17:50.103347 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:50.103347 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:17:50.103347 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:50.103347 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:17:50.104152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:50.104152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:17:50.104152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:17:50.104152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:17:50.104152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:50.104152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:17:50.104152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:50.104152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:17:50.104152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:50.104152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:17:50.104152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:50.104152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:17:50.104152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:50.104152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:17:50.104152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:17:50.104947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:50.104947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:17:50.104947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:50.104947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:17:50.104947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:17:50.104947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:50.104947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:17:50.104947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:50.104947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:17:50.104947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:50.104947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:17:50.104947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:50.104947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:17:50.104947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:17:50.105351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:17:50.105351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:17:50.105351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:17:50.105351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:17:50.105351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:17:50.105351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:50.105351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:50.105351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:50.105351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:17:50.105351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:50.105351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:50.105351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:50.105351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:17:50.105835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:50.105835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:17:50.105835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:17:50.105835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:50.105835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:17:50.105835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:50.105835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:17:50.105835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:17:50.105835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:50.105835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:17:50.105835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:50.105835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:17:50.106155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:50.106155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:17:50.106155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:17:50.106155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:17:50.106155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:17:50.106155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:17:50.106155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:50.106155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:17:50.106155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:17:50.106155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:17:50.106155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:17:50.106155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:17:50.106155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:50.106155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:50.106155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:17:50.106155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:17:50.106584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:17:50.106584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:17:50.106584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:17:50.106584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:17:50.106584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:17:50.106584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:50.106584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:50.106584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:50.106584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:17:50.106584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:50.106584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:50.106584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:50.106584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:17:50.106584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:50.106584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:17:50.106584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:17:50.106584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:50.106584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:17:50.107081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:50.107081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:17:50.107081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:17:50.107081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:50.107081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:17:50.107081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:50.107081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:17:50.146298 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:17:50.146700 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:17:50.146897 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:17:50.147041 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:17:50.147557 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:17:50.147823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:50.148451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:17:50.148451 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:50.148451 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:17:50.148451 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:17:50.148451 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:50.148451 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:17:50.148451 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:50.148451 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:17:50.149657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:50.149657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:17:50.149657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:17:50.149657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:17:50.149657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:50.149657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:17:50.149657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:50.149657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:17:50.149657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:50.149657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:17:50.149657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:50.149657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:17:50.149657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:50.149657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:17:50.149657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:17:50.150575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:50.150575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:17:50.150575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:50.150575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:17:50.150575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:17:50.150575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:50.150575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:17:50.150575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:50.150575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:17:50.151336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:50.151336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:17:50.151336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:50.151336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:17:50.151336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:17:50.151336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:17:50.151336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:17:50.151336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:17:50.151336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:17:50.151336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:50.151336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:50.152174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:50.152174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:17:50.152174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:50.152174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:50.152174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:50.152174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:17:50.152174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:50.152174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:17:50.152174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:17:50.152174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:50.152174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:17:50.152174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:50.152174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:17:50.152174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:17:50.153308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:50.153308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:17:50.153308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:50.153308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:17:50.153308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:50.153308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:17:50.153308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:17:50.153308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:17:50.153308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:17:50.153308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:17:50.153308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:50.153308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:17:50.153308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:17:50.153308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:17:50.155916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:17:50.155916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:17:50.155916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:50.155916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:50.155916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:17:50.155916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:17:50.155916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:17:50.155916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:17:50.155916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:17:50.155916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:17:50.155916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:50.155916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:50.155916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:50.155916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:17:50.155916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:50.155916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:50.155916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:50.155916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:17:50.155916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:50.156397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:17:50.156397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:17:50.156397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:50.156397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:17:50.156397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:50.156397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:17:50.156397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:17:50.156397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:50.156397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:17:50.156397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:50.156397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:17:50.156397 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:17:50.156397 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:17:50.156791 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:17:50.156791 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:17:50.156791 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:17:50.156791 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:17:50.156791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:50.157117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:17:50.157117 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:50.157117 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:17:50.157117 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:17:50.157117 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:50.157117 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:17:50.157117 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:50.157117 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:17:50.157925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:50.157925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:17:50.157925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:50.157925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:17:50.157925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:17:50.157925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:17:50.157925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:17:50.157925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:50.157925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:50.157925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:50.157925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:17:50.157925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:50.158294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:50.158294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:50.158294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:17:50.158294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:50.158294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:17:50.158294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:17:50.158294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:50.158294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:17:50.158294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:50.158294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:17:50.158294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:17:50.158294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:50.158294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:17:50.158630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:50.158630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:17:50.158888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:50.158888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:17:50.158888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:17:50.158888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:17:50.158888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:17:50.158888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:17:50.158888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:50.158888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:17:50.158888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:17:50.158888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:17:50.158888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:17:50.158888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:17:50.158888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:50.158888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:50.158888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:17:50.158888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:17:50.159344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:17:50.159344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:17:50.159344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:50.159344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:50.159344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:50.159344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:17:50.159344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:50.159344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:50.159344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:50.159344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:17:50.159344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:50.159344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:17:50.159344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:17:50.159344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:50.159344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:17:50.159344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:50.159344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:17:50.159344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:17:50.159344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:50.159820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:17:50.159820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:50.159820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:17:54.846771 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:17:54.847625 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:17:54.847625 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:17:54.849066 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:17:54.884876 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:17:54.885062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:54.885499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:17:54.885499 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:54.885499 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:17:54.885499 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:17:54.885499 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:54.885499 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:17:54.885499 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:54.885499 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:17:54.886222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:54.886222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:17:54.886222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:54.886222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:17:54.886222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:17:54.886222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:17:54.886222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:17:54.886222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:54.886222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:54.886222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:54.886222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:17:54.886222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:54.886743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:54.886743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:54.886743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:17:54.886743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:54.886743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:17:54.886743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:17:54.886743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:54.886743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:17:54.886743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:54.886743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:17:54.886743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:17:54.886743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:54.886743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:17:54.887298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:54.887298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:17:54.887298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:54.887298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:17:54.887298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:17:54.887298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:17:54.887298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:17:54.887298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:17:54.887298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:54.887298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:17:54.887298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:17:54.887298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:17:54.887298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:17:54.887298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:17:54.887298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:54.887841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:54.887841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:17:54.887841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:17:54.887841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:17:54.887841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:17:54.887841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:54.887841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:54.887841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:54.887841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:17:54.887841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:54.887841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:54.887841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:54.887841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:17:54.887841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:54.887841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:17:54.887841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:17:54.887841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:54.887841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:17:54.887841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:54.888478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:17:54.888478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:17:54.888478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:54.888478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:17:54.888478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:54.888478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:17:55.094901 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:17:55.095432 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:17:55.095584 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:17:55.096258 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:17:55.096466 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:17:55.096586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:55.097290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:17:55.097290 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:55.097290 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:17:55.097290 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:17:55.097290 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:55.097290 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:17:55.097290 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:55.097290 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:17:55.098280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:55.098280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:17:55.098280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:55.098280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:17:55.098280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:17:55.098280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:17:55.098280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:17:55.098280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:55.098280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:55.098280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:55.098280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:17:55.098280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:55.099012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:55.099012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:55.099012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:17:55.099012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:55.099012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:17:55.099012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:17:55.099012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:55.099012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:17:55.099012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:55.099012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:17:55.099012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:17:55.099012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:55.099012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:17:55.099640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:55.099640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:17:55.099640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:55.099640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:17:55.099640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:17:55.099640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:17:55.099640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:17:55.099640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:17:55.099640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:55.099640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:17:55.099640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:17:55.099640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:17:55.099640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:17:55.099640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:17:55.099640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:55.100453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:55.100453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:17:55.100453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:17:55.100453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:17:55.100453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:17:55.100453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:55.100453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:55.100453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:55.100453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:17:55.100453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:55.100453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:55.100453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:55.100453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:17:55.100453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:55.100453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:17:55.100453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:17:55.100453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:55.100453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:17:55.100453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:55.101378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:17:55.101378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:17:55.101378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:55.101378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:17:55.101378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:55.101378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:17:55.101933 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:17:55.101933 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:17:55.102099 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:17:55.102179 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:17:55.102430 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:17:55.102628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:55.103047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:17:55.103047 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:55.103047 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:17:55.103047 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:17:55.103047 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:55.103047 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:17:55.103047 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:55.103047 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:17:55.103756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:55.103756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:17:55.103756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:17:55.103756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:17:55.103756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:55.103756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:17:55.103756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:55.103756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:17:55.103756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:55.103756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:17:55.103756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:55.103756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:17:55.103756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:55.103756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:17:55.103756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:17:55.104235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:55.104235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:17:55.104235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:55.104235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:17:55.104235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:17:55.104235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:55.104235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:17:55.104235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:55.104235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:17:55.104596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:55.104596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:17:55.104596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:55.104596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:17:55.104596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:17:55.104596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:17:55.104596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:17:55.104596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:17:55.104596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:17:55.104596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:17:55.104596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:55.104596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:55.104991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:55.104991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:17:55.104991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:55.104991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:55.104991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:55.104991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:17:55.104991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:55.104991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:17:55.104991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:17:55.104991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:55.104991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:17:55.104991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:55.104991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:17:55.104991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:17:55.105380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:55.105380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:17:55.105380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:55.105380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:17:55.105947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:55.105947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:17:55.105947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:17:55.105947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:17:55.105947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:17:55.105947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:17:55.105947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:55.105947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:17:55.105947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:17:55.105947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:17:55.105947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:17:55.105947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:17:55.105947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:55.105947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:55.105947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:17:55.105947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:17:55.106618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:17:55.106618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:17:55.106618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:17:55.106618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:17:55.106618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:17:55.106618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:55.106618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:55.106618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:55.106618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:17:55.106618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:55.106618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:55.106618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:55.106618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:17:55.106618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:55.106618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:17:55.106618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:17:55.106618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:55.106618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:17:55.107237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:55.107237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:17:55.107237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:17:55.107237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:55.107237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:17:55.107237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:55.107237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:17:55.147109 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:17:55.147805 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:17:55.148021 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:17:55.148305 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:17:55.148915 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:17:55.149308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:55.151040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:17:55.151040 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:55.151040 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:17:55.151040 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:17:55.151040 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:55.151040 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:17:55.151040 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:55.151040 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:17:55.151841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:55.151841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:17:55.151841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:17:55.151841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:17:55.151841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:55.151841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:17:55.151841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:55.151841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:17:55.151841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:55.151841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:17:55.151841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:55.151841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:17:55.151841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:55.151841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:17:55.151841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:17:55.152668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:55.152668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:17:55.152668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:55.152668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:17:55.152668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:17:55.152668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:55.152668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:17:55.152668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:55.152668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:17:55.153300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:55.153300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:17:55.153300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:55.153300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:17:55.153300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:17:55.153300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:17:55.153300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:17:55.153300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:17:55.153300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:17:55.153300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:55.153300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:55.153982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:55.153982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:17:55.153982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:55.153982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:55.153982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:55.153982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:17:55.153982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:55.153982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:17:55.153982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:17:55.153982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:55.153982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:17:55.153982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:55.153982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:17:55.153982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:17:55.154688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:55.154688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:17:55.154688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:55.154688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:17:55.154688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:55.154688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:17:55.154688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:17:55.154688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:17:55.154688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:17:55.154688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:17:55.154688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:55.154688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:17:55.154688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:17:55.154688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:17:55.155565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:17:55.155565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:17:55.155565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:55.155565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:55.155565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:17:55.155565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:17:55.155565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:17:55.155565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:17:55.155565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:17:55.155565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:17:55.155565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:55.155565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:55.155565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:55.155565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:17:55.155565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:55.155565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:55.155565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:55.155565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:17:55.155565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:55.156553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:17:55.156553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:17:55.156553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:55.156553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:17:55.156553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:55.156553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:17:55.156553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:17:55.156553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:55.156553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:17:55.156553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:55.156553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:17:55.156553 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:17:55.156553 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:17:55.157255 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:17:55.157255 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:17:55.157255 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:17:55.157255 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:17:55.157255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:55.157588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:17:55.157588 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:55.157588 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:17:55.157588 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:17:55.157588 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:55.157588 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:17:55.157588 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:55.157588 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:17:55.158971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:55.158971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:17:55.158971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:55.158971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:17:55.158971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:17:55.158971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:17:55.158971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:17:55.158971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:55.158971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:55.158971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:55.158971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:17:55.158971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:55.159735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:55.159735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:55.159735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:17:55.159735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:55.159735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:17:55.159735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:17:55.159735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:55.159735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:17:55.159735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:55.159735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:17:55.159735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:17:55.159735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:55.159735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:17:55.160107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:55.160107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:17:55.160107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:55.160107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:17:55.160107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:17:55.160107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:17:55.160107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:17:55.160107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:17:55.160107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:55.160107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:17:55.160107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:17:55.160107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:17:55.160107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:17:55.160107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:17:55.160107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:55.160571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:55.160571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:17:55.160571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:17:55.160571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:17:55.160571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:17:55.160571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:55.160571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:55.160571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:55.160571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:17:55.160571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:55.160571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:55.160571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:55.160571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:17:55.160571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:55.160571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:17:55.160571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:17:55.160571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:55.160571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:17:55.160571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:55.161132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:17:55.161132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:17:55.161132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:55.161132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:17:55.161132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:55.161132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:17:59.847321 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:17:59.848180 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:17:59.848180 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:17:59.849204 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:17:59.886437 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:17:59.886767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:59.887447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:17:59.887447 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:59.887447 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:17:59.887447 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:17:59.887447 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:59.887447 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:17:59.887447 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:59.887447 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:17:59.888468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:59.888468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:17:59.888468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:59.888468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:17:59.888468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:17:59.888468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:17:59.888468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:17:59.888468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:59.888468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:59.888468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:59.888468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:17:59.888468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:59.889140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:17:59.889140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:59.889140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:17:59.889140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:59.889140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:17:59.889140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:17:59.889140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:59.889140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:17:59.889140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:59.889140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:17:59.889140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:17:59.889140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:59.889140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:17:59.889804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:59.889804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:17:59.889804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:59.889804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:17:59.889804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:17:59.889804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:17:59.889804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:17:59.889804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:17:59.889804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:59.889804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:17:59.889804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:17:59.889804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:17:59.889804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:17:59.889804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:17:59.889804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:17:59.890587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:17:59.890587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:17:59.890587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:17:59.890587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:17:59.890587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:17:59.890587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:59.890587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:59.890587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:17:59.890587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:17:59.890587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:17:59.890587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:17:59.890587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:17:59.890587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:17:59.890587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:17:59.890587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:17:59.890587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:17:59.890587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:17:59.890587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:17:59.890587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:17:59.891508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:17:59.891508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:17:59.891508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:17:59.891508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:17:59.891508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:17:59.891508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:18:00.095540 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:18:00.095985 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:18:00.096185 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:18:00.097357 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:18:00.097841 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:18:00.098110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:00.098996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:18:00.098996 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:00.098996 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:18:00.098996 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:18:00.098996 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:00.098996 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:18:00.098996 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:00.098996 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:18:00.100568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:00.100568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:18:00.100568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:00.100568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:18:00.100568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:18:00.100568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:18:00.100568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:18:00.100568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:00.100568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:00.100568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:00.100568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:18:00.100568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:00.101157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:00.101157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:00.101157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:18:00.101157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:00.101157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:18:00.101157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:18:00.101157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:00.101157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:18:00.101157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:00.101157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:18:00.101157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:18:00.101157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:00.101157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:18:00.101768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:00.101768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:18:00.102259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:00.102259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:18:00.102259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:18:00.102259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:18:00.102259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:18:00.102259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:18:00.102259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:00.102259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:18:00.102259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:18:00.102259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:18:00.102259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:18:00.102259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:18:00.102259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:00.102259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:00.102259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:18:00.102259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:18:00.102994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:18:00.102994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:18:00.102994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:00.102994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:00.102994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:00.102994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:18:00.102994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:00.102994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:00.102994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:00.102994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:18:00.102994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:00.102994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:18:00.102994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:18:00.102994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:00.102994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:18:00.102994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:00.102994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:18:00.102994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:18:00.102994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:00.103943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:18:00.103943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:00.103943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:18:00.103943 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:18:00.103943 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:18:00.103943 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:18:00.103943 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:18:00.103943 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:18:00.104424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:00.105021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:18:00.105021 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:00.105021 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:18:00.105021 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:18:00.105021 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:00.105021 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:18:00.105021 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:00.105021 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:18:00.105707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:00.105707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:18:00.105707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:18:00.105707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:18:00.105707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:00.105707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:18:00.105707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:00.105707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:18:00.105707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:00.105707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:18:00.105707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:00.105707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:18:00.105707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:00.105707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:18:00.105707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:18:00.106310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:00.106310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:18:00.106310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:00.106310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:18:00.106310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:18:00.106310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:00.106310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:18:00.106310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:00.106310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:18:00.106593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:00.106593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:18:00.106593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:00.106593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:18:00.106593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:18:00.106593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:18:00.106593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:18:00.106593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:18:00.106593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:18:00.106593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:18:00.106593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:00.106593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:00.106930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:00.106930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:18:00.106930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:00.106930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:00.106930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:00.106930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:18:00.106930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:00.106930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:18:00.106930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:18:00.106930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:00.106930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:18:00.106930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:00.106930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:18:00.106930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:18:00.107318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:00.107318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:18:00.107318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:00.107318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:18:00.107467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:00.107467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:18:00.107467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:18:00.107467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:18:00.107467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:18:00.107467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:18:00.107467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:00.107467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:18:00.107467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:18:00.107467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:18:00.107467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:18:00.107467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:18:00.107467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:00.107467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:00.107467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:18:00.107467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:18:00.107899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:18:00.107899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:18:00.107899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:18:00.107899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:18:00.107899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:18:00.107899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:00.107899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:00.107899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:00.107899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:18:00.107899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:00.107899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:00.107899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:00.107899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:18:00.107899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:00.107899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:18:00.107899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:18:00.107899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:00.107899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:18:00.108453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:00.108453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:18:00.108453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:18:00.108453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:00.108453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:18:00.108453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:00.108453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:18:00.147690 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:18:00.148028 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:18:00.148234 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:18:00.148428 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:18:00.149071 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:18:00.149658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:00.150513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:18:00.150513 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:00.150513 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:18:00.150513 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:18:00.150513 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:00.150513 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:18:00.150513 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:00.150513 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:18:00.151687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:00.151687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:18:00.151687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:18:00.151687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:18:00.151687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:00.151687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:18:00.151687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:00.151687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:18:00.151687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:00.151687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:18:00.151687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:00.151687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:18:00.151687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:00.151687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:18:00.151687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:18:00.152394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:00.152394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:18:00.152394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:00.152394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:18:00.152394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:18:00.152394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:00.152394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:18:00.152394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:00.152394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:18:00.153119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:00.153119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:18:00.153119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:00.153119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:18:00.153119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:18:00.153119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:18:00.153119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:18:00.153119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:18:00.153119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:18:00.153119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:00.153119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:00.153522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:00.153522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:18:00.153522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:00.153522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:00.153522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:00.153522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:18:00.153522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:00.153522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:18:00.153522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:18:00.153522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:00.153522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:18:00.153522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:00.153522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:18:00.153522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:18:00.153894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:00.153894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:18:00.153894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:00.153894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:18:00.154881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:00.154881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:18:00.154881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:18:00.154881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:18:00.154881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:18:00.154881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:18:00.154881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:00.154881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:18:00.154881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:18:00.154881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:18:00.154881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:18:00.154881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:18:00.154881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:00.154881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:00.154881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:18:00.154881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:18:00.154881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:18:00.155592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:18:00.155592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:18:00.155592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:18:00.155592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:00.155592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:00.155592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:00.155592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:18:00.155592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:00.155592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:00.155592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:00.155592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:18:00.155592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:00.155592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:18:00.155592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:18:00.155592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:00.155592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:18:00.155592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:00.155592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:18:00.155592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:18:00.156038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:00.156038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:18:00.156038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:00.156038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:18:00.159094 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:18:00.159094 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:18:00.159094 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:18:00.159094 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:18:00.159094 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:18:00.159094 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:18:00.159349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:00.159645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:18:00.159645 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:00.159645 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:18:00.159645 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:18:00.159645 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:00.159645 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:18:00.159645 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:00.159645 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:18:00.160440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:00.160440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:18:00.160440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:00.160440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:18:00.160440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:18:00.160440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:18:00.160440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:18:00.160440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:00.160440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:00.160440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:00.160440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:18:00.160440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:00.160911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:00.160911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:00.160911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:18:00.160911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:00.160911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:18:00.160911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:18:00.160911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:00.160911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:18:00.160911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:00.160911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:18:00.160911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:18:00.160911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:00.160911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:18:00.161270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:00.161270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:18:00.162094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:00.162094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:18:00.162094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:18:00.162094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:18:00.162094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:18:00.162094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:18:00.162094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:00.162094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:18:00.162094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:18:00.162094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:18:00.162094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:18:00.162094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:18:00.162094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:00.162094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:00.162094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:18:00.162094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:18:00.162984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:18:00.162984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:18:00.162984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:00.162984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:00.162984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:00.162984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:18:00.162984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:00.162984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:00.162984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:00.162984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:18:00.162984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:00.162984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:18:00.162984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:18:00.162984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:00.162984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:18:00.162984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:00.162984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:18:00.162984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:18:00.162984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:00.163527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:18:00.163527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:00.163527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:18:04.848046 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:18:04.849087 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:18:04.849087 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:18:04.849957 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:18:04.884179 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:18:04.884489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:04.885182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:18:04.885182 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:04.885182 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:18:04.885182 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:18:04.885182 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:04.885182 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:18:04.885182 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:04.885182 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:18:04.885996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:04.885996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:18:04.885996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:04.885996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:18:04.885996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:18:04.885996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:18:04.885996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:18:04.885996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:04.885996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:04.885996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:04.885996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:18:04.885996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:04.886886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:04.886886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:04.886886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:18:04.886886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:04.886886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:18:04.886886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:18:04.886886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:04.886886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:18:04.886886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:04.886886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:18:04.886886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:18:04.886886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:04.886886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:18:04.887832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:04.887832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:18:04.887832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:04.887832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:18:04.887832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:18:04.887832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:18:04.887832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:18:04.887832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:18:04.887832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:04.887832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:18:04.887832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:18:04.887832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:18:04.887832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:18:04.887832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:18:04.887832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:04.888644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:04.888644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:18:04.888644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:18:04.888644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:18:04.888644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:18:04.888644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:04.888644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:04.888644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:04.888644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:18:04.888644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:04.888644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:04.888644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:04.888644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:18:04.888644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:04.888644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:18:04.888644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:18:04.888644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:04.888644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:18:04.888644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:04.889332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:18:04.889332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:18:04.889332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:04.889332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:18:04.889332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:04.889332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:18:05.097382 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:18:05.097960 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:18:05.098261 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:18:05.099421 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:18:05.099875 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:18:05.100213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:05.102072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:18:05.102072 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:05.102072 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:18:05.102072 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:18:05.102072 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:05.102072 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:18:05.102072 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:05.102072 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:18:05.102895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:05.102895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:18:05.102895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:05.102895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:18:05.102895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:18:05.102895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:18:05.102895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:18:05.102895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:05.102895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:05.102895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:05.102895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:18:05.102895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:05.103323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:05.103323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:05.103323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:18:05.103323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:05.103323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:18:05.103323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:18:05.103323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:05.103323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:18:05.103323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:05.103323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:18:05.103323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:18:05.103323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:05.103323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:18:05.103686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:05.103686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:18:05.105126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:05.105126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:18:05.105126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:18:05.105126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:18:05.105126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:18:05.105126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:18:05.105126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:05.105126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:18:05.105126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:18:05.105126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:18:05.105126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:18:05.105126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:18:05.105126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:05.105126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:05.105126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:18:05.105126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:18:05.105588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:18:05.105588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:18:05.105588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:05.105588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:05.105588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:05.105588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:18:05.105588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:05.105588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:05.105588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:05.105588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:18:05.105588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:05.105588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:18:05.105588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:18:05.105588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:05.105588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:18:05.105588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:05.105588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:18:05.105588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:18:05.105588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:05.106099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:18:05.106099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:05.106099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:18:05.106099 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:18:05.106099 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:18:05.106099 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:18:05.106294 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:18:05.106705 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:18:05.106921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:05.107528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:18:05.107528 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:05.107528 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:18:05.107528 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:18:05.107528 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:05.107528 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:18:05.107528 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:05.107528 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:18:05.108762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:05.108762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:18:05.108762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:18:05.108762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:18:05.108762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:05.108762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:18:05.108762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:05.108762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:18:05.108762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:05.108762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:18:05.108762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:05.108762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:18:05.108762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:05.108762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:18:05.108762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:18:05.109228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:05.109228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:18:05.109228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:05.109228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:18:05.109228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:18:05.109228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:05.109228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:18:05.109228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:05.109228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:18:05.110108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:05.110108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:18:05.110108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:05.110108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:18:05.110108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:18:05.110108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:18:05.110108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:18:05.110108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:18:05.110108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:18:05.110108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:18:05.110108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:05.110108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:05.110516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:05.110516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:18:05.110516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:05.110516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:05.110516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:05.110516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:18:05.110516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:05.110516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:18:05.110516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:18:05.110516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:05.110516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:18:05.110516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:05.110516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:18:05.110516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:18:05.110851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:05.110851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:18:05.110851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:05.110851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:18:05.112376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:05.112376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:18:05.112376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:18:05.112376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:18:05.112376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:18:05.112376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:18:05.112376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:05.112376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:18:05.112376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:18:05.112376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:18:05.112376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:18:05.112376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:18:05.112376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:05.112376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:05.112376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:18:05.112376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:18:05.112995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:18:05.112995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:18:05.112995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:18:05.112995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:18:05.112995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:18:05.112995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:05.112995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:05.112995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:05.112995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:18:05.112995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:05.112995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:05.112995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:05.112995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:18:05.112995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:05.112995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:18:05.112995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:18:05.112995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:05.112995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:18:05.113533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:05.113533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:18:05.113533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:18:05.113533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:05.113533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:18:05.113533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:05.113533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:18:05.148161 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:18:05.148658 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:18:05.148825 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:18:05.149011 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:18:05.149622 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:18:05.150179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:05.151493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:18:05.151493 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:05.151493 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:18:05.151493 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:18:05.151493 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:05.151493 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:18:05.151493 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:05.151493 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:18:05.152632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:05.152632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:18:05.152632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:18:05.152632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:18:05.152632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:05.152632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:18:05.152632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:05.152632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:18:05.152632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:05.152632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:18:05.152632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:05.152632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:18:05.152632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:05.152632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:18:05.152632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:18:05.153279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:05.153279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:18:05.153279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:05.153279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:18:05.153279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:18:05.153279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:05.153279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:18:05.153279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:05.153279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:18:05.153979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:05.153979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:18:05.153979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:05.153979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:18:05.153979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:18:05.153979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:18:05.153979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:18:05.153979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:18:05.153979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:18:05.153979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:05.153979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:05.154490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:05.154490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:18:05.154490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:05.154490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:05.154490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:05.154490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:18:05.154490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:05.154490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:18:05.154490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:18:05.154490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:05.154490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:18:05.154490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:05.154490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:18:05.154490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:18:05.155075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:05.155075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:18:05.155075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:05.155075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:18:05.155662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:05.155662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:18:05.155662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:18:05.155662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:18:05.155662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:18:05.155662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:18:05.155662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:05.155662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:18:05.155662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:18:05.155662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:18:05.155662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:18:05.155662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:18:05.155662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:05.155662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:05.155662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:18:05.155662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:18:05.155662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:18:05.156243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:18:05.156243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:18:05.156243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:18:05.156243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:05.156243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:05.156243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:05.156243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:18:05.156243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:05.156243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:05.156243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:05.156243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:18:05.156243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:05.156243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:18:05.156243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:18:05.156243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:05.156243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:18:05.156243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:05.156243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:18:05.156243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:18:05.156799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:05.156799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:18:05.156799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:05.156799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:18:05.158164 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:18:05.158497 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:18:05.158653 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:18:05.158868 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:18:05.159133 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:18:05.159336 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:18:05.159456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:05.159950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:18:05.159950 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:05.159950 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:18:05.159950 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:18:05.159950 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:05.159950 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:18:05.159950 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:05.159950 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:18:05.160948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:05.160948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:18:05.160948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:05.160948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:18:05.160948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:18:05.160948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:18:05.160948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:18:05.160948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:05.160948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:05.160948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:05.160948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:18:05.160948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:05.161543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:05.161543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:05.161543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:18:05.161543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:05.161543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:18:05.161543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:18:05.161543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:05.161543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:18:05.161543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:05.161543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:18:05.161543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:18:05.161543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:05.161543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:18:05.162132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:05.162132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:18:05.162427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:05.162427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:18:05.162427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:18:05.162427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:18:05.162427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:18:05.162427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:18:05.162427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:05.162427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:18:05.162427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:18:05.162427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:18:05.162427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:18:05.162427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:18:05.162427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:05.162427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:05.162427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:18:05.162427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:18:05.162949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:18:05.162949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:18:05.162949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:05.162949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:05.162949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:05.162949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:18:05.162949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:05.162949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:05.162949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:05.162949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:18:05.162949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:05.162949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:18:05.162949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:18:05.162949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:05.162949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:18:05.162949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:05.162949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:18:05.162949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:18:05.162949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:05.163672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:18:05.163672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:05.163672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:18:09.848796 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:18:09.849310 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:18:09.849614 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:18:09.850718 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:18:09.889068 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:18:09.889503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:09.890347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:18:09.890347 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:09.890347 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:18:09.890347 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:18:09.890347 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:09.890347 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:18:09.890347 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:09.890347 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:18:09.891243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:09.891243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:18:09.891243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:09.891243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:18:09.891243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:18:09.891243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:18:09.891243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:18:09.891243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:09.891243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:09.891243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:09.891243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:18:09.891243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:09.892118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:09.892118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:09.892118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:18:09.892118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:09.892118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:18:09.892118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:18:09.892118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:09.892118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:18:09.892118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:09.892118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:18:09.892118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:18:09.892118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:09.892118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:18:09.893437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:09.893437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:18:09.893437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:09.893437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:18:09.893437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:18:09.893437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:18:09.893437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:18:09.893437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:18:09.893437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:09.893437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:18:09.893437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:18:09.893437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:18:09.893437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:18:09.893437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:18:09.893437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:09.894966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:09.894966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:18:09.894966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:18:09.894966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:18:09.894966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:18:09.894966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:09.894966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:09.894966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:09.894966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:18:09.894966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:09.894966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:09.894966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:09.894966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:18:09.894966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:09.894966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:18:09.894966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:18:09.894966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:09.894966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:18:09.894966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:09.895379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:18:09.895379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:18:09.895379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:09.895379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:18:09.895379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:09.895379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:18:10.098028 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:18:10.098668 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:18:10.098903 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:18:10.100540 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:18:10.100540 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:18:10.100540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:10.101386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:18:10.101386 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:10.101386 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:18:10.101386 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:18:10.101386 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:10.101386 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:18:10.101386 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:10.101386 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:18:10.103032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:10.103032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:18:10.103032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:10.103032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:18:10.103032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:18:10.103032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:18:10.103032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:18:10.103032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:10.103032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:10.103032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:10.103032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:18:10.103032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:10.103517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:10.103517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:10.103517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:18:10.103517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:10.103517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:18:10.103517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:18:10.103517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:10.103517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:18:10.103517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:10.103517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:18:10.103517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:18:10.103517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:10.103517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:18:10.103910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:10.103910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:18:10.104877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:10.104877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:18:10.104877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:18:10.104877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:18:10.104877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:18:10.104877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:18:10.104877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:10.104877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:18:10.104877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:18:10.104877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:18:10.104877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:18:10.104877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:18:10.104877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:10.104877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:10.104877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:18:10.104877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:18:10.105372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:18:10.105372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:18:10.105372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:10.105372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:10.105372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:10.105372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:18:10.105372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:10.105372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:10.105372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:10.105372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:18:10.105372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:10.105372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:18:10.105372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:18:10.105372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:10.105372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:18:10.105372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:10.105372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:18:10.105372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:18:10.105372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:10.105969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:18:10.105969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:10.105969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:18:10.105969 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:18:10.106311 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:18:10.106636 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:18:10.106702 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:18:10.107116 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:18:10.107223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:10.107947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:18:10.107947 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:10.107947 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:18:10.107947 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:18:10.107947 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:10.107947 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:18:10.107947 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:10.107947 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:18:10.108917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:10.108917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:18:10.108917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:18:10.108917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:18:10.108917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:10.108917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:18:10.108917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:10.108917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:18:10.108917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:10.108917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:18:10.108917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:10.108917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:18:10.108917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:10.108917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:18:10.108917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:18:10.109396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:10.109396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:18:10.109396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:10.109396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:18:10.109396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:18:10.109396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:10.109396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:18:10.109396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:10.109396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:18:10.109720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:10.109720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:18:10.109720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:10.109720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:18:10.109720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:18:10.109720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:18:10.109720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:18:10.109720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:18:10.109720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:18:10.109720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:18:10.109720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:10.109720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:10.110111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:10.110111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:18:10.110111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:10.110111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:10.110111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:10.110111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:18:10.110111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:10.110111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:18:10.110111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:18:10.110111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:10.110111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:18:10.110111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:10.110111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:18:10.110111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:18:10.110524 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:10.110524 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:18:10.110524 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:10.110524 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:18:10.110893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:10.110893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:18:10.110893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:18:10.110893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:18:10.110893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:18:10.110893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:18:10.110893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:10.110893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:18:10.110893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:18:10.110893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:18:10.110893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:18:10.110893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:18:10.110893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:10.110893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:10.110893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:18:10.110893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:18:10.111340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:18:10.111340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:18:10.111340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:18:10.111340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:18:10.111340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:18:10.111340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:10.111340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:10.111340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:10.111340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:18:10.111340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:10.111340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:10.111340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:10.111340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:18:10.111340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:10.111340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:18:10.111340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:18:10.111340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:10.111340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:18:10.111884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:10.111884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:18:10.111884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:18:10.111884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:10.111884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:18:10.111884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:10.111884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:18:10.148743 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:18:10.149135 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:18:10.149352 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:18:10.149601 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:18:10.150261 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:18:10.150712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:10.152322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:18:10.152322 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:10.152322 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:18:10.152322 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:18:10.152322 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:10.152322 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:18:10.152322 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:10.152322 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:18:10.153580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:10.153580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:18:10.153580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:18:10.153580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:18:10.153580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:10.153580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:18:10.153580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:10.153580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:18:10.153580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:10.153580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:18:10.153580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:10.153580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:18:10.153580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:10.153580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:18:10.153580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:18:10.154178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:10.154178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:18:10.154178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:10.154178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:18:10.154178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:18:10.154178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:10.154178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:18:10.154178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:10.154178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:18:10.155079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:10.155079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:18:10.155079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:10.155079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:18:10.155079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:18:10.155079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:18:10.155079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:18:10.155079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:18:10.155079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:18:10.155079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:10.155079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:10.155469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:10.155469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:18:10.155469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:10.155469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:10.155469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:10.155469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:18:10.155469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:10.155469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:18:10.155469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:18:10.155469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:10.155469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:18:10.155469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:10.155469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:18:10.155469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:18:10.156121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:10.156121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:18:10.156121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:10.156121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:18:10.156839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:10.156839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:18:10.156839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:18:10.156839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:18:10.156839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:18:10.156839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:18:10.156839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:10.156839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:18:10.156839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:18:10.156839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:18:10.156839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:18:10.156839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:18:10.156839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:10.156839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:10.156839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:18:10.156839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:18:10.156839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:18:10.157375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:18:10.157375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:18:10.157375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:18:10.157375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:10.157375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:10.157375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:10.157375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:18:10.157375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:10.157375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:10.157375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:10.157375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:18:10.157375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:10.157375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:18:10.157375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:18:10.157375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:10.157375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:18:10.157375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:10.157375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:18:10.157375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:18:10.157886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:10.157886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:18:10.157886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:10.157886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:18:10.158688 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:18:10.159002 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:18:10.159112 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:18:10.159265 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:18:10.159862 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:18:10.160257 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:18:10.160575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:10.161240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:18:10.161240 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:10.161240 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:18:10.161240 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:18:10.161240 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:10.161240 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:18:10.161240 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:10.161240 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:18:10.162126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:10.162126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:18:10.162126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:10.162126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:18:10.162126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:18:10.162126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:18:10.162126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:18:10.162126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:10.162126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:10.162126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:10.162126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:18:10.162126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:10.162578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:10.162578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:10.162578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:18:10.162578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:10.162578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:18:10.162578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:18:10.162578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:10.162578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:18:10.162578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:10.162578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:18:10.162578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:18:10.162578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:10.162578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:18:10.163079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:10.163079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:18:10.163079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:10.163079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:18:10.163079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:18:10.163079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:18:10.163079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:18:10.163079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:18:10.163079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:10.163079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:18:10.163079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:18:10.163079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:18:10.163079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:18:10.163079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:18:10.163079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:10.163610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:10.163610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:18:10.163610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:18:10.163610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:18:10.163610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:18:10.163610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:10.163610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:10.163610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:10.163610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:18:10.163610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:10.163610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:10.163610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:10.163610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:18:10.163610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:10.163610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:18:10.163610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:18:10.163610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:10.163610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:18:10.163610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:10.164140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:18:10.164140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:18:10.164140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:10.164140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:18:10.164140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:10.164140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:18:14.849290 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:18:14.849958 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:18:14.850011 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:18:14.851222 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:18:14.881814 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:18:14.882233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:14.883201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:18:14.883201 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:14.883201 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:18:14.883201 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:18:14.883201 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:14.883201 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:18:14.883201 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:14.883201 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:18:14.883858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:14.883858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:18:14.883858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:14.883858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:18:14.883858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:18:14.883858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:18:14.883858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:18:14.883858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:14.883858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:14.883858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:14.883858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:18:14.883858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:14.884258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:14.884258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:14.884258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:18:14.884258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:14.884258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:18:14.884258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:18:14.884258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:14.884258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:18:14.884258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:14.884258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:18:14.884258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:18:14.884258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:14.884258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:18:14.884740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:14.884740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:18:14.884996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:14.884996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:18:14.884996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:18:14.884996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:18:14.884996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:18:14.884996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:18:14.884996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:14.884996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:18:14.884996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:18:14.884996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:18:14.884996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:18:14.884996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:18:14.884996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:14.884996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:14.884996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:18:14.884996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:18:14.885907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:18:14.885907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:18:14.885907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:14.885907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:14.885907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:14.885907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:18:14.885907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:14.885907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:14.885907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:14.885907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:18:14.885907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:14.885907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:18:14.885907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:18:14.885907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:14.885907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:18:14.885907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:14.885907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:18:14.885907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:18:14.885907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:14.887177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:18:14.887177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:14.887177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:18:15.098681 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:18:15.099461 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:18:15.099626 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:18:15.100982 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:18:15.101374 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:18:15.101760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:15.102934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:18:15.102934 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:15.102934 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:18:15.102934 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:18:15.102934 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:15.102934 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:18:15.102934 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:15.102934 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:18:15.104786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:15.104786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:18:15.104786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:15.104786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:18:15.104786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:18:15.104786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:18:15.104786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:18:15.104786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:15.104786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:15.104786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:15.104786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:18:15.104786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:15.105530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:15.105530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:15.105530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:18:15.105530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:15.105530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:18:15.105530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:18:15.105530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:15.105530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:18:15.105530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:15.105530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:18:15.105530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:18:15.105530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:15.105530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:18:15.106146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:15.106146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:18:15.106731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:15.106731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:18:15.106731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:18:15.106731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:18:15.106731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:18:15.106731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:18:15.106731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:15.106731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:18:15.106731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:18:15.106731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:18:15.106731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:18:15.106731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:18:15.106731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:15.106731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:15.106731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:18:15.106731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:18:15.107073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:18:15.107073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:18:15.107073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:15.107073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:15.107073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:15.107073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:18:15.107073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:15.107073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:15.107073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:15.107073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:18:15.107073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:15.107073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:18:15.107073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:18:15.107073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:15.107073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:18:15.107073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:15.107073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:18:15.107073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:18:15.107073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:15.107490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:18:15.107490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:15.107490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:18:15.107847 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:18:15.108131 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:18:15.108245 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:18:15.108363 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:18:15.108733 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:18:15.108921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:15.109356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:18:15.109356 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:15.109356 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:18:15.109356 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:18:15.109356 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:15.109356 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:18:15.109356 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:15.109356 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:18:15.110117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:15.110117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:18:15.110117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:18:15.110117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:18:15.110117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:15.110117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:18:15.110117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:15.110117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:18:15.110117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:15.110117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:18:15.110117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:15.110117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:18:15.110117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:15.110117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:18:15.110117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:18:15.110439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:15.110439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:18:15.110439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:15.110439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:18:15.110439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:18:15.110439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:15.110439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:18:15.110439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:15.110439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:18:15.110996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:15.110996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:18:15.110996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:15.110996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:18:15.110996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:18:15.110996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:18:15.110996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:18:15.110996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:18:15.110996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:18:15.110996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:18:15.110996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:15.110996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:15.111252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:15.111252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:18:15.111252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:15.111252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:15.111252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:15.111252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:18:15.111252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:15.111252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:18:15.111252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:18:15.111252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:15.111252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:18:15.111252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:15.111252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:18:15.111252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:18:15.111542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:15.111542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:18:15.111542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:15.111542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:18:15.111994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:15.111994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:18:15.111994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:18:15.111994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:18:15.111994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:18:15.111994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:18:15.111994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:15.111994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:18:15.111994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:18:15.111994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:18:15.111994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:18:15.111994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:18:15.111994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:15.111994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:15.111994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:18:15.111994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:18:15.112357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:18:15.112357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:18:15.112357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:18:15.112357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:18:15.112357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:18:15.112357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:15.112357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:15.112357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:15.112357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:18:15.112357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:15.112357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:15.112357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:15.112357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:18:15.112357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:15.112357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:18:15.112357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:18:15.112357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:15.112357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:18:15.112915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:15.112915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:18:15.112915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:18:15.112915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:15.112915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:18:15.112915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:15.112915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:18:15.149233 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:18:15.149596 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:18:15.149689 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:18:15.150290 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:18:15.150290 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:18:15.150603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:15.153114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:18:15.153114 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:15.153114 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:18:15.153114 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:18:15.153114 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:15.153114 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:18:15.153114 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:15.153114 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:18:15.154540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:15.154540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:18:15.154540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:18:15.154540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:18:15.154540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:15.154540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:18:15.154540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:15.154540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:18:15.154540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:15.154540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:18:15.154540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:15.154540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:18:15.154540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:15.154540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:18:15.154540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:18:15.155202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:15.155202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:18:15.155202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:15.155202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:18:15.155202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:18:15.155202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:15.155202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:18:15.155202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:15.155202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:18:15.155962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:15.155962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:18:15.155962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:15.155962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:18:15.155962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:18:15.155962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:18:15.155962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:18:15.155962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:18:15.155962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:18:15.155962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:15.155962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:15.156501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:15.156501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:18:15.156501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:15.156501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:15.156501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:15.156501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:18:15.156501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:15.156501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:18:15.156501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:18:15.156501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:15.156501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:18:15.156501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:15.156501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:18:15.156501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:18:15.157192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:15.157192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:18:15.157192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:15.157192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:18:15.157782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:15.157782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:18:15.157782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:18:15.157782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:18:15.157782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:18:15.157782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:18:15.157782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:15.157782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:18:15.157782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:18:15.157782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:18:15.157782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:18:15.157782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:18:15.157782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:15.157782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:15.157782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:18:15.157782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:18:15.157782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:18:15.158677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:18:15.158677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:18:15.158677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:18:15.158677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:15.158677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:15.158677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:15.158677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:18:15.158677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:15.158677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:15.158677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:15.158677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:18:15.158677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:15.158677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:18:15.158677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:18:15.158677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:15.158677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:18:15.158677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:15.158677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:18:15.158677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:18:15.159747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:15.159747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:18:15.159747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:15.159747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:18:15.159747 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:18:15.159747 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:18:15.160068 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:18:15.160133 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:18:15.160508 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:18:15.160754 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:18:15.160896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:15.161482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:18:15.161482 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:15.161482 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:18:15.161482 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:18:15.161482 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:15.161482 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:18:15.161482 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:15.161482 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:18:15.162435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:15.162435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:18:15.162435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:15.162435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:18:15.162435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:18:15.162435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:18:15.162435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:18:15.162435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:15.162435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:15.162435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:15.162435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:18:15.162435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:15.162790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:15.162790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:15.162790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:18:15.162790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:15.162790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:18:15.162790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:18:15.162790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:15.162790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:18:15.162790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:15.162790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:18:15.162790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:18:15.162790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:15.162790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:18:15.163149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:15.163149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:18:15.163593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:15.163593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:18:15.163593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:18:15.163593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:18:15.163593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:18:15.163593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:18:15.163593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:15.163593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:18:15.163593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:18:15.163593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:18:15.163593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:18:15.163593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:18:15.163593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:15.163593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:15.163593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:18:15.163593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:18:15.164032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:18:15.164032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:18:15.164032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:15.164032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:15.164032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:15.164032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:18:15.164032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:15.164032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:15.164032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:15.164032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:18:15.164032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:15.164032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:18:15.164032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:18:15.164032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:15.164032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:18:15.164032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:15.164032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:18:15.164032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:18:15.164032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:15.164665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:18:15.164665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:15.164665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:18:19.850024 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:18:19.850646 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:18:19.850752 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:18:19.852092 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:18:19.891000 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:18:19.891000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:19.891231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:18:19.891231 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:19.891231 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:18:19.891231 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:18:19.891231 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:19.891231 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:18:19.891231 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:19.891231 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:18:19.891886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:19.891886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:18:19.891886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:19.891886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:18:19.891886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:18:19.891886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:18:19.891886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:18:19.891886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:19.891886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:19.891886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:19.891886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:18:19.891886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:19.892397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:19.892397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:19.892397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:18:19.892397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:19.892397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:18:19.892397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:18:19.892397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:19.892397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:18:19.892397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:19.892397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:18:19.892397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:18:19.892397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:19.892397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:18:19.892953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:19.892953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:18:19.892953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:19.892953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:18:19.892953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:18:19.892953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:18:19.892953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:18:19.892953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:18:19.892953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:19.892953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:18:19.892953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:18:19.892953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:18:19.892953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:18:19.892953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:18:19.892953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:19.893554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:19.893554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:18:19.893554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:18:19.893554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:18:19.893554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:18:19.893554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:19.893554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:19.893554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:19.893554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:18:19.893554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:19.893554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:19.893554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:19.893554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:18:19.893554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:19.893554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:18:19.893554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:18:19.893554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:19.893554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:18:19.893554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:19.894323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:18:19.894323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:18:19.894323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:19.894323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:18:19.894323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:19.894323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:18:20.099198 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:18:20.099612 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:18:20.099820 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:18:20.100608 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:18:20.100740 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:18:20.100923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:20.101786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:18:20.101786 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:20.101786 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:18:20.101786 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:18:20.101786 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:20.101786 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:18:20.101786 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:20.101786 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:18:20.103710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:20.103710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:18:20.103710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:20.103710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:18:20.103710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:18:20.103710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:18:20.103710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:18:20.103710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:20.103710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:20.103710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:20.103710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:18:20.103710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:20.104261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:20.104261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:20.104261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:18:20.104261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:20.104261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:18:20.104261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:18:20.104261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:20.104261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:18:20.104261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:20.104261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:18:20.104261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:18:20.104261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:20.104261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:18:20.104754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:20.104754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:18:20.104754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:20.104754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:18:20.104754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:18:20.104754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:18:20.104754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:18:20.104754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:18:20.104754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:20.104754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:18:20.104754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:18:20.104754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:18:20.104754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:18:20.104754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:18:20.104754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:20.105226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:20.105226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:18:20.105226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:18:20.105226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:18:20.105226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:18:20.105226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:20.105226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:20.105226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:20.105226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:18:20.105226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:20.105226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:20.105226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:20.105226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:18:20.105226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:20.105226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:18:20.105226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:18:20.105226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:20.105226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:18:20.105226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:20.106323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:18:20.106323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:18:20.106323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:20.106323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:18:20.106323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:20.106323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:18:20.108182 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:18:20.108372 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:18:20.108473 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:18:20.108563 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:18:20.109082 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:18:20.109663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:20.110428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:18:20.110428 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:20.110428 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:18:20.110428 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:18:20.110428 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:20.110428 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:18:20.110428 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:20.110428 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:18:20.110797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:20.110797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:18:20.110797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:18:20.110797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:18:20.110797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:20.110797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:18:20.110797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:20.110797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:18:20.110797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:20.110797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:18:20.110797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:20.110797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:18:20.110797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:20.110797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:18:20.110797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:18:20.111680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:20.111680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:18:20.111680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:20.111680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:18:20.111680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:18:20.111680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:20.111680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:18:20.111680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:20.111680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:18:20.111680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:20.111680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:18:20.111680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:20.111680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:18:20.111680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:18:20.112290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:18:20.112290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:18:20.112290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:18:20.112290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:18:20.112290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:18:20.112290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:20.112290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:20.112290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:20.112290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:18:20.112290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:20.112290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:20.112290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:20.112290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:18:20.112888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:20.112888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:18:20.112888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:18:20.112888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:20.112888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:18:20.112888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:20.112888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:18:20.112888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:18:20.112888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:20.112888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:18:20.112888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:20.112888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:18:20.113614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:20.113614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:18:20.113614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:18:20.113614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:18:20.113614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:18:20.113614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:18:20.113614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:20.113614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:18:20.113614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:18:20.113614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:18:20.113614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:18:20.113614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:18:20.113614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:20.113614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:20.113614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:18:20.113614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:18:20.114636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:18:20.114636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:18:20.114636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:18:20.114636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:18:20.114636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:18:20.114636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:20.114636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:20.114636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:20.114636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:18:20.114636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:20.114636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:20.114636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:20.114636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:18:20.114636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:20.114636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:18:20.114636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:18:20.114636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:20.114636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:18:20.115873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:20.115873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:18:20.115873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:18:20.115873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:20.115873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:18:20.115873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:20.115873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:18:20.149921 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:18:20.150639 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:18:20.150873 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:18:20.151304 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:18:20.152062 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:18:20.152476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:20.153276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:18:20.153276 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:20.153276 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:18:20.153276 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:18:20.153276 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:20.153276 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:18:20.153276 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:20.153276 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:18:20.154538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:20.154538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:18:20.154538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:18:20.154538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:18:20.154538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:20.154538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:18:20.154538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:20.154538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:18:20.154538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:20.154538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:18:20.154538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:20.154538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:18:20.154538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:20.154538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:18:20.154538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:18:20.155297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:20.155297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:18:20.155297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:20.155297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:18:20.155297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:18:20.155297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:20.155297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:18:20.155297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:20.155297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:18:20.156033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:20.156033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:18:20.156033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:20.156033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:18:20.156033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:18:20.156033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:18:20.156033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:18:20.156033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:18:20.156033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:18:20.156033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:20.156033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:20.156602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:20.156602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:18:20.156602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:20.156602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:20.156602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:20.156602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:18:20.156602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:20.156602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:18:20.156602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:18:20.156602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:20.156602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:18:20.156602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:20.156602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:18:20.156602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:18:20.157226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:20.157226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:18:20.157226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:20.157226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:18:20.157226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:20.157226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:18:20.157226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:18:20.157226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:18:20.157226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:18:20.157226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:18:20.157226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:20.157226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:18:20.157226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:18:20.157226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:18:20.157858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:18:20.157858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:18:20.157858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:20.157858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:20.157858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:18:20.157858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:18:20.157858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:18:20.157858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:18:20.157858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:18:20.157858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:18:20.157858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:20.157858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:20.157858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:20.157858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:18:20.157858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:20.157858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:20.157858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:20.157858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:18:20.157858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:20.158715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:18:20.158715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:18:20.158715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:20.158715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:18:20.158715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:20.158715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:18:20.158715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:18:20.158715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:20.158715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:18:20.158715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:20.158715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:18:20.159804 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:18:20.160045 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:18:20.160122 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:18:20.160274 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:18:20.160915 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:18:20.161398 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:18:20.161743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:20.162682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:18:20.162682 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:20.162682 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:18:20.162682 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:18:20.162682 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:20.162682 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:18:20.162682 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:20.162682 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:18:20.163600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:20.163600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:18:20.163600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:20.163600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:18:20.163600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:18:20.163600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:18:20.163600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:18:20.163600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:20.163600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:20.163600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:20.163600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:18:20.163600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:20.164005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:20.164005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:20.164005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:18:20.164005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:20.164005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:18:20.164005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:18:20.164005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:20.164005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:18:20.164005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:20.164005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:18:20.164005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:18:20.164005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:20.164005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:18:20.164387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:20.164387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:18:20.164387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:20.164387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:18:20.164387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:18:20.164387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:18:20.164387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:18:20.164387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:18:20.164387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:20.164387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:18:20.164387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:18:20.164387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:18:20.164387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:18:20.164387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:18:20.164387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:20.164859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:20.164859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:18:20.164859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:18:20.164859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:18:20.164859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:18:20.164859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:20.164859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:20.164859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:20.164859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:18:20.164859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:20.164859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:20.164859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:20.164859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:18:20.164859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:20.164859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:18:20.164859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:18:20.164859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:20.164859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:18:20.164859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:20.165382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:18:20.165382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:18:20.165382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:20.165382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:18:20.165382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:20.165382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:18:24.850663 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:18:24.851557 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:18:24.851557 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:18:24.852766 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:18:24.858922 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG dbcounter [-] [105581] Writing DB stats neutron:SELECT=240 {{(pid=105581) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 14:18:24.884151 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:18:24.884489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:24.885301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:18:24.885301 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:24.885301 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:18:24.885301 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:18:24.885301 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:24.885301 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:18:24.885301 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:24.885301 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:18:24.886066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:24.886066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:18:24.886066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:24.886066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:18:24.886066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:18:24.886066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:18:24.886066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:18:24.886066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:24.886066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:24.886066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:24.886066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:18:24.886066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:24.886773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:24.886773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:24.886773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:18:24.886773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:24.886773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:18:24.886773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:18:24.886773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:24.886773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:18:24.886773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:24.886773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:18:24.886773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:18:24.886773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:24.886773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:18:24.887431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:24.887431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:18:24.887431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:24.887431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:18:24.887431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:18:24.887431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:18:24.887431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:18:24.887431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:18:24.887431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:24.887431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:18:24.887431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:18:24.887431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:18:24.887431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:18:24.887431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:18:24.887431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:24.888195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:24.888195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:18:24.888195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:18:24.888195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:18:24.888195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:18:24.888195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:24.888195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:24.888195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:24.888195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:18:24.888195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:24.888195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:24.888195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:24.888195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:18:24.888195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:24.888195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:18:24.888195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:18:24.888195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:24.888195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:18:24.888195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:24.889267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:18:24.889267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:18:24.889267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:24.889267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:18:24.889267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:24.889267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:18:25.099854 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:18:25.100479 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:18:25.100678 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:18:25.101775 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:18:25.102265 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:18:25.102588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:25.103448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:18:25.103448 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:25.103448 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:18:25.103448 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:18:25.103448 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:25.103448 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:18:25.103448 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:25.103448 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:18:25.105053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:25.105053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:18:25.105053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:25.105053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:18:25.105053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:18:25.105053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:18:25.105053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:18:25.105053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:25.105053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:25.105053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:25.105053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:18:25.105053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:25.105459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:25.105459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:25.105459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:18:25.105459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:25.105459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:18:25.105459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:18:25.105459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:25.105459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:18:25.105459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:25.105459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:18:25.105459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:18:25.105459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:25.105459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:18:25.105805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:25.105805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:18:25.106895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:25.106895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:18:25.106895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:18:25.106895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:18:25.106895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:18:25.106895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:18:25.106895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:25.106895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:18:25.106895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:18:25.106895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:18:25.106895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:18:25.106895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:18:25.106895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:25.106895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:25.106895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:18:25.106895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:18:25.107438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:18:25.107438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:18:25.107438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:25.107438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:25.107438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:25.107438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:18:25.107438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:25.107438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:25.107438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:25.107438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:18:25.107438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:25.107438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:18:25.107438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:18:25.107438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:25.107438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:18:25.107438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:25.107438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:18:25.107438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:18:25.107438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:25.107956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:18:25.107956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:25.107956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:18:25.108576 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:18:25.108934 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:18:25.109108 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:18:25.109247 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:18:25.109607 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:18:25.109798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:25.110324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:18:25.110324 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:25.110324 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:18:25.110324 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:18:25.110324 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:25.110324 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:18:25.110324 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:25.110324 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:18:25.111055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:25.111055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:18:25.111055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:18:25.111055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:18:25.111055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:25.111055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:18:25.111055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:25.111055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:18:25.111055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:25.111055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:18:25.111055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:25.111055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:18:25.111055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:25.111055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:18:25.111055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:18:25.111520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:25.111520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:18:25.111520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:25.111520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:18:25.111520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:18:25.111520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:25.111520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:18:25.111520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:25.111520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:18:25.111861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:25.111861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:18:25.111861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:25.111861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:18:25.111861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:18:25.111861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:18:25.111861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:18:25.111861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:18:25.111861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:18:25.111861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:18:25.111861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:25.111861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:25.112199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:25.112199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:18:25.112199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:25.112199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:25.112199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:25.112199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:18:25.112199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:25.112199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:18:25.112199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:18:25.112199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:25.112199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:18:25.112199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:25.112199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:18:25.112199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:18:25.112587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:25.112587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:18:25.112587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:25.112587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:18:25.112838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:25.112838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:18:25.112838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:18:25.112838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:18:25.112838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:18:25.112838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:18:25.112838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:25.112838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:18:25.112838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:18:25.112838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:18:25.112838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:18:25.112838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:18:25.112838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:25.112838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:25.112838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:18:25.112838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:18:25.113278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:18:25.113278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:18:25.113278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:18:25.113278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:18:25.113278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:18:25.113278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:25.113278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:25.113278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:25.113278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:18:25.113278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:25.113278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:25.113278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:25.113278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:18:25.113278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:25.113278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:18:25.113278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:18:25.113278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:25.113278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:18:25.113994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:25.113994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:18:25.113994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:18:25.113994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:25.113994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:18:25.113994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:25.113994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:18:25.150631 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:18:25.151092 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:18:25.151292 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:18:25.151515 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:18:25.153022 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:18:25.153242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:25.153957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:18:25.153957 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:25.153957 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:18:25.153957 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:18:25.153957 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:25.153957 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:18:25.153957 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:25.153957 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:18:25.155488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:25.155488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:18:25.155488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:18:25.155488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:18:25.155488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:25.155488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:18:25.155488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:25.155488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:18:25.155488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:25.155488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:18:25.155488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:25.155488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:18:25.155488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:25.155488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:18:25.155488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:18:25.156161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:25.156161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:18:25.156161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:25.156161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:18:25.156161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:18:25.156161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:25.156161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:18:25.156161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:25.156161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:18:25.156943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:25.156943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:18:25.156943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:25.156943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:18:25.156943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:18:25.156943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:18:25.156943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:18:25.156943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:18:25.156943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:18:25.156943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:25.156943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:25.157510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:25.157510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:18:25.157510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:25.157510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:25.157510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:25.157510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:18:25.157510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:25.157510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:18:25.157510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:18:25.157510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:25.157510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:18:25.157510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:25.157510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:18:25.157510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:18:25.158130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:25.158130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:18:25.158130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:25.158130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:18:25.158915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:25.158915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:18:25.158915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:18:25.158915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:18:25.158915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:18:25.158915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:18:25.158915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:25.158915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:18:25.158915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:18:25.158915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:18:25.158915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:18:25.158915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:18:25.158915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:25.158915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:25.158915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:18:25.158915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:18:25.158915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:18:25.159760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:18:25.159760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:18:25.159760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:18:25.159760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:25.159760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:25.159760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:25.159760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:18:25.159760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:25.159760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:25.159760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:25.159760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:18:25.159760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:25.159760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:18:25.159760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:18:25.159760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:25.159760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:18:25.159760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:25.159760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:18:25.159760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:18:25.160268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:25.160268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:18:25.160268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:25.160268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:18:25.160268 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:18:25.160623 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:18:25.160803 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:18:25.161076 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:18:25.161469 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:18:25.161726 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:18:25.161880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:25.162452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:18:25.162452 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:25.162452 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:18:25.162452 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:18:25.162452 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:25.162452 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:18:25.162452 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:25.162452 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:18:25.163248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:25.163248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:18:25.163248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:25.163248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:18:25.163248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:18:25.163248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:18:25.163248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:18:25.163248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:25.163248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:25.163248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:25.163248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:18:25.163248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:25.163598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:25.163598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:25.163598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:18:25.163598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:25.163598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:18:25.163598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:18:25.163598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:25.163598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:18:25.163598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:25.163598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:18:25.163598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:18:25.163598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:25.163598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:18:25.163942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:25.163942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:18:25.164207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:25.164207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:18:25.164207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:18:25.164207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:18:25.164207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:18:25.164207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:18:25.164207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:25.164207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:18:25.164207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:18:25.164207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:18:25.164207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:18:25.164207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:18:25.164207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:25.164207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:25.164207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:18:25.164207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:18:25.164666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:18:25.164666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:18:25.164666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:25.164666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:25.164666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:25.164666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:18:25.164666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:25.164666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:25.164666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:25.164666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:18:25.164666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:25.164666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:18:25.164666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:18:25.164666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:25.164666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:18:25.164666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:25.164666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:18:25.164666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:18:25.164666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:25.165244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:18:25.165244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:25.165244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:18:29.851268 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:18:29.852211 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:18:29.852211 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:18:29.853357 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:18:29.884241 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:18:29.884687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:29.885571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:18:29.885571 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:29.885571 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:18:29.885571 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:18:29.885571 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:29.885571 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:18:29.885571 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:29.885571 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:18:29.886297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:29.886297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:18:29.886297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:29.886297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:18:29.886297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:18:29.886297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:18:29.886297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:18:29.886297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:29.886297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:29.886297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:29.886297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:18:29.886297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:29.886966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:29.886966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:29.886966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:18:29.886966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:29.886966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:18:29.886966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:18:29.886966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:29.886966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:18:29.886966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:29.886966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:18:29.886966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:18:29.886966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:29.886966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:18:29.887718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:29.887718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:18:29.887718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:29.887718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:18:29.887718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:18:29.887718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:18:29.887718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:18:29.887718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:18:29.887718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:29.887718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:18:29.887718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:18:29.887718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:18:29.887718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:18:29.887718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:18:29.887718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:29.888592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:29.888592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:18:29.888592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:18:29.888592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:18:29.888592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:18:29.888592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:29.888592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:29.888592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:29.888592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:18:29.888592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:29.888592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:29.888592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:29.888592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:18:29.888592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:29.888592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:18:29.888592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:18:29.888592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:29.888592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:18:29.888592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:29.889596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:18:29.889596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:18:29.889596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:29.889596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:18:29.889596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:29.889596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:18:30.100703 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:18:30.101235 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:18:30.101480 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:18:30.102553 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:18:30.102811 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:18:30.102984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:30.103738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:18:30.103738 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:30.103738 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:18:30.103738 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:18:30.103738 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:30.103738 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:18:30.103738 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:30.103738 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:18:30.105210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:30.105210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:18:30.105210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:30.105210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:18:30.105210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:18:30.105210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:18:30.105210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:18:30.105210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:30.105210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:30.105210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:30.105210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:18:30.105210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:30.105959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:30.105959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:30.105959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:18:30.105959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:30.105959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:18:30.105959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:18:30.105959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:30.105959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:18:30.105959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:30.105959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:18:30.105959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:18:30.105959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:30.105959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:18:30.106507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:30.106507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:18:30.106839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:30.106839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:18:30.106839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:18:30.106839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:18:30.106839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:18:30.106839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:18:30.106839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:30.106839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:18:30.106839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:18:30.106839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:18:30.106839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:18:30.106839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:18:30.106839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:30.106839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:30.106839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:18:30.106839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:18:30.107561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:18:30.107561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:18:30.107561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:30.107561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:30.107561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:30.107561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:18:30.107561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:30.107561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:30.107561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:30.107561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:18:30.107561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:30.107561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:18:30.107561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:18:30.107561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:30.107561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:18:30.107561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:30.107561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:18:30.107561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:18:30.107561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:30.108090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:18:30.108090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:30.108090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:18:30.109056 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:18:30.109427 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:18:30.109589 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:18:30.109691 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:18:30.110220 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:18:30.110504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:30.111113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:18:30.111113 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:30.111113 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:18:30.111113 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:18:30.111113 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:30.111113 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:18:30.111113 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:30.111113 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:18:30.111890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:30.111890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:18:30.111890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:18:30.111890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:18:30.111890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:30.111890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:18:30.111890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:30.111890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:18:30.111890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:30.111890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:18:30.111890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:30.111890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:18:30.111890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:30.111890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:18:30.111890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:18:30.112464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:30.112464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:18:30.112464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:30.112464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:18:30.112464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:18:30.112464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:30.112464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:18:30.112464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:30.112464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:18:30.112816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:30.112816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:18:30.112816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:30.112816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:18:30.112816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:18:30.112816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:18:30.112816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:18:30.112816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:18:30.112816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:18:30.112816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:18:30.112816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:30.112816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:30.113244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:30.113244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:18:30.113244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:30.113244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:30.113244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:30.113244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:18:30.113244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:30.113244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:18:30.113244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:18:30.113244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:30.113244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:18:30.113244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:30.113244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:18:30.113244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:18:30.113716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:30.113716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:18:30.113716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:30.113716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:18:30.113716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:30.113716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:18:30.113716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:18:30.113716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:18:30.113716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:18:30.113716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:18:30.113716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:30.113716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:18:30.113716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:18:30.113716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:18:30.114125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:18:30.114125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:18:30.114125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:30.114125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:30.114125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:18:30.114125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:18:30.114125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:18:30.114125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:18:30.114125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:18:30.114125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:18:30.114125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:18:30.114125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:30.114125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:30.114125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:30.114125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:18:30.114125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:30.114125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:30.114125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:30.114125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:18:30.114950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:30.114950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:18:30.114950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:18:30.114950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:30.114950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:18:30.114950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:30.114950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:18:30.114950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:18:30.114950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:30.114950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:18:30.114950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:30.114950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:18:30.151132 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:18:30.151532 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:18:30.151709 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:18:30.151893 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:18:30.154050 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:18:30.154469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:30.155279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:18:30.155279 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:30.155279 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:18:30.155279 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:18:30.155279 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:30.155279 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:18:30.155279 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:30.155279 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:18:30.156474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:30.156474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:18:30.156474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:18:30.156474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:18:30.156474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:30.156474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:18:30.156474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:30.156474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:18:30.156474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:30.156474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:18:30.156474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:30.156474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:18:30.156474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:30.156474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:18:30.156474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:18:30.157193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:30.157193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:18:30.157193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:30.157193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:18:30.157193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:18:30.157193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:30.157193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:18:30.157193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:30.157193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:18:30.157828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:30.157828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:18:30.157828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:30.157828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:18:30.157828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:18:30.157828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:18:30.157828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:18:30.157828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:18:30.157828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:18:30.157828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:30.157828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:30.158333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:30.158333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:18:30.158333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:30.158333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:30.158333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:30.158333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:18:30.158333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:30.158333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:18:30.158333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:18:30.158333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:30.158333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:18:30.158333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:30.158333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:18:30.158333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:18:30.158722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:30.158722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:18:30.158722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:30.158722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:18:30.159496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:30.159496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:18:30.159496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:18:30.159496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:18:30.159496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:18:30.159496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:18:30.159496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:30.159496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:18:30.159496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:18:30.159496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:18:30.159496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:18:30.159496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:18:30.159496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:30.159496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:30.159496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:18:30.159496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:18:30.159496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:18:30.159929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:18:30.159929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:18:30.159929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:18:30.159929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:30.159929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:30.159929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:30.159929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:18:30.159929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:30.159929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:30.159929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:30.159929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:18:30.159929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:30.159929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:18:30.159929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:18:30.159929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:30.159929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:18:30.159929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:30.159929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:18:30.159929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:18:30.160265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:30.160265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:18:30.160265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:30.160265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:18:30.160782 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:18:30.161118 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:18:30.161285 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:18:30.161482 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:18:30.162309 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:18:30.162309 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:18:30.162309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:30.162462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:18:30.162462 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:30.162462 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:18:30.162462 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:18:30.162462 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:30.162462 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:18:30.162462 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:30.162462 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:18:30.163229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:30.163229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:18:30.163229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:30.163229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:18:30.163229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:18:30.163229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:18:30.163229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:18:30.163229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:30.163229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:30.163229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:30.163229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:18:30.163229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:30.163479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:30.163479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:30.163479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:18:30.163479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:30.163479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:18:30.163479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:18:30.163479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:30.163479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:18:30.163479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:30.163479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:18:30.163479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:18:30.163479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:30.163479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:18:30.163705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:30.163705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:18:30.164157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:30.164157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:18:30.164157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:18:30.164157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:18:30.164157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:18:30.164157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:18:30.164157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:30.164157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:18:30.164157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:18:30.164157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:18:30.164157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:18:30.164157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:18:30.164157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:30.164157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:30.164157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:18:30.164157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:18:30.164478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:18:30.164478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:18:30.164478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:30.164478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:30.164478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:30.164478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:18:30.164478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:30.164478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:30.164478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:30.164478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:18:30.164478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:30.164478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:18:30.164478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:18:30.164478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:30.164478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:18:30.164478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:30.164478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:18:30.164478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:18:30.164478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:30.164851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:18:30.164851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:30.164851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:18:34.852035 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:18:34.852941 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:18:34.852941 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:18:34.853935 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:18:34.890905 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:18:34.890905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:34.890905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:18:34.890905 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:34.890905 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:18:34.890905 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:18:34.890905 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:34.890905 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:18:34.890905 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:34.890905 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:18:34.891798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:34.891798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:18:34.891798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:34.891798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:18:34.891798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:18:34.891798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:18:34.891798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:18:34.891798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:34.891798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:34.891798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:34.891798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:18:34.891798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:34.892635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:34.892635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:34.892635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:18:34.892635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:34.892635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:18:34.892635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:18:34.892635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:34.892635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:18:34.892635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:34.892635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:18:34.892635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:18:34.892635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:34.892635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:18:34.893263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:34.893263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:18:34.893263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:34.893263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:18:34.893263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:18:34.893263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:18:34.893263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:18:34.893263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:18:34.893263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:34.893263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:18:34.893263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:18:34.893263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:18:34.893263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:18:34.893263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:18:34.893263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:34.894531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:34.894531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:18:34.894531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:18:34.894531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:18:34.894531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:18:34.894531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:34.894531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:34.894531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:34.894531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:18:34.894531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:34.894531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:34.894531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:34.894531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:18:34.894531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:34.894531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:18:34.894531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:18:34.894531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:34.894531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:18:34.894531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:34.895520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:18:34.895520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:18:34.895520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:34.895520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:18:34.895520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:34.895520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:18:35.101367 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:18:35.101940 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:18:35.102182 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:18:35.103111 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:18:35.103580 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:18:35.103820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:35.104554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:18:35.104554 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:35.104554 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:18:35.104554 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:18:35.104554 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:35.104554 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:18:35.104554 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:35.104554 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:18:35.106490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:35.106490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:18:35.106490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:35.106490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:18:35.106490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:18:35.106490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:18:35.106490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:18:35.106490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:35.106490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:35.106490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:35.106490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:18:35.106490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:35.107077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:35.107077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:35.107077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:18:35.107077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:35.107077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:18:35.107077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:18:35.107077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:35.107077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:18:35.107077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:35.107077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:18:35.107077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:18:35.107077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:35.107077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:18:35.107744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:35.107744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:18:35.108306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:35.108306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:18:35.108306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:18:35.108306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:18:35.108306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:18:35.108306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:18:35.108306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:35.108306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:18:35.108306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:18:35.108306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:18:35.108306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:18:35.108306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:18:35.108306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:35.108306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:35.108306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:18:35.108306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:18:35.108924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:18:35.108924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:18:35.108924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:35.108924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:35.108924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:35.108924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:18:35.108924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:35.108924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:35.108924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:35.108924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:18:35.108924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:35.108924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:18:35.108924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:18:35.108924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:35.108924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:18:35.108924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:35.108924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:18:35.108924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:18:35.108924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:35.109472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:18:35.109472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:35.109472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:18:35.109564 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:18:35.109826 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:18:35.109942 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:18:35.110065 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:18:35.110496 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:18:35.110711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:35.111237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:18:35.111237 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:35.111237 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:18:35.111237 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:18:35.111237 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:35.111237 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:18:35.111237 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:35.111237 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:18:35.112175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:35.112175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:18:35.112175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:18:35.112175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:18:35.112175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:35.112175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:18:35.112175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:35.112175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:18:35.112175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:35.112175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:18:35.112175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:35.112175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:18:35.112175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:35.112175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:18:35.112175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:18:35.112684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:35.112684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:18:35.112684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:35.112684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:18:35.112684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:18:35.112684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:35.112684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:18:35.112684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:35.112684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:18:35.113074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:35.113074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:18:35.113074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:35.113074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:18:35.113074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:18:35.113074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:18:35.113074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:18:35.113074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:18:35.113074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:18:35.113074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:18:35.113074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:35.113074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:35.113492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:35.113492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:18:35.113492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:35.113492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:35.113492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:35.113492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:18:35.113492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:35.113492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:18:35.113492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:18:35.113492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:35.113492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:18:35.113492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:35.113492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:18:35.113492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:18:35.113894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:35.113894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:18:35.113894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:35.113894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:18:35.114147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:35.114147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:18:35.114147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:18:35.114147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:18:35.114147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:18:35.114147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:18:35.114147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:35.114147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:18:35.114147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:18:35.114147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:18:35.114147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:18:35.114147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:18:35.114147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:35.114147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:35.114147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:18:35.114147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:18:35.114638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:18:35.114638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:18:35.114638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:18:35.114638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:18:35.114638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:18:35.114638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:35.114638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:35.114638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:35.114638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:18:35.114638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:35.114638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:35.114638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:35.114638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:18:35.114638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:35.114638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:18:35.114638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:18:35.114638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:35.114638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:18:35.115187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:35.115187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:18:35.115187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:18:35.115187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:35.115187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:18:35.115187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:35.115187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:18:35.151724 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:18:35.152138 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:18:35.152302 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:18:35.152519 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:18:35.153136 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:18:35.153658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:35.154795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:18:35.154795 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:35.154795 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:18:35.154795 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:18:35.154795 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:35.154795 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:18:35.154795 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:35.154795 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:18:35.156035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:35.156035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:18:35.156035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:18:35.156035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:18:35.156035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:35.156035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:18:35.156035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:35.156035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:18:35.156035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:35.156035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:18:35.156035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:35.156035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:18:35.156035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:35.156035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:18:35.156035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:18:35.156704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:35.156704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:18:35.156704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:35.156704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:18:35.156704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:18:35.156704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:35.156704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:18:35.156704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:35.156704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:18:35.157498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:35.157498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:18:35.157498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:35.157498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:18:35.157498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:18:35.157498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:18:35.157498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:18:35.157498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:18:35.157498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:18:35.157498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:35.157498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:35.158485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:35.158485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:18:35.158485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:35.158485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:35.158485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:35.158485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:18:35.158485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:35.158485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:18:35.158485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:18:35.158485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:35.158485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:18:35.158485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:35.158485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:18:35.158485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:18:35.159096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:35.159096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:18:35.159096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:35.159096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:18:35.159866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:35.159866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:18:35.159866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:18:35.159866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:18:35.159866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:18:35.159866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:18:35.159866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:35.159866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:18:35.159866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:18:35.159866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:18:35.159866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:18:35.159866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:18:35.159866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:35.159866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:35.159866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:18:35.159866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:18:35.159866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:18:35.160425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:18:35.160425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:18:35.160425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:18:35.160425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:35.160425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:35.160425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:35.160425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:18:35.160425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:35.160425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:35.160425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:35.160425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:18:35.160425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:35.160425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:18:35.160425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:18:35.160425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:35.160425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:18:35.160425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:35.160425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:18:35.160425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:18:35.160923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:35.160923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:18:35.160923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:35.160923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:18:35.161270 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:18:35.161676 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:18:35.161850 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:18:35.162080 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:18:35.162425 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:18:35.162670 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:18:35.162820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:35.163283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:18:35.163283 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:35.163283 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:18:35.163283 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:18:35.163283 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:35.163283 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:18:35.163283 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:35.163283 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:18:35.163944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:35.163944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:18:35.163944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:35.163944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:18:35.163944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:18:35.163944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:18:35.163944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:18:35.163944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:35.163944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:35.163944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:35.163944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:18:35.163944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:35.164319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:35.164319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:35.164319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:18:35.164319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:35.164319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:18:35.164319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:18:35.164319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:35.164319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:18:35.164319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:35.164319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:18:35.164319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:18:35.164319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:35.164319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:18:35.164697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:35.164697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:18:35.166031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:35.166031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:18:35.166031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:18:35.166031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:18:35.166031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:18:35.166031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:18:35.166031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:35.166031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:18:35.166031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:18:35.166031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:18:35.166031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:18:35.166031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:18:35.166031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:35.166031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:35.166031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:18:35.166031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:18:35.166528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:18:35.166528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:18:35.166528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:35.166528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:35.166528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:35.166528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:18:35.166528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:35.166528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:35.166528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:35.166528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:18:35.166528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:35.166528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:18:35.166528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:18:35.166528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:35.166528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:18:35.166528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:35.166528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:18:35.166528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:18:35.166528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:35.167067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:18:35.167067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:35.167067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:18:39.852861 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:18:39.853598 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:18:39.853721 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:18:39.854686 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:18:39.886569 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:18:39.886768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:39.887339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:18:39.887339 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:39.887339 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:18:39.887339 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:18:39.887339 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:39.887339 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:18:39.887339 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:39.887339 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:18:39.888114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:39.888114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:18:39.888114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:39.888114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:18:39.888114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:18:39.888114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:18:39.888114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:18:39.888114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:39.888114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:39.888114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:39.888114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:18:39.888114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:39.888553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:39.888553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:39.888553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:18:39.888553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:39.888553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:18:39.888553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:18:39.888553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:39.888553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:18:39.888553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:39.888553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:18:39.888553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:18:39.888553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:39.888553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:18:39.888950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:39.888950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:18:39.888950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:39.888950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:18:39.888950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:18:39.888950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:18:39.888950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:18:39.888950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:18:39.888950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:39.888950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:18:39.888950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:18:39.888950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:18:39.888950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:18:39.888950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:18:39.888950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:39.889444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:39.889444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:18:39.889444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:18:39.889444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:18:39.889444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:18:39.889444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:39.889444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:39.889444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:39.889444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:18:39.889444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:39.889444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:39.889444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:39.889444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:18:39.889444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:39.889444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:18:39.889444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:18:39.889444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:39.889444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:18:39.889444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:39.890335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:18:39.890335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:18:39.890335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:39.890335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:18:39.890335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:39.890335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:18:40.102148 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:18:40.102816 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:18:40.103014 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:18:40.104204 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:18:40.104534 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:18:40.104822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:40.105704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:18:40.105704 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:40.105704 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:18:40.105704 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:18:40.105704 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:40.105704 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:18:40.105704 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:40.105704 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:18:40.107351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:40.107351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:18:40.107351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:40.107351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:18:40.107351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:18:40.107351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:18:40.107351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:18:40.107351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:40.107351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:40.107351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:40.107351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:18:40.107351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:40.107654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:40.107654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:40.107654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:18:40.107654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:40.107654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:18:40.107654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:18:40.107654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:40.107654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:18:40.107654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:40.107654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:18:40.107654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:18:40.107654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:40.107654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:18:40.107913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:40.107913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:18:40.109133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:40.109133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:18:40.109133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:18:40.109133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:18:40.109133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:18:40.109133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:18:40.109133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:40.109133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:18:40.109133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:18:40.109133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:18:40.109133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:18:40.109133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:18:40.109133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:40.109133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:40.109133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:18:40.109133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:18:40.109466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:18:40.109466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:18:40.109466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:40.109466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:40.109466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:40.109466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:18:40.109466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:40.109466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:40.109466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:40.109466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:18:40.109466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:40.109466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:18:40.109466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:18:40.109466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:40.109466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:18:40.109466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:40.109466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:18:40.109466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:18:40.109466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:40.109818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:18:40.109818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:40.109818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:18:40.109984 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:18:40.110392 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:18:40.110623 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:18:40.110816 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:18:40.111270 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:18:40.111614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:40.112331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:18:40.112331 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:40.112331 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:18:40.112331 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:18:40.112331 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:40.112331 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:18:40.112331 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:40.112331 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:18:40.113587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:40.113587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:18:40.113587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:18:40.113587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:18:40.113587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:40.113587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:18:40.113587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:40.113587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:18:40.113587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:40.113587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:18:40.113587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:40.113587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:18:40.113587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:40.113587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:18:40.113587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:18:40.114077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:40.114077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:18:40.114077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:40.114077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:18:40.114077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:18:40.114077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:40.114077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:18:40.114077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:40.114077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:18:40.114952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:40.114952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:18:40.114952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:40.114952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:18:40.114952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:18:40.114952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:18:40.114952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:18:40.114952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:18:40.114952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:18:40.114952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:18:40.114952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:40.114952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:40.115372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:40.115372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:18:40.115372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:40.115372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:40.115372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:40.115372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:18:40.115372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:40.115372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:18:40.115372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:18:40.115372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:40.115372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:18:40.115372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:40.115372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:18:40.115372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:18:40.115752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:40.115752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:18:40.115752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:40.115752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:18:40.116046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:40.116046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:18:40.116046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:18:40.116046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:18:40.116046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:18:40.116046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:18:40.116046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:40.116046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:18:40.116046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:18:40.116046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:18:40.116046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:18:40.116046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:18:40.116046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:40.116046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:40.116046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:18:40.116046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:18:40.117303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:18:40.117303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:18:40.117303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:18:40.117303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:18:40.117303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:18:40.117303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:40.117303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:40.117303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:40.117303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:18:40.117303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:40.117303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:40.117303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:40.117303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:18:40.117303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:40.117303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:18:40.117303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:18:40.117303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:40.117303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:18:40.117839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:40.117839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:18:40.117839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:18:40.117839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:40.117839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:18:40.117839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:40.117839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:18:40.152571 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:18:40.152989 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:18:40.152989 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:18:40.153157 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:18:40.153697 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:18:40.154070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:40.154795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:18:40.154795 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:40.154795 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:18:40.154795 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:18:40.154795 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:40.154795 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:18:40.154795 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:40.154795 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:18:40.155665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:40.155665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:18:40.155665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:18:40.155665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:18:40.155665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:40.155665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:18:40.155665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:40.155665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:18:40.155665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:40.155665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:18:40.155665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:40.155665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:18:40.155665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:40.155665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:18:40.155665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:18:40.156376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:40.156376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:18:40.156376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:40.156376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:18:40.156376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:18:40.156376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:40.156376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:18:40.156376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:40.156376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:18:40.156800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:40.156800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:18:40.156800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:40.156800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:18:40.156800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:18:40.156800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:18:40.156800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:18:40.156800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:18:40.156800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:18:40.156800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:40.156800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:40.157472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:40.157472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:18:40.157472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:40.157472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:40.157472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:40.157472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:18:40.157472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:40.157472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:18:40.157472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:18:40.157472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:40.157472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:18:40.157472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:40.157472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:18:40.157472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:18:40.158103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:40.158103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:18:40.158103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:40.158103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:18:40.158103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:40.158103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:18:40.158103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:18:40.158103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:18:40.158103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:18:40.158103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:18:40.158103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:40.158103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:18:40.158103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:18:40.158103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:18:40.158944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:18:40.158944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:18:40.158944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:40.158944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:40.158944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:18:40.158944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:18:40.158944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:18:40.158944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:18:40.158944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:18:40.158944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:18:40.158944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:40.158944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:40.158944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:40.158944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:18:40.158944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:40.158944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:40.158944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:40.158944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:18:40.158944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:40.159780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:18:40.159780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:18:40.159780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:40.159780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:18:40.159780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:40.159780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:18:40.159780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:18:40.159780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:40.159780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:18:40.159780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:40.159780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:18:40.161738 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:18:40.162110 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:18:40.162222 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:18:40.162374 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:18:40.162687 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:18:40.162907 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:18:40.163046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:40.163537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:18:40.163537 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:40.163537 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:18:40.163537 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:18:40.163537 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:40.163537 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:18:40.163537 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:40.163537 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:18:40.164318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:40.164318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:18:40.164318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:40.164318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:18:40.164318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:18:40.164318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:18:40.164318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:18:40.164318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:40.164318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:40.164318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:40.164318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:18:40.164318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:40.164603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:40.164603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:40.164603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:18:40.164603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:40.164603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:18:40.164603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:18:40.164603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:40.164603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:18:40.164603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:40.164603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:18:40.164603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:18:40.164603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:40.164603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:18:40.164846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:40.164846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:18:40.165505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:40.165505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:18:40.165505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:18:40.165505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:18:40.165505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:18:40.165505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:18:40.165505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:40.165505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:18:40.165505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:18:40.165505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:18:40.165505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:18:40.165505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:18:40.165505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:40.165505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:40.165505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:18:40.165505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:18:40.165844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:18:40.165844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:18:40.165844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:40.165844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:40.165844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:40.165844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:18:40.165844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:40.165844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:40.165844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:40.165844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:18:40.165844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:40.165844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:18:40.165844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:18:40.165844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:40.165844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:18:40.165844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:40.165844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:18:40.165844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:18:40.165844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:40.166213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:18:40.166213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:40.166213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:18:44.853774 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:18:44.854715 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:18:44.854715 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:18:44.855231 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:18:44.889724 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:18:44.890190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:44.891076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:18:44.891076 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:44.891076 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:18:44.891076 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:18:44.891076 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:44.891076 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:18:44.891076 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:44.891076 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:18:44.891852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:44.891852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:18:44.891852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:44.891852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:18:44.891852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:18:44.891852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:18:44.891852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:18:44.891852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:44.891852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:44.891852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:44.891852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:18:44.891852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:44.892678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:44.892678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:44.892678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:18:44.892678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:44.892678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:18:44.892678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:18:44.892678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:44.892678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:18:44.892678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:44.892678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:18:44.892678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:18:44.892678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:44.892678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:18:44.893306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:44.893306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:18:44.893306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:44.893306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:18:44.893306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:18:44.893306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:18:44.893306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:18:44.893306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:18:44.893306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:44.893306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:18:44.893306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:18:44.893306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:18:44.893306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:18:44.893306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:18:44.893306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:44.893808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:44.893808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:18:44.893808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:18:44.893808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:18:44.893808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:18:44.893808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:44.893808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:44.893808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:44.893808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:18:44.893808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:44.893808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:44.893808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:44.893808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:18:44.893808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:44.893808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:18:44.893808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:18:44.893808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:44.893808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:18:44.893808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:44.894881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:18:44.894881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:18:44.894881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:44.894881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:18:44.894881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:44.894881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:18:45.102775 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:18:45.103353 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:18:45.103620 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:18:45.105010 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:18:45.105190 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:18:45.105353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:45.106215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:18:45.106215 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:45.106215 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:18:45.106215 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:18:45.106215 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:45.106215 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:18:45.106215 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:45.106215 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:18:45.108224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:45.108224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:18:45.108224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:45.108224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:18:45.108224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:18:45.108224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:18:45.108224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:18:45.108224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:45.108224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:45.108224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:45.108224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:18:45.108224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:45.108692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:45.108692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:45.108692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:18:45.108692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:45.108692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:18:45.108692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:18:45.108692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:45.108692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:18:45.108692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:45.108692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:18:45.108692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:18:45.108692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:45.108692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:18:45.109043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:45.109043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:18:45.109960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:45.109960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:18:45.109960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:18:45.109960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:18:45.109960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:18:45.109960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:18:45.109960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:45.109960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:18:45.109960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:18:45.109960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:18:45.109960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:18:45.109960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:18:45.109960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:45.109960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:45.109960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:18:45.109960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:18:45.110466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:18:45.110466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:18:45.110466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:45.110466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:45.110466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:45.110466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:18:45.110466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:45.110466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:45.110466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:45.110466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:18:45.110466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:45.110466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:18:45.110466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:18:45.110466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:45.110466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:18:45.110466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:45.110466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:18:45.110466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:18:45.110466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:45.111004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:18:45.111004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:45.111004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:18:45.111004 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:18:45.111004 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:18:45.111004 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:18:45.111004 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:18:45.111399 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:18:45.111646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:45.112839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:18:45.112839 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:45.112839 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:18:45.112839 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:18:45.112839 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:45.112839 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:18:45.112839 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:45.112839 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:18:45.114040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:45.114040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:18:45.114040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:18:45.114040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:18:45.114040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:45.114040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:18:45.114040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:45.114040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:18:45.114040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:45.114040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:18:45.114040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:45.114040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:18:45.114040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:45.114040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:18:45.114040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:18:45.114693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:45.114693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:18:45.114693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:45.114693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:18:45.114693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:18:45.114693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:45.114693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:18:45.114693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:45.114693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:18:45.115329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:45.115329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:18:45.115329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:45.115329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:18:45.115329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:18:45.115329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:18:45.115329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:18:45.115329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:18:45.115329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:18:45.115329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:18:45.115329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:45.115329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:45.115890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:45.115890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:18:45.115890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:45.115890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:45.115890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:45.115890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:18:45.115890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:45.115890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:18:45.115890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:18:45.115890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:45.115890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:18:45.115890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:45.115890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:18:45.115890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:18:45.116298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:45.116298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:18:45.116298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:45.116298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:18:45.116592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:45.116592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:18:45.116592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:18:45.116592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:18:45.116592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:18:45.116592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:18:45.116592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:45.116592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:18:45.116592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:18:45.116592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:18:45.116592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:18:45.116592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:18:45.116592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:45.116592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:45.116592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:18:45.116592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:18:45.117068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:18:45.117068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:18:45.117068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:18:45.117068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:18:45.117068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:18:45.117068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:45.117068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:45.117068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:45.117068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:18:45.117068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:45.117068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:45.117068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:45.117068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:18:45.117068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:45.117068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:18:45.117068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:18:45.117068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:45.117068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:18:45.117616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:45.117616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:18:45.117616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:18:45.117616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:45.117616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:18:45.117616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:45.117616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:18:45.152788 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:18:45.153304 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:18:45.153485 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:18:45.153670 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:18:45.154263 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:18:45.154558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:45.155179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:18:45.155179 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:45.155179 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:18:45.155179 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:18:45.155179 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:45.155179 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:18:45.155179 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:45.155179 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:18:45.156474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:45.156474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:18:45.156474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:18:45.156474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:18:45.156474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:45.156474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:18:45.156474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:45.156474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:18:45.156474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:45.156474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:18:45.156474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:45.156474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:18:45.156474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:45.156474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:18:45.156474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:18:45.157450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:45.157450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:18:45.157450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:45.157450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:18:45.157450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:18:45.157450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:45.157450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:18:45.157450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:45.157450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:18:45.157975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:45.157975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:18:45.157975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:45.157975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:18:45.157975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:18:45.157975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:18:45.157975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:18:45.157975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:18:45.157975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:18:45.157975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:45.157975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:45.158709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:45.158709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:18:45.158709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:45.158709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:45.158709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:45.158709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:18:45.158709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:45.158709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:18:45.158709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:18:45.158709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:45.158709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:18:45.158709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:45.158709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:18:45.158709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:18:45.160010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:45.160010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:18:45.160010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:45.160010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:18:45.160010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:45.160010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:18:45.160010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:18:45.160010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:18:45.160010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:18:45.160010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:18:45.160010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:45.160010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:18:45.160010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:18:45.160010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:18:45.160776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:18:45.160776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:18:45.160776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:45.160776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:45.160776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:18:45.160776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:18:45.160776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:18:45.160776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:18:45.160776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:18:45.160776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:18:45.160776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:45.160776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:45.160776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:45.160776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:18:45.160776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:45.160776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:45.160776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:45.160776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:18:45.160776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:45.162141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:18:45.162141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:18:45.162141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:45.162141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:18:45.162141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:45.162141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:18:45.162141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:18:45.162141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:45.162141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:18:45.162141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:45.162141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:18:45.162756 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:18:45.162756 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:18:45.162963 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:18:45.163027 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:18:45.163349 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:18:45.163597 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:18:45.163706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:45.164195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:18:45.164195 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:45.164195 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:18:45.164195 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:18:45.164195 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:45.164195 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:18:45.164195 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:45.164195 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:18:45.165093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:45.165093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:18:45.165093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:45.165093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:18:45.165093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:18:45.165093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:18:45.165093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:18:45.165093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:45.165093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:45.165093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:45.165093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:18:45.165093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:45.165507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:45.165507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:45.165507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:18:45.165507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:45.165507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:18:45.165507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:18:45.165507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:45.165507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:18:45.165507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:45.165507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:18:45.165507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:18:45.165507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:45.165507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:18:45.165914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:45.165914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:18:45.166116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:45.166116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:18:45.166116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:18:45.166116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:18:45.166116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:18:45.166116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:18:45.166116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:45.166116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:18:45.166116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:18:45.166116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:18:45.166116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:18:45.166116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:18:45.166116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:45.166116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:45.166116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:18:45.166116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:18:45.166644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:18:45.166644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:18:45.166644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:45.166644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:45.166644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:45.166644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:18:45.166644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:45.166644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:45.166644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:45.166644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:18:45.166644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:45.166644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:18:45.166644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:18:45.166644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:45.166644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:18:45.166644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:45.166644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:18:45.166644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:18:45.166644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:45.167167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:18:45.167167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:45.167167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:18:49.854337 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:18:49.855014 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:18:49.855196 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:18:49.856466 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:18:49.892304 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:18:49.892719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:49.893549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:18:49.893549 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:49.893549 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:18:49.893549 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:18:49.893549 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:49.893549 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:18:49.893549 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:49.893549 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:18:49.894327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:49.894327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:18:49.894327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:49.894327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:18:49.894327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:18:49.894327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:18:49.894327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:18:49.894327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:49.894327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:49.894327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:49.894327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:18:49.894327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:49.895071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:49.895071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:49.895071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:18:49.895071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:49.895071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:18:49.895071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:18:49.895071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:49.895071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:18:49.895071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:49.895071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:18:49.895071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:18:49.895071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:49.895071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:18:49.896075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:49.896075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:18:49.896075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:49.896075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:18:49.896075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:18:49.896075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:18:49.896075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:18:49.896075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:18:49.896075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:49.896075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:18:49.896075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:18:49.896075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:18:49.896075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:18:49.896075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:18:49.896075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:49.896676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:49.896676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:18:49.896676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:18:49.896676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:18:49.896676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:18:49.896676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:49.896676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:49.896676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:49.896676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:18:49.896676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:49.896676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:49.896676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:49.896676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:18:49.896676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:49.896676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:18:49.896676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:18:49.896676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:49.896676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:18:49.896676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:49.897226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:18:49.897226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:18:49.897226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:49.897226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:18:49.897226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:49.897226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:18:50.103655 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:18:50.104102 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:18:50.104190 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:18:50.105152 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:18:50.105559 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:18:50.105835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:50.106654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:18:50.106654 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:50.106654 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:18:50.106654 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:18:50.106654 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:50.106654 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:18:50.106654 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:50.106654 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:18:50.107947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:50.107947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:18:50.107947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:50.107947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:18:50.107947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:18:50.107947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:18:50.107947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:18:50.107947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:50.107947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:50.107947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:50.107947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:18:50.107947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:50.108727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:50.108727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:50.108727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:18:50.108727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:50.108727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:18:50.108727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:18:50.108727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:50.108727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:18:50.108727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:50.108727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:18:50.108727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:18:50.108727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:50.108727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:18:50.109283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:50.109283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:18:50.109805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:50.109805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:18:50.109805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:18:50.109805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:18:50.109805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:18:50.109805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:18:50.109805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:50.109805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:18:50.109805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:18:50.109805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:18:50.109805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:18:50.109805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:18:50.109805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:50.109805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:50.109805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:18:50.109805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:18:50.110574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:18:50.110574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:18:50.110574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:50.110574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:50.110574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:50.110574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:18:50.110574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:50.110574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:50.110574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:50.110574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:18:50.110574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:50.110574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:18:50.110574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:18:50.110574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:50.110574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:18:50.110574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:50.110574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:18:50.110574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:18:50.110574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:50.111211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:18:50.111211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:50.111211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:18:50.111211 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:18:50.111211 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:18:50.111428 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:18:50.111677 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:18:50.112033 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:18:50.112252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:50.112859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:18:50.112859 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:50.112859 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:18:50.112859 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:18:50.112859 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:50.112859 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:18:50.112859 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:50.112859 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:18:50.113989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:50.113989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:18:50.113989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:18:50.113989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:18:50.113989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:50.113989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:18:50.113989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:50.113989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:18:50.113989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:50.113989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:18:50.113989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:50.113989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:18:50.113989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:50.113989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:18:50.113989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:18:50.114611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:50.114611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:18:50.114611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:50.114611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:18:50.114611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:18:50.114611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:50.114611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:18:50.114611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:50.114611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:18:50.115277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:50.115277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:18:50.115277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:50.115277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:18:50.115277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:18:50.115277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:18:50.115277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:18:50.115277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:18:50.115277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:18:50.115277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:18:50.115277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:50.115277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:50.115745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:50.115745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:18:50.115745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:50.115745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:50.115745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:50.115745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:18:50.115745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:50.115745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:18:50.115745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:18:50.115745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:50.115745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:18:50.115745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:50.115745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:18:50.115745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:18:50.116209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:50.116209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:18:50.116209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:50.116209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:18:50.116902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:50.116902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:18:50.116902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:18:50.116902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:18:50.116902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:18:50.116902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:18:50.116902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:50.116902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:18:50.116902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:18:50.116902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:18:50.116902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:18:50.116902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:18:50.116902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:50.116902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:50.116902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:18:50.116902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:18:50.117433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:18:50.117433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:18:50.117433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:18:50.117433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:18:50.117433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:18:50.117433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:50.117433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:50.117433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:50.117433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:18:50.117433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:50.117433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:50.117433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:50.117433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:18:50.117433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:50.117433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:18:50.117433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:18:50.117433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:50.117433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:18:50.117992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:50.117992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:18:50.117992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:18:50.117992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:50.117992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:18:50.117992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:50.117992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:18:50.153389 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:18:50.153640 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:18:50.153745 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:18:50.153878 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:18:50.154553 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:18:50.154973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:50.156021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:18:50.156021 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:50.156021 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:18:50.156021 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:18:50.156021 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:50.156021 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:18:50.156021 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:50.156021 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:18:50.157074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:50.157074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:18:50.157074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:18:50.157074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:18:50.157074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:50.157074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:18:50.157074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:50.157074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:18:50.157074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:50.157074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:18:50.157074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:50.157074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:18:50.157074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:50.157074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:18:50.157074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:18:50.157741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:50.157741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:18:50.157741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:50.157741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:18:50.157741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:18:50.157741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:50.157741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:18:50.157741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:50.157741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:18:50.158748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:50.158748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:18:50.158748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:50.158748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:18:50.158748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:18:50.158748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:18:50.158748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:18:50.158748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:18:50.158748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:18:50.158748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:50.158748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:50.159120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:50.159120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:18:50.159120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:50.159120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:50.159120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:50.159120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:18:50.159120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:50.159120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:18:50.159120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:18:50.159120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:50.159120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:18:50.159120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:50.159120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:18:50.159120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:18:50.159585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:50.159585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:18:50.159585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:50.159585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:18:50.160487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:50.160487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:18:50.160487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:18:50.160487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:18:50.160487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:18:50.160487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:18:50.160487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:50.160487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:18:50.160487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:18:50.160487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:18:50.160487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:18:50.160487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:18:50.160487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:50.160487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:50.160487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:18:50.160487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:18:50.160487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:18:50.161060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:18:50.161060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:18:50.161060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:18:50.161060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:50.161060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:50.161060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:50.161060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:18:50.161060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:50.161060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:50.161060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:50.161060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:18:50.161060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:50.161060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:18:50.161060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:18:50.161060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:50.161060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:18:50.161060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:50.161060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:18:50.161060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:18:50.161654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:50.161654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:18:50.161654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:50.161654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:18:50.162870 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:18:50.163215 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:18:50.163500 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:18:50.163879 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:18:50.164275 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:18:50.164612 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:18:50.164797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:50.165478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:18:50.165478 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:50.165478 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:18:50.165478 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:18:50.165478 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:50.165478 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:18:50.165478 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:50.165478 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:18:50.166252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:50.166252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:18:50.166252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:50.166252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:18:50.166252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:18:50.166252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:18:50.166252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:18:50.166252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:50.166252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:50.166252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:50.166252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:18:50.166252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:50.166812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:50.166812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:50.166812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:18:50.166812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:50.166812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:18:50.166812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:18:50.166812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:50.166812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:18:50.166812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:50.166812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:18:50.166812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:18:50.166812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:50.166812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:18:50.167257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:50.167257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:18:50.167257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:50.167257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:18:50.167257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:18:50.167257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:18:50.167257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:18:50.167257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:18:50.167257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:50.167257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:18:50.167257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:18:50.167257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:18:50.167257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:18:50.167257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:18:50.167257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:50.167888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:50.167888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:18:50.167888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:18:50.167888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:18:50.167888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:18:50.167888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:50.167888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:50.167888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:50.167888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:18:50.167888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:50.167888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:50.167888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:50.167888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:18:50.167888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:50.167888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:18:50.167888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:18:50.167888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:50.167888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:18:50.167888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:50.168472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:18:50.168472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:18:50.168472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:50.168472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:18:50.168472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:50.168472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:18:54.856034 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:18:54.856677 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:18:54.856874 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:18:54.858250 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:18:54.891936 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:18:54.892321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:54.893187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:18:54.893187 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:54.893187 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:18:54.893187 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:18:54.893187 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:54.893187 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:18:54.893187 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:54.893187 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:18:54.893953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:54.893953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:18:54.893953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:54.893953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:18:54.893953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:18:54.893953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:18:54.893953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:18:54.893953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:54.893953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:54.893953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:54.893953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:18:54.893953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:54.894591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:54.894591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:54.894591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:18:54.894591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:54.894591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:18:54.894591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:18:54.894591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:54.894591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:18:54.894591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:54.894591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:18:54.894591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:18:54.894591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:54.894591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:18:54.895219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:54.895219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:18:54.895219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:54.895219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:18:54.895219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:18:54.895219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:18:54.895219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:18:54.895219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:18:54.895219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:54.895219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:18:54.895219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:18:54.895219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:18:54.895219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:18:54.895219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:18:54.895219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:54.896053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:54.896053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:18:54.896053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:18:54.896053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:18:54.896053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:18:54.896053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:54.896053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:54.896053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:54.896053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:18:54.896053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:54.896053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:54.896053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:54.896053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:18:54.896053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:54.896053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:18:54.896053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:18:54.896053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:54.896053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:18:54.896053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:54.897006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:18:54.897006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:18:54.897006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:54.897006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:18:54.897006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:54.897006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:18:55.104619 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:18:55.105362 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:18:55.105514 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:18:55.106789 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:18:55.107091 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:18:55.107363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:55.108263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:18:55.108263 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:55.108263 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:18:55.108263 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:18:55.108263 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:55.108263 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:18:55.108263 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:55.108263 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:18:55.109574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:55.109574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:18:55.109574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:55.109574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:18:55.109574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:18:55.109574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:18:55.109574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:18:55.109574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:55.109574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:55.109574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:55.109574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:18:55.109574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:55.110204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:55.110204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:55.110204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:18:55.110204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:55.110204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:18:55.110204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:18:55.110204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:55.110204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:18:55.110204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:55.110204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:18:55.110204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:18:55.110204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:55.110204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:18:55.110793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:55.110793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:18:55.111088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:55.111088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:18:55.111088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:18:55.111088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:18:55.111088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:18:55.111088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:18:55.111088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:55.111088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:18:55.111088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:18:55.111088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:18:55.111088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:18:55.111088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:18:55.111088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:55.111088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:55.111088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:18:55.111088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:18:55.111894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:18:55.111894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:18:55.111894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:55.111894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:55.111894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:55.111894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:18:55.111894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:55.111894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:55.111894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:55.111894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:18:55.111894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:55.111894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:18:55.111894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:18:55.111894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:55.111894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:18:55.111894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:55.111894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:18:55.111894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:18:55.111894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:55.112788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:18:55.112788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:55.112788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:18:55.112788 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:18:55.112788 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:18:55.112788 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:18:55.112788 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:18:55.112788 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:18:55.113218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:55.113645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:18:55.113645 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:55.113645 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:18:55.113645 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:18:55.113645 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:55.113645 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:18:55.113645 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:55.113645 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:18:55.114647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:55.114647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:18:55.114647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:18:55.114647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:18:55.114647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:55.114647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:18:55.114647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:55.114647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:18:55.114647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:55.114647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:18:55.114647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:55.114647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:18:55.114647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:55.114647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:18:55.114647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:18:55.115417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:55.115417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:18:55.115417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:55.115417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:18:55.115417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:18:55.115417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:55.115417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:18:55.115417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:55.115417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:18:55.115843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:55.115843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:18:55.115843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:55.115843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:18:55.115843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:18:55.115843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:18:55.115843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:18:55.115843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:18:55.115843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:18:55.115843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:18:55.115843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:55.115843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:55.116174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:55.116174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:18:55.116174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:55.116174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:55.116174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:55.116174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:18:55.116174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:55.116174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:18:55.116174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:18:55.116174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:55.116174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:18:55.116174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:55.116174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:18:55.116174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:18:55.116525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:55.116525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:18:55.116525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:55.116525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:18:55.117093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:55.117093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:18:55.117093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:18:55.117093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:18:55.117093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:18:55.117093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:18:55.117093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:55.117093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:18:55.117093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:18:55.117093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:18:55.117093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:18:55.117093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:18:55.117093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:55.117093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:55.117093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:18:55.117093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:18:55.117550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:18:55.117550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:18:55.117550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:18:55.117550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:18:55.117550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:18:55.117550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:55.117550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:55.117550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:55.117550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:18:55.117550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:55.117550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:55.117550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:55.117550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:18:55.117550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:55.117550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:18:55.117550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:18:55.117550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:55.117550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:18:55.118070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:55.118070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:18:55.118070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:18:55.118070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:55.118070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:18:55.118070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:55.118070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:18:55.153934 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:18:55.154424 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:18:55.154637 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:18:55.154843 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:18:55.155345 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:18:55.155718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:55.156514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:18:55.156514 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:55.156514 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:18:55.156514 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:18:55.156514 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:55.156514 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:18:55.156514 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:55.156514 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:18:55.158145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:55.158145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:18:55.158145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:18:55.158145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:18:55.158145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:55.158145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:18:55.158145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:55.158145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:18:55.158145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:55.158145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:18:55.158145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:55.158145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:18:55.158145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:55.158145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:18:55.158145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:18:55.159109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:55.159109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:18:55.159109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:55.159109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:18:55.159109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:18:55.159109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:55.159109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:18:55.159109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:55.159109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:18:55.159683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:55.159683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:18:55.159683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:55.159683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:18:55.159683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:18:55.159683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:18:55.159683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:18:55.159683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:18:55.159683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:18:55.159683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:55.159683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:55.160359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:55.160359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:18:55.160359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:55.160359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:55.160359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:55.160359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:18:55.160359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:55.160359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:18:55.160359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:18:55.160359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:55.160359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:18:55.160359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:55.160359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:18:55.160359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:18:55.161223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:55.161223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:18:55.161223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:55.161223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:18:55.161517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:55.161517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:18:55.161517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:18:55.161517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:18:55.161517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:18:55.161517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:18:55.161517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:55.161517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:18:55.161517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:18:55.161517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:18:55.161517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:18:55.161517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:18:55.161517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:55.161517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:55.161517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:18:55.161517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:18:55.161517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:18:55.162109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:18:55.162109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:18:55.162109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:18:55.162109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:55.162109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:55.162109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:55.162109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:18:55.162109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:55.162109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:55.162109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:55.162109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:18:55.162109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:55.162109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:18:55.162109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:18:55.162109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:55.162109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:18:55.162109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:55.162109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:18:55.162109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:18:55.162816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:55.162816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:18:55.162816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:55.162816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:18:55.163417 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:18:55.163727 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:18:55.163866 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:18:55.164016 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:18:55.164446 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:18:55.164811 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:18:55.165018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:55.165749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:18:55.165749 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:55.165749 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:18:55.165749 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:18:55.165749 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:55.165749 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:18:55.165749 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:55.165749 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:18:55.166552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:55.166552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:18:55.166552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:55.166552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:18:55.166552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:18:55.166552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:18:55.166552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:18:55.166552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:55.166552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:55.166552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:55.166552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:18:55.166552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:55.166993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:55.166993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:55.166993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:18:55.166993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:55.166993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:18:55.166993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:18:55.166993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:55.166993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:18:55.166993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:55.166993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:18:55.166993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:18:55.166993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:55.166993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:18:55.167392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:55.167392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:18:55.167499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:55.167499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:18:55.167499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:18:55.167499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:18:55.167499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:18:55.167499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:18:55.167499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:55.167499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:18:55.167499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:18:55.167499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:18:55.167499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:18:55.167499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:18:55.167499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:55.167499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:55.167499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:18:55.167499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:18:55.168026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:18:55.168026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:18:55.168026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:55.168026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:55.168026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:55.168026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:18:55.168026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:55.168026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:55.168026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:55.168026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:18:55.168026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:55.168026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:18:55.168026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:18:55.168026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:55.168026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:18:55.168026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:55.168026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:18:55.168026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:18:55.168026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:55.168695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:18:55.168695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:55.168695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:18:59.857006 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:18:59.858098 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:18:59.858098 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:18:59.858970 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:18:59.890678 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:18:59.890678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:59.890969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:18:59.890969 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:59.890969 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:18:59.890969 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:18:59.890969 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:59.890969 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:18:59.890969 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:59.890969 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:18:59.891641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:59.891641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:18:59.891641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:59.891641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:18:59.891641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:18:59.891641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:18:59.891641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:18:59.891641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:59.891641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:59.891641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:59.891641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:18:59.891641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:59.892327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:18:59.892327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:59.892327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:18:59.892327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:59.892327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:18:59.892327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:18:59.892327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:59.892327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:18:59.892327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:59.892327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:18:59.892327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:18:59.892327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:59.892327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:18:59.893021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:59.893021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:18:59.893021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:59.893021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:18:59.893021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:18:59.893021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:18:59.893021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:18:59.893021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:18:59.893021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:59.893021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:18:59.893021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:18:59.893021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:18:59.893021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:18:59.893021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:18:59.893021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:18:59.893924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:18:59.893924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:18:59.893924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:18:59.893924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:18:59.893924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:18:59.893924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:59.893924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:59.893924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:18:59.893924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:18:59.893924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:18:59.893924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:18:59.893924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:18:59.893924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:18:59.893924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:18:59.893924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:18:59.893924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:18:59.893924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:18:59.893924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:18:59.893924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:18:59.895148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:18:59.895148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:18:59.895148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:18:59.895148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:18:59.895148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:18:59.895148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:19:00.105494 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:19:00.106094 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:19:00.106288 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:19:00.108542 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:00.108542 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:00.108542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:00.108784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:19:00.108784 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:00.108784 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:19:00.108784 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:19:00.108784 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:00.108784 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:19:00.108784 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:00.108784 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:19:00.110326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:00.110326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:19:00.110326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:00.110326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:19:00.110326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:19:00.110326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:19:00.110326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:19:00.110326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:00.110326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:00.110326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:00.110326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:19:00.110326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:00.110842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:00.110842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:00.110842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:19:00.110842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:00.110842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:19:00.110842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:19:00.110842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:00.110842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:19:00.110842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:00.110842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:19:00.110842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:19:00.110842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:00.110842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:19:00.111186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:00.111186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:19:00.112202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:00.112202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:19:00.112202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:19:00.112202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:19:00.112202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:19:00.112202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:19:00.112202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:00.112202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:19:00.112202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:19:00.112202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:19:00.112202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:19:00.112202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:19:00.112202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:00.112202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:00.112202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:19:00.112202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:19:00.112650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:19:00.112650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:19:00.112650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:00.112650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:00.112650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:00.112650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:19:00.112650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:00.112650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:00.112650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:00.112650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:19:00.112650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:00.112650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:19:00.112650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:19:00.112650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:00.112650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:19:00.112650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:00.112650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:19:00.112650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:19:00.112650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:00.113152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:19:00.113152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:00.113152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:19:00.113152 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:19:00.113152 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:19:00.113152 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:19:00.113344 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:19:00.114280 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:00.114280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:00.114496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:19:00.114496 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:00.114496 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:19:00.114496 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:19:00.114496 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:00.114496 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:19:00.114496 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:00.114496 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:19:00.115315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:00.115315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:19:00.115315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:19:00.115315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:19:00.115315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:00.115315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:19:00.115315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:00.115315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:19:00.115315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:00.115315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:19:00.115315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:00.115315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:19:00.115315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:00.115315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:19:00.115315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:19:00.115737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:00.115737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:19:00.115737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:00.115737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:19:00.115737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:19:00.115737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:00.115737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:19:00.115737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:00.115737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:19:00.116381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:00.116381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:19:00.116381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:00.116381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:19:00.116381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:19:00.116381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:19:00.116381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:19:00.116381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:19:00.116381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:19:00.116381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:19:00.116381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:00.116381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:00.116777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:00.116777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:19:00.116777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:00.116777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:00.116777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:00.116777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:19:00.116777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:00.116777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:19:00.116777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:19:00.116777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:00.116777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:19:00.116777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:00.116777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:19:00.116777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:19:00.117130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:00.117130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:19:00.117130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:00.117130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:19:00.117623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:00.117623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:19:00.117623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:19:00.117623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:19:00.117623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:19:00.117623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:19:00.117623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:00.117623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:19:00.117623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:19:00.117623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:19:00.117623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:19:00.117623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:19:00.117623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:00.117623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:00.117623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:19:00.117623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:19:00.118049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:19:00.118049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:19:00.118049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:19:00.118049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:19:00.118049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:19:00.118049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:00.118049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:00.118049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:00.118049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:19:00.118049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:00.118049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:00.118049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:00.118049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:19:00.118049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:00.118049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:19:00.118049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:19:00.118049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:00.118049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:19:00.118588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:00.118588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:19:00.118588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:19:00.118588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:00.118588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:19:00.118588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:00.118588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:19:00.155651 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:19:00.156253 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:19:00.156963 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:19:00.157802 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:19:00.160750 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:00.160750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:00.161007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:19:00.161007 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:00.161007 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:19:00.161007 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:19:00.161007 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:00.161007 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:19:00.161007 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:00.161007 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:19:00.162254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:00.162254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:19:00.162254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:19:00.162254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:19:00.162254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:00.162254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:19:00.162254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:00.162254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:19:00.162254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:00.162254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:19:00.162254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:00.162254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:19:00.162254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:00.162254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:19:00.162254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:19:00.162954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:00.162954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:19:00.162954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:00.162954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:19:00.162954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:19:00.162954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:00.162954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:19:00.162954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:00.162954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:19:00.163996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:00.163996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:19:00.163996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:00.163996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:19:00.163996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:19:00.163996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:19:00.163996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:19:00.163996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:19:00.163996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:19:00.163996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:00.163996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:00.164598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:00.164598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:19:00.164598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:00.164598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:00.164598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:00.164598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:19:00.164598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:00.164598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:19:00.164598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:19:00.164598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:00.164598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:19:00.164598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:00.164598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:19:00.164598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:19:00.165533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:00.165533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:19:00.165533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:00.165533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:19:00.165970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:00.165970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:19:00.165970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:19:00.165970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:19:00.165970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:19:00.165970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:19:00.165970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:00.165970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:19:00.165970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:19:00.165970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:19:00.165970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:19:00.165970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:19:00.165970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:00.165970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:00.165970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:19:00.165970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:19:00.165970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:19:00.166979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:19:00.166979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:19:00.166979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:19:00.166979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:00.166979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:00.166979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:00.166979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:19:00.166979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:00.166979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:00.166979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:00.166979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:19:00.166979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:00.166979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:19:00.166979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:19:00.166979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:00.166979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:19:00.166979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:00.166979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:19:00.166979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:19:00.167639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:00.167639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:19:00.167639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:00.167639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:19:00.167639 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:19:00.167639 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:19:00.167639 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:19:00.167639 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:19:00.168320 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:00.168668 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:00.168928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:00.169522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:19:00.169522 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:00.169522 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:19:00.169522 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:19:00.169522 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:00.169522 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:19:00.169522 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:00.169522 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:19:00.170105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:00.170105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:19:00.170105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:00.170105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:19:00.170105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:19:00.170105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:19:00.170105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:19:00.170105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:00.170105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:00.170105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:00.170105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:19:00.170105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:00.170470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:00.170470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:00.170470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:19:00.170470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:00.170470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:19:00.170470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:19:00.170470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:00.170470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:19:00.170470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:00.170470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:19:00.170470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:19:00.170470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:00.170470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:19:00.170794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:00.170794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:19:00.170794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:00.170794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:19:00.170794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:19:00.170794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:19:00.170794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:19:00.170794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:19:00.170794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:00.170794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:19:00.170794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:19:00.170794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:19:00.170794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:19:00.170794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:19:00.170794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:00.171204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:00.171204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:19:00.171204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:19:00.171204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:19:00.171204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:19:00.171204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:00.171204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:00.171204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:00.171204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:19:00.171204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:00.171204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:00.171204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:00.171204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:19:00.171204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:00.171204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:19:00.171204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:19:00.171204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:00.171204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:19:00.171204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:00.171729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:19:00.171729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:19:00.171729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:00.171729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:19:00.171729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:00.171729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:19:04.858146 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:19:04.858928 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:19:04.858928 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:19:04.860067 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:19:04.933672 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:04.934030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:04.934873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:19:04.934873 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:04.934873 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:19:04.934873 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:19:04.934873 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:04.934873 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:19:04.934873 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:04.934873 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:19:04.935671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.076 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:04.935671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:19:04.935671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:04.935671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:19:04.935671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:19:04.935671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:19:04.935671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:19:04.935671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:04.935671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:04.935671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:04.935671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:19:04.935671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:04.936325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:04.936325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:04.936325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:19:04.936325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:04.936325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:19:04.936325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:19:04.936325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:04.936325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:19:04.936325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:04.936325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:19:04.936325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:19:04.936325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:04.936325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:19:04.937094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:04.937094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:19:04.937094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:04.937094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:19:04.937094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:19:04.937094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:19:04.937094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:19:04.937094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:19:04.937094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:04.937094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:19:04.937094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:19:04.937094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:19:04.937094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:19:04.937094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:19:04.937094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:04.937840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:04.937840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:19:04.937840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:19:04.937840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:19:04.937840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:19:04.937840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:04.937840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:04.937840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:04.937840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:19:04.937840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:04.937840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:04.937840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:04.937840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:19:04.937840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:04.937840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:19:04.937840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:19:04.937840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:04.937840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:19:04.937840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:04.939321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:19:04.939321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:19:04.939321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:04.939321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:19:04.939321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:04.939321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:19:05.106199 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:19:05.106685 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:19:05.106915 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:19:05.107872 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:05.108266 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:05.108505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:05.109256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:19:05.109256 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:05.109256 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:19:05.109256 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:19:05.109256 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:05.109256 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:19:05.109256 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:05.109256 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:19:05.110644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:05.110644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:19:05.110644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:05.110644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:19:05.110644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:19:05.110644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:19:05.110644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:19:05.110644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:05.110644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:05.110644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:05.110644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:19:05.110644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:05.112113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:05.112113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:05.112113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:19:05.112113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:05.112113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:19:05.112113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:19:05.112113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:05.112113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:19:05.112113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:05.112113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:19:05.112113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:19:05.112113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:05.112113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:19:05.113293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:05.113293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:19:05.113293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:05.113293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:19:05.113293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:19:05.113293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:19:05.113293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:19:05.113293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:19:05.113293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:05.113293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:19:05.113293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:19:05.113293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:19:05.113293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:19:05.113293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:19:05.113293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:05.114159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:05.114159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:19:05.114159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:19:05.114159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:19:05.114159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:19:05.114159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:05.114159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:05.114159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:05.114159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:19:05.114159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:05.114159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:05.114159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:05.114159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:19:05.114159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:05.114159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:19:05.114159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:19:05.114159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:05.114159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:19:05.114159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:05.116647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:19:05.116647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:19:05.116647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:05.116647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:19:05.116647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:05.116647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:19:05.116647 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:19:05.116647 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:19:05.116647 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:19:05.116647 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:19:05.117602 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:05.117602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:05.117602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:19:05.117602 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:05.117602 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:19:05.117602 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:19:05.117602 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:05.117602 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:19:05.117602 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:05.117602 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:19:05.117602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:05.117602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:19:05.117602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:19:05.117602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:19:05.118294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:05.118294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:19:05.118294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:05.118294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:19:05.118294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:05.118294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:19:05.118294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:05.118294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:19:05.118294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:05.118294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:19:05.118294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:19:05.118294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:05.118294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:19:05.118294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:05.118294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:19:05.118294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:19:05.118294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:05.118294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:19:05.119009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:05.119009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:19:05.119009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:05.119009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:19:05.119009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:05.119009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:19:05.119009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:19:05.119009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:19:05.119009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:19:05.119009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:19:05.119009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:19:05.119009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:19:05.119248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:05.119248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:05.119248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:05.119248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:19:05.119248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:05.119248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:05.119248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:05.119248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:19:05.119248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:05.119248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:19:05.119248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:19:05.119248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:05.119248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:19:05.119248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:05.119959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:19:05.119959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:19:05.119959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:05.119959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:19:05.119959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:05.119959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:19:05.119959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:05.119959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:19:05.119959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:19:05.119959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:19:05.119959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:19:05.119959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:19:05.119959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:05.119959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:19:05.120592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:19:05.120592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:19:05.120592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:19:05.120592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:19:05.120592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:05.120592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:05.120592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:19:05.120592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:19:05.120592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:19:05.120592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:19:05.120592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:19:05.120592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:19:05.120592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:19:05.120592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:05.120592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:05.120592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:05.120592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:19:05.120592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:05.120592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:05.121443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:05.121443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:19:05.121443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:05.121443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:19:05.121443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:19:05.121443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:05.121443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:19:05.121443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:05.121443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:19:05.121443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:19:05.121443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:05.121443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:19:05.121443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:05.121443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:19:05.156361 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:19:05.156803 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:19:05.156998 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:19:05.157156 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:19:05.157797 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:05.158167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:05.159063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:19:05.159063 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:05.159063 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:19:05.159063 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:19:05.159063 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:05.159063 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:19:05.159063 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:05.159063 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:19:05.160343 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:05.160343 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:19:05.160343 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:19:05.160343 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:19:05.160343 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:05.160343 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:19:05.160343 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:05.160343 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:19:05.160343 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:05.160343 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:19:05.160343 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:05.160343 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:19:05.160343 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:05.160343 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:19:05.160343 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:19:05.161003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:05.161003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:19:05.161003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:05.161003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:19:05.161003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:19:05.161003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:05.161003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:19:05.161003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:05.161003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:19:05.161815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:05.161815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:19:05.161815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:05.161815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:19:05.161815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:19:05.161815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:19:05.161815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:19:05.161815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:19:05.161815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:19:05.161815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:05.161815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:05.163269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:05.163269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:19:05.163269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:05.163269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:05.163269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:05.163269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:19:05.163269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:05.163269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:19:05.163269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:19:05.163269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:05.163269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:19:05.163269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:05.163269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:19:05.163269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:19:05.164300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:05.164300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:19:05.164300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:05.164300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:19:05.164300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:05.164300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:19:05.164300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:19:05.164300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:19:05.164300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:19:05.164300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:19:05.164300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:05.164300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:19:05.164300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:19:05.164300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:19:05.164777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:19:05.164777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:19:05.164777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:05.164777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:05.164777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:19:05.164777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:19:05.164777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:19:05.164777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:19:05.164777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:19:05.164777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:19:05.164777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:05.164777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:05.164777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:05.164777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:19:05.164777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:05.164777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:05.164777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:05.164777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:19:05.164777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:05.165245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:19:05.165245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:19:05.165245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:05.165245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:19:05.165245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:05.165245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:19:05.165245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:19:05.165245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:05.165245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:19:05.165245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:05.165245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:19:05.166742 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:19:05.167042 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:19:05.167148 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:19:05.167303 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:19:05.167777 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:05.168141 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:05.168377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:05.169107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:19:05.169107 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:05.169107 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:19:05.169107 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:19:05.169107 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:05.169107 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:19:05.169107 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:05.169107 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:19:05.169958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:05.169958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:19:05.169958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:05.169958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:19:05.169958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:19:05.169958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:19:05.169958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:19:05.169958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:05.169958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:05.169958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:05.169958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:19:05.169958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:05.170284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:05.170284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:05.170284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:19:05.170284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:05.170284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:19:05.170284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:19:05.170284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:05.170284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:19:05.170284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:05.170284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:19:05.170284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:19:05.170284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:05.170284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:19:05.170730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:05.170730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:19:05.171140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:05.171140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:19:05.171140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:19:05.171140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:19:05.171140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:19:05.171140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:19:05.171140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:05.171140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:19:05.171140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:19:05.171140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:19:05.171140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:19:05.171140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:19:05.171140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:05.171140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:05.171140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:19:05.171140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:19:05.171581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:19:05.171581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:19:05.171581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:05.171581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:05.171581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:05.171581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:19:05.171581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:05.171581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:05.171581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:05.171581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:19:05.171581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:05.171581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:19:05.171581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:19:05.171581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:05.171581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:19:05.171581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:05.171581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:19:05.171581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:19:05.171581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:05.172031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:19:05.172031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:05.172031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:19:09.858984 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:19:09.859733 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:19:09.859801 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:19:09.861110 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:19:09.894066 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:09.894393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:09.895199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:19:09.895199 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:09.895199 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:19:09.895199 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:19:09.895199 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:09.895199 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:19:09.895199 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:09.895199 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:19:09.895955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:09.895955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:19:09.895955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:09.895955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:19:09.895955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:19:09.895955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:19:09.895955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:19:09.895955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:09.895955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:09.895955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:09.895955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:19:09.895955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:09.896452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:09.896452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:09.896452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:19:09.896452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:09.896452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:19:09.896452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:19:09.896452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:09.896452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:19:09.896452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:09.896452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:19:09.896452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:19:09.896452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:09.896452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:19:09.896916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:09.896916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:19:09.896916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:09.896916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:19:09.896916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:19:09.896916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:19:09.896916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:19:09.896916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:19:09.896916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:09.896916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:19:09.896916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:19:09.896916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:19:09.896916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:19:09.896916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:19:09.896916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:09.897518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:09.897518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:19:09.897518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:19:09.897518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:19:09.897518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:19:09.897518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:09.897518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:09.897518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:09.897518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:19:09.897518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:09.897518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:09.897518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:09.897518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:19:09.897518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:09.897518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:19:09.897518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:19:09.897518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:09.897518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:19:09.897518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:09.898087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:19:09.898087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:19:09.898087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:09.898087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:19:09.898087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:09.898087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:19:10.106787 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:19:10.107443 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:19:10.107764 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:19:10.108933 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:10.109308 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:10.109534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:10.110524 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:19:10.110524 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:10.110524 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:19:10.110524 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:19:10.110524 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:10.110524 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:19:10.110524 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:10.110524 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:19:10.112323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:10.112323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:19:10.112323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:10.112323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:19:10.112323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:19:10.112323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:19:10.112323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:19:10.112323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:10.112323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:10.112323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:10.112323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:19:10.112323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:10.113225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:10.113225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:10.113225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:19:10.113225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:10.113225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:19:10.113225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:19:10.113225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:10.113225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:19:10.113225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:10.113225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:19:10.113225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:19:10.113225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:10.113225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:19:10.114038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:10.114038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:19:10.115328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:10.115328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:19:10.115328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:19:10.115328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:19:10.115328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:19:10.115328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:19:10.115328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:10.115328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:19:10.115328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:19:10.115328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:19:10.115328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:19:10.115328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:19:10.115328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:10.115328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:10.115328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:19:10.115328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:19:10.115977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:19:10.115977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:19:10.115977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:10.115977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:10.115977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:10.115977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:19:10.115977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:10.115977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:10.115977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:10.115977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:19:10.115977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:10.115977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:19:10.115977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:19:10.115977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:10.115977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:19:10.115977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:10.115977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:19:10.115977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:19:10.115977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:10.116643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:19:10.116643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:10.116643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:19:10.116643 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:19:10.116643 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:19:10.116643 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:19:10.116643 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:19:10.116896 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:10.117117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:10.117856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:19:10.117856 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:10.117856 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:19:10.117856 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:19:10.117856 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:10.117856 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:19:10.117856 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:10.117856 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:19:10.119195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:10.119195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:19:10.119195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:19:10.119195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:19:10.119195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:10.119195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:19:10.119195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:10.119195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:19:10.119195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:10.119195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:19:10.119195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:10.119195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:19:10.119195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:10.119195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:19:10.119195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:19:10.119990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:10.119990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:19:10.119990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:10.119990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:19:10.119990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:19:10.119990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:10.119990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:19:10.119990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:10.119990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:19:10.120643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:10.120643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:19:10.120643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:10.120643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:19:10.120643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:19:10.120643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:19:10.120643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:19:10.120643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:19:10.120643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:19:10.120643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:19:10.120643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:10.120643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:10.121190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:10.121190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:19:10.121190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:10.121190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:10.121190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:10.121190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:19:10.121190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:10.121190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:19:10.121190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:19:10.121190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:10.121190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:19:10.121190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:10.121190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:19:10.121190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:19:10.121822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:10.121822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:19:10.121822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:10.121822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:19:10.122064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:10.122064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:19:10.122064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:19:10.122064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:19:10.122064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:19:10.122064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:19:10.122064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:10.122064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:19:10.122064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:19:10.122064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:19:10.122064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:19:10.122064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:19:10.122064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:10.122064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:10.122064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:19:10.122064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:19:10.122682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:19:10.122682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:19:10.122682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:19:10.122682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:19:10.122682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:19:10.122682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:10.122682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:10.122682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:10.122682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:19:10.122682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:10.122682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:10.122682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:10.122682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:19:10.122682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:10.122682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:19:10.122682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:19:10.122682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:10.122682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:19:10.123208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:10.123208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:19:10.123208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:19:10.123208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:10.123208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:19:10.123208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:10.123208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:19:10.156807 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:19:10.157218 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:19:10.157426 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:19:10.157653 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:19:10.158223 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:10.158708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:10.159601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:19:10.159601 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:10.159601 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:19:10.159601 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:19:10.159601 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:10.159601 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:19:10.159601 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:10.159601 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:19:10.160897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:10.160897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:19:10.160897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:19:10.160897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:19:10.160897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:10.160897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:19:10.160897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:10.160897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:19:10.160897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:10.160897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:19:10.160897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:10.160897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:19:10.160897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:10.160897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:19:10.160897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:19:10.161878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:10.161878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:19:10.161878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:10.161878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:19:10.161878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:19:10.161878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:10.161878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:19:10.161878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:10.161878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:19:10.162347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:10.162347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:19:10.162347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:10.162347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:19:10.162347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:19:10.162347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:19:10.162347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:19:10.162347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:19:10.162347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:19:10.162347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:10.162347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:10.163042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:10.163042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:19:10.163042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:10.163042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:10.163042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:10.163042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:19:10.163042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:10.163042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:19:10.163042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:19:10.163042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:10.163042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:19:10.163042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:10.163042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:19:10.163042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:19:10.163948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:10.163948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:19:10.163948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:10.163948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:19:10.164898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:10.164898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:19:10.164898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:19:10.164898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:19:10.164898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:19:10.164898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:19:10.164898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:10.164898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:19:10.164898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:19:10.164898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:19:10.164898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:19:10.164898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:19:10.164898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:10.164898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:10.164898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:19:10.164898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:19:10.164898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:19:10.165948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:19:10.165948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:19:10.165948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:19:10.165948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:10.165948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:10.165948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:10.165948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:19:10.165948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:10.165948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:10.165948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:10.165948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:19:10.165948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:10.165948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:19:10.165948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:19:10.165948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:10.165948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:19:10.165948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:10.165948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:19:10.165948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:19:10.167011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:10.167011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:19:10.167011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:10.167011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:19:10.167206 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:19:10.167711 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:19:10.167823 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:19:10.168008 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:19:10.168551 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:10.168873 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:10.169131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:10.169871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:19:10.169871 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:10.169871 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:19:10.169871 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:19:10.169871 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:10.169871 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:19:10.169871 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:10.169871 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:19:10.170774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:10.170774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:19:10.170774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:10.170774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:19:10.170774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:19:10.170774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:19:10.170774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:19:10.170774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:10.170774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:10.170774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:10.170774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:19:10.170774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:10.171145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:10.171145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:10.171145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:19:10.171145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:10.171145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:19:10.171145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:19:10.171145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:10.171145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:19:10.171145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:10.171145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:19:10.171145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:19:10.171145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:10.171145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:19:10.171562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:10.171562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:19:10.171739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:10.171739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:19:10.171739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:19:10.171739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:19:10.171739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:19:10.171739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:19:10.171739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:10.171739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:19:10.171739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:19:10.171739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:19:10.171739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:19:10.171739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:19:10.171739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:10.171739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:10.171739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:19:10.171739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:19:10.172177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:19:10.172177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:19:10.172177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:10.172177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:10.172177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:10.172177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:19:10.172177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:10.172177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:10.172177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:10.172177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:19:10.172177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:10.172177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:19:10.172177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:19:10.172177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:10.172177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:19:10.172177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:10.172177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:19:10.172177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:19:10.172177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:10.172725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:19:10.172725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:10.172725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:19:14.859702 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:19:14.860696 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:19:14.860696 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:19:14.861580 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:19:14.897568 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:14.898036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:14.899022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:19:14.899022 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:14.899022 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:19:14.899022 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:19:14.899022 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:14.899022 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:19:14.899022 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:14.899022 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:19:14.899991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:14.899991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:19:14.899991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:14.899991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:19:14.899991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:19:14.899991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:19:14.899991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:19:14.899991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:14.899991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:14.899991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:14.899991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:19:14.899991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:14.900720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:14.900720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:14.900720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:19:14.900720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:14.900720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:19:14.900720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:19:14.900720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:14.900720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:19:14.900720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:14.900720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:19:14.900720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:19:14.900720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:14.900720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:19:14.901460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:14.901460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:19:14.901460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:14.901460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:19:14.901460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:19:14.901460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:19:14.901460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:19:14.901460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:19:14.901460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:14.901460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:19:14.901460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:19:14.901460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:19:14.901460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:19:14.901460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:19:14.901460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:14.902284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:14.902284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:19:14.902284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:19:14.902284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:19:14.902284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:19:14.902284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:14.902284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:14.902284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:14.902284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:19:14.902284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:14.902284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:14.902284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:14.902284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:19:14.902284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:14.902284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:19:14.902284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:19:14.902284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:14.902284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:19:14.902284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:14.903273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:19:14.903273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:19:14.903273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:14.903273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:19:14.903273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:14.903273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:19:15.107349 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:19:15.107764 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:19:15.107874 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:19:15.108841 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:15.109188 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:15.109500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:15.110309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:19:15.110309 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:15.110309 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:19:15.110309 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:19:15.110309 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:15.110309 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:19:15.110309 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:15.110309 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:19:15.111004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:15.111004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:19:15.111004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:15.111004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:19:15.111004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:19:15.111004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:19:15.111004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:19:15.111004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:15.111004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:15.111004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:15.111004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:19:15.111004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:15.111645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:15.111645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:15.111645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:19:15.111645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:15.111645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:19:15.111645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:19:15.111645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:15.111645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:19:15.111645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:15.111645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:19:15.111645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:19:15.111645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:15.111645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:19:15.112270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:15.112270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:19:15.112270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:15.112270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:19:15.112270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:19:15.112270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:19:15.112270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:19:15.112270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:19:15.112270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:15.112270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:19:15.112270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:19:15.112270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:19:15.112270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:19:15.112270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:19:15.112270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:15.113063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:15.113063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:19:15.113063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:19:15.113063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:19:15.113063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:19:15.113063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:15.113063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:15.113063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:15.113063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:19:15.113063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:15.113063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:15.113063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:15.113063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:19:15.113063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:15.113063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:19:15.113063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:19:15.113063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:15.113063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:19:15.113063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:15.114211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:19:15.114211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:19:15.114211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:15.114211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:19:15.114211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:15.114211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:19:15.116094 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:19:15.116428 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:19:15.116505 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:19:15.116762 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:19:15.116905 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:15.117131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:15.117602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:19:15.117602 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:15.117602 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:19:15.117602 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:19:15.117602 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:15.117602 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:19:15.117602 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:15.117602 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:19:15.118126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:15.118126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:19:15.118126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:19:15.118126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:19:15.118126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:15.118126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:19:15.118126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:15.118126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:19:15.118126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:15.118126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:19:15.118126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:15.118126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:19:15.118126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:15.118126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:19:15.118126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:19:15.118590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:15.118590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:19:15.118590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:15.118590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:19:15.118590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:19:15.118590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:15.118590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:19:15.118590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:15.118590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:19:15.118848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:15.118848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:19:15.118848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:15.118848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:19:15.118848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:19:15.118848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:19:15.118848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:19:15.118848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:19:15.118848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:19:15.118848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:19:15.118848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:15.118848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:15.119202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:15.119202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:19:15.119202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:15.119202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:15.119202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:15.119202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:19:15.119202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:15.119202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:19:15.119202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:19:15.119202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:15.119202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:19:15.119202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:15.119202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:19:15.119202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:19:15.119593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:15.119593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:19:15.119593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:15.119593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:19:15.119593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:15.119593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:19:15.119593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:19:15.119593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:19:15.119593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:19:15.119593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:19:15.119593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:15.119593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:19:15.119593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:19:15.119593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:19:15.119972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:19:15.119972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:19:15.119972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:15.119972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:15.119972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:19:15.119972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:19:15.119972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:19:15.119972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:19:15.119972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:19:15.119972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:19:15.119972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:19:15.119972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:15.119972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:15.119972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:15.119972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:19:15.119972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:15.119972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:15.119972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:15.119972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:19:15.120520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:15.120520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:19:15.120520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:19:15.120520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:15.120520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:19:15.120520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:15.120520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:19:15.120520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:19:15.120520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:15.120520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:19:15.120520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:15.120520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:19:15.157369 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:19:15.157978 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:19:15.158140 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:19:15.158352 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:19:15.158939 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:15.159321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:15.160127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:19:15.160127 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:15.160127 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:19:15.160127 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:19:15.160127 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:15.160127 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:19:15.160127 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:15.160127 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:19:15.161417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:15.161417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:19:15.161417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:19:15.161417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:19:15.161417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:15.161417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:19:15.161417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:15.161417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:19:15.161417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:15.161417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:19:15.161417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:15.161417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:19:15.161417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:15.161417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:19:15.161417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:19:15.161845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:15.161845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:19:15.161845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:15.161845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:19:15.161845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:19:15.161845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:15.161845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:19:15.161845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:15.161845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:19:15.162263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:15.162263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:19:15.162263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:15.162263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:19:15.162263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:19:15.162263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:19:15.162263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:19:15.162263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:19:15.162263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:19:15.162263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:15.162263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:15.162652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:15.162652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:19:15.162652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:15.162652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:15.162652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:15.162652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:19:15.162652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:15.162652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:19:15.162652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:19:15.162652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:15.162652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:19:15.162652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:15.162652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:19:15.162652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:19:15.163031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:15.163031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:19:15.163031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:15.163031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:19:15.163425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:15.163425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:19:15.163425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:19:15.163425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:19:15.163425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:19:15.163425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:19:15.163425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:15.163425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:19:15.163425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:19:15.163425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:19:15.163425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:19:15.163425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:19:15.163425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:15.163425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:15.163425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:19:15.163425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:19:15.163425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:19:15.163923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:19:15.163923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:19:15.163923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:19:15.163923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:15.163923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:15.163923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:15.163923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:19:15.163923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:15.163923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:15.163923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:15.163923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:19:15.163923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:15.163923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:19:15.163923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:19:15.163923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:15.163923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:19:15.163923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:15.163923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:19:15.163923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:19:15.164436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:15.164436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:19:15.164436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:15.164436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:19:15.167903 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:19:15.168214 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:19:15.168321 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:19:15.168493 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:19:15.168970 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:15.169313 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:15.169616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:15.170296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:19:15.170296 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:15.170296 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:19:15.170296 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:19:15.170296 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:15.170296 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:19:15.170296 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:15.170296 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:19:15.171169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:15.171169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:19:15.171169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:15.171169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:19:15.171169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:19:15.171169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:19:15.171169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:19:15.171169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:15.171169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:15.171169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:15.171169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:19:15.171169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:15.171536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:15.171536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:15.171536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:19:15.171536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:15.171536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:19:15.171536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:19:15.171536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:15.171536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:19:15.171536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:15.171536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:19:15.171536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:19:15.171536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:15.171536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:19:15.171884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:15.171884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:19:15.172339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:15.172339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:19:15.172339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:19:15.172339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:19:15.172339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:19:15.172339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:19:15.172339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:15.172339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:19:15.172339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:19:15.172339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:19:15.172339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:19:15.172339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:19:15.172339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:15.172339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:15.172339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:19:15.172339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:19:15.172813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:19:15.172813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:19:15.172813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:15.172813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:15.172813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:15.172813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:19:15.172813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:15.172813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:15.172813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:15.172813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:19:15.172813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:15.172813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:19:15.172813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:19:15.172813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:15.172813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:19:15.172813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:15.172813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:19:15.172813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:19:15.172813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:15.173296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:19:15.173296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:15.173296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:19:19.860304 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:19:19.861184 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:19:19.861184 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:19:19.861716 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:19:19.894080 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:19.894500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:19.895400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:19:19.895400 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:19.895400 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:19:19.895400 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:19:19.895400 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:19.895400 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:19:19.895400 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:19.895400 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:19:19.896422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:19.896422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:19:19.896422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:19.896422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:19:19.896422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:19:19.896422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:19:19.896422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:19:19.896422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:19.896422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:19.896422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:19.896422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:19:19.896422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:19.897058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:19.897058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:19.897058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:19:19.897058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:19.897058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:19:19.897058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:19:19.897058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:19.897058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:19:19.897058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:19.897058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:19:19.897058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:19:19.897058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:19.897058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:19:19.897635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:19.897635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:19:19.897635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:19.897635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:19:19.897635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:19:19.897635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:19:19.897635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:19:19.897635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:19:19.897635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:19.897635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:19:19.897635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:19:19.897635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:19:19.897635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:19:19.897635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:19:19.897635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:19.898298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:19.898298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:19:19.898298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:19:19.898298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:19:19.898298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:19:19.898298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:19.898298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:19.898298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:19.898298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:19:19.898298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:19.898298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:19.898298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:19.898298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:19:19.898298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:19.898298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:19:19.898298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:19:19.898298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:19.898298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:19:19.898298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:19.899139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:19:19.899139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:19:19.899139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:19.899139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:19:19.899139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:19.899139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:19:20.107964 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:19:20.108387 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:19:20.108669 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:19:20.109905 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:20.110227 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:20.110564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:20.111320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:19:20.111320 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:20.111320 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:19:20.111320 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:19:20.111320 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:20.111320 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:19:20.111320 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:20.111320 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:19:20.112482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:20.112482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:19:20.112482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:20.112482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:19:20.112482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:19:20.112482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:19:20.112482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:19:20.112482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:20.112482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:20.112482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:20.112482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:19:20.112482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:20.112859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:20.112859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:20.112859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:19:20.112859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:20.112859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:19:20.112859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:19:20.112859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:20.112859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:19:20.112859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:20.112859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:19:20.112859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:19:20.112859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:20.112859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:19:20.113233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:20.113233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:19:20.113860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:20.113860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:19:20.113860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:19:20.113860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:19:20.113860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:19:20.113860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:19:20.113860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:20.113860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:19:20.113860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:19:20.113860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:19:20.113860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:19:20.113860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:19:20.113860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:20.113860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:20.113860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:19:20.113860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:19:20.114619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:19:20.114619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:19:20.114619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:20.114619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:20.114619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:20.114619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:19:20.114619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:20.114619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:20.114619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:20.114619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:19:20.114619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:20.114619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:19:20.114619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:19:20.114619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:20.114619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:19:20.114619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:20.114619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:19:20.114619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:19:20.114619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:20.115268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:19:20.115268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:20.115268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:19:20.116525 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:19:20.116791 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:19:20.116929 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:19:20.117133 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:19:20.117656 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:20.117955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:20.118733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:19:20.118733 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:20.118733 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:19:20.118733 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:19:20.118733 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:20.118733 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:19:20.118733 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:20.118733 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:19:20.119617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:20.119617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:19:20.119617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:19:20.119617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:19:20.119617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:20.119617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:19:20.119617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:20.119617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:19:20.119617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:20.119617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:19:20.119617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:20.119617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:19:20.119617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:20.119617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:19:20.119617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:19:20.120073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:20.120073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:19:20.120073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:20.120073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:19:20.120073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:19:20.120073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:20.120073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:19:20.120073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:20.120073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:19:20.120700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:20.120700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:19:20.120700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:20.120700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:19:20.120700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:19:20.120700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:19:20.120700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:19:20.120700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:19:20.120700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:19:20.120700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:19:20.120700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:20.120700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:20.121102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:20.121102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:19:20.121102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:20.121102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:20.121102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:20.121102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:19:20.121102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:20.121102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:19:20.121102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:19:20.121102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:20.121102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:19:20.121102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:20.121102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:19:20.121102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:19:20.121578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:20.121578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:19:20.121578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:20.121578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:19:20.122169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:20.122169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:19:20.122169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:19:20.122169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:19:20.122169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:19:20.122169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:19:20.122169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:20.122169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:19:20.122169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:19:20.122169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:19:20.122169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:19:20.122169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:19:20.122169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:20.122169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:20.122169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:19:20.122169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:19:20.122671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:19:20.122671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:19:20.122671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:19:20.122671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:19:20.122671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:19:20.122671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:20.122671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:20.122671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:20.122671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:19:20.122671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:20.122671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:20.122671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:20.122671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:19:20.122671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:20.122671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:19:20.122671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:19:20.122671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:20.122671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:19:20.123169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:20.123169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:19:20.123169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:19:20.123169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:20.123169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:19:20.123169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:20.123169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:19:20.158094 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:19:20.158999 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:19:20.159227 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:19:20.159654 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:19:20.160695 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:20.161098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:20.162165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:19:20.162165 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:20.162165 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:19:20.162165 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:19:20.162165 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:20.162165 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:19:20.162165 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:20.162165 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:19:20.163163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:20.163163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:19:20.163163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:19:20.163163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:19:20.163163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:20.163163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:19:20.163163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:20.163163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:19:20.163163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:20.163163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:19:20.163163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:20.163163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:19:20.163163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:20.163163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:19:20.163163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:19:20.163575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:20.163575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:19:20.163575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:20.163575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:19:20.163575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:19:20.163575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:20.163575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:19:20.163575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:20.163575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:19:20.164433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:20.164433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:19:20.164433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:20.164433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:19:20.164433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:19:20.164433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:19:20.164433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:19:20.164433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:19:20.164433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:19:20.164433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:20.164433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:20.164759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:20.164759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:19:20.164759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:20.164759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:20.164759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:20.164759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:19:20.164759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:20.164759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:19:20.164759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:19:20.164759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:20.164759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:19:20.164759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:20.164759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:19:20.164759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:19:20.165182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:20.165182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:19:20.165182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:20.165182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:19:20.165869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:20.165869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:19:20.165869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:19:20.165869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:19:20.165869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:19:20.165869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:19:20.165869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:20.165869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:19:20.165869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:19:20.165869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:19:20.165869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:19:20.165869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:19:20.165869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:20.165869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:20.165869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:19:20.165869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:19:20.165869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:19:20.166333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:19:20.166333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:19:20.166333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:19:20.166333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:20.166333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:20.166333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:20.166333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:19:20.166333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:20.166333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:20.166333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:20.166333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:19:20.166333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:20.166333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:19:20.166333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:19:20.166333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:20.166333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:19:20.166333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:20.166333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:19:20.166333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:19:20.166853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:20.166853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:19:20.166853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:20.166853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:19:20.168968 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:19:20.168968 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:19:20.168968 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:19:20.168968 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:19:20.169958 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:20.170321 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:20.170559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:20.171273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:19:20.171273 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:20.171273 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:19:20.171273 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:19:20.171273 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:20.171273 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:19:20.171273 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:20.171273 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:19:20.171888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:20.171888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:19:20.171888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:20.171888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:19:20.171888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:19:20.171888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:19:20.171888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:19:20.171888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:20.171888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:20.171888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:20.171888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:19:20.171888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:20.172285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:20.172285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:20.172285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:19:20.172285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:20.172285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:19:20.172285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:19:20.172285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:20.172285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:19:20.172285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:20.172285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:19:20.172285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:19:20.172285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:20.172285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:19:20.172673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:20.172673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:19:20.172673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:20.172673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:19:20.172673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:19:20.172673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:19:20.172673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:19:20.172673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:19:20.172673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:20.172673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:19:20.172673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:19:20.172673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:19:20.172673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:19:20.172673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:19:20.172673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:20.173158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:20.173158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:19:20.173158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:19:20.173158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:19:20.173158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:19:20.173158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:20.173158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:20.173158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:20.173158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:19:20.173158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:20.173158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:20.173158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:20.173158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:19:20.173158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:20.173158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:19:20.173158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:19:20.173158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:20.173158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:19:20.173158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:20.173559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:19:20.173559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:19:20.173559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:20.173559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:19:20.173559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:20.173559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:19:24.860987 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:19:24.861687 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:19:24.861803 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:19:24.863114 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:19:24.865127 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG dbcounter [-] [105581] Writing DB stats neutron:SELECT=239 {{(pid=105581) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 14:19:24.896165 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:24.896291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:24.896788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:19:24.896788 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:24.896788 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:19:24.896788 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:19:24.896788 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:24.896788 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:19:24.896788 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:24.896788 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:19:24.897372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:24.897372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:19:24.897372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:24.897372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:19:24.897372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:19:24.897372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:19:24.897372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:19:24.897372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:24.897372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:24.897372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:24.897372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:19:24.897372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:24.897991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:24.897991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:24.897991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:19:24.897991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:24.897991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:19:24.897991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:19:24.897991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:24.897991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:19:24.897991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:24.897991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:19:24.897991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:19:24.897991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:24.897991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:19:24.898827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:24.898827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:19:24.898827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:24.898827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:19:24.898827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:19:24.898827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:19:24.898827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:19:24.898827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:19:24.898827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:24.898827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:19:24.898827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:19:24.898827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:19:24.898827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:19:24.898827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:19:24.898827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:24.900574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:24.900574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:19:24.900574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:19:24.900574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:19:24.900574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:19:24.900574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:24.900574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:24.900574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:24.900574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:19:24.900574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:24.900574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:24.900574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:24.900574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:19:24.900574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:24.900574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:19:24.900574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:19:24.900574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:24.900574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:19:24.900574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:24.901250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:19:24.901250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:19:24.901250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:24.901250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:19:24.901250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:24.901250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:19:25.108497 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:19:25.109057 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:19:25.109241 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:19:25.110144 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:25.110424 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:25.110674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:25.111450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:19:25.111450 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:25.111450 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:19:25.111450 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:19:25.111450 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:25.111450 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:19:25.111450 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:25.111450 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:19:25.112872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:25.112872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:19:25.112872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:25.112872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:19:25.112872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:19:25.112872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:19:25.112872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:19:25.112872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:25.112872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:25.112872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:25.112872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:19:25.112872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:25.113457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:25.113457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:25.113457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:19:25.113457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:25.113457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:19:25.113457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:19:25.113457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:25.113457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:19:25.113457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:25.113457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:19:25.113457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:19:25.113457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:25.113457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:19:25.114016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:25.114016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:19:25.114623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:25.114623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:19:25.114623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:19:25.114623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:19:25.114623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:19:25.114623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:19:25.114623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:25.114623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:19:25.114623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:19:25.114623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:19:25.114623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:19:25.114623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:19:25.114623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:25.114623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:25.114623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:19:25.114623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:19:25.115316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:19:25.115316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:19:25.115316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:25.115316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:25.115316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:25.115316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:19:25.115316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:25.115316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:25.115316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:25.115316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:19:25.115316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:25.115316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:19:25.115316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:19:25.115316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:25.115316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:19:25.115316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:25.115316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:19:25.115316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:19:25.115316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:25.116341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:19:25.116341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:25.116341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:19:25.116987 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:19:25.117212 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:19:25.117311 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:19:25.117443 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:19:25.117698 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:25.117902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:25.118371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:19:25.118371 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:25.118371 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:19:25.118371 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:19:25.118371 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:25.118371 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:19:25.118371 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:25.118371 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:19:25.119081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:25.119081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:19:25.119081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:19:25.119081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:19:25.119081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:25.119081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:19:25.119081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:25.119081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:19:25.119081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:25.119081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:19:25.119081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:25.119081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:19:25.119081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:25.119081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:19:25.119081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:19:25.119494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:25.119494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:19:25.119494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:25.119494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:19:25.119494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:19:25.119494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:25.119494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:19:25.119494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:25.119494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:19:25.119896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:25.119896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:19:25.119896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:25.119896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:19:25.119896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:19:25.119896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:19:25.119896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:19:25.119896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:19:25.119896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:19:25.119896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:19:25.119896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:25.119896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:25.120236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:25.120236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:19:25.120236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:25.120236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:25.120236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:25.120236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:19:25.120236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:25.120236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:19:25.120236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:19:25.120236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:25.120236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:19:25.120236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:25.120236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:19:25.120236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:19:25.120596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:25.120596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:19:25.120596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:25.120596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:19:25.120909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:25.120909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:19:25.120909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:19:25.120909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:19:25.120909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:19:25.120909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:19:25.120909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:25.120909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:19:25.120909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:19:25.120909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:19:25.120909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:19:25.120909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:19:25.120909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:25.120909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:25.120909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:19:25.120909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:19:25.121323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:19:25.121323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:19:25.121323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:19:25.121323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:19:25.121323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:19:25.121323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:25.121323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:25.121323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:25.121323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:19:25.121323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:25.121323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:25.121323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:25.121323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:19:25.121323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:25.121323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:19:25.121323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:19:25.121323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:25.121323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:19:25.121836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:25.121836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:19:25.121836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:19:25.121836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:25.121836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:19:25.121836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:25.121836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:19:25.159620 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:19:25.159904 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:19:25.160151 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:19:25.160287 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:19:25.160754 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:25.161110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:25.161797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:19:25.161797 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:25.161797 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:19:25.161797 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:19:25.161797 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:25.161797 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:19:25.161797 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:25.161797 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:19:25.162552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:25.162552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:19:25.162552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:19:25.162552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:19:25.162552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:25.162552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:19:25.162552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:25.162552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:19:25.162552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:25.162552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:19:25.162552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:25.162552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:19:25.162552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:25.162552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:19:25.162552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:19:25.162831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:25.162831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:19:25.162831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:25.162831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:19:25.162831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:19:25.162831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:25.162831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:19:25.162831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:25.162831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:19:25.163392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:25.163392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:19:25.163392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:25.163392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:19:25.163392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:19:25.163392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:19:25.163392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:19:25.163392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:19:25.163392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:19:25.163392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:25.163392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:25.163937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:25.163937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:19:25.163937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:25.163937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:25.163937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:25.163937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:19:25.163937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:25.163937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:19:25.163937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:19:25.163937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:25.163937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:19:25.163937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:25.163937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:19:25.163937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:19:25.164542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:25.164542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:19:25.164542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:25.164542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:19:25.164542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:25.164542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:19:25.164542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:19:25.164542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:19:25.164542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:19:25.164542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:19:25.164542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:25.164542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:19:25.164542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:19:25.164542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:19:25.165241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:19:25.165241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:19:25.165241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:25.165241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:25.165241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:19:25.165241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:19:25.165241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:19:25.165241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:19:25.165241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:19:25.165241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:19:25.165241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:25.165241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:25.165241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:25.165241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:19:25.165241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:25.165241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:25.165241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:25.165241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:19:25.165241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:25.166140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:19:25.166140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:19:25.166140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:25.166140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:19:25.166140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:25.166140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:19:25.166140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:19:25.166140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:25.166140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:19:25.166140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:25.166140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:19:25.168597 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:19:25.168843 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:19:25.168946 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:19:25.169113 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:19:25.169386 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:25.169590 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:25.169729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:25.170237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:19:25.170237 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:25.170237 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:19:25.170237 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:19:25.170237 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:25.170237 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:19:25.170237 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:25.170237 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:19:25.171055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:25.171055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:19:25.171055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:25.171055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:19:25.171055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:19:25.171055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:19:25.171055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:19:25.171055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:25.171055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:25.171055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:25.171055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:19:25.171055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:25.171675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:25.171675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:25.171675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:19:25.171675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:25.171675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:19:25.171675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:19:25.171675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:25.171675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:19:25.171675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:25.171675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:19:25.171675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:19:25.171675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:25.171675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:19:25.172100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:25.172100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:19:25.172100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:25.172100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:19:25.172100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:19:25.172100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:19:25.172100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:19:25.172100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:19:25.172100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:25.172100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:19:25.172100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:19:25.172100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:19:25.172100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:19:25.172100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:19:25.172100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:25.172536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:25.172536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:19:25.172536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:19:25.172536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:19:25.172536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:19:25.172536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:25.172536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:25.172536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:25.172536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:19:25.172536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:25.172536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:25.172536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:25.172536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:19:25.172536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:25.172536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:19:25.172536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:19:25.172536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:25.172536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:19:25.172536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:25.173047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:19:25.173047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:19:25.173047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:25.173047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:19:25.173047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:25.173047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:19:29.861846 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:19:29.862510 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:19:29.862689 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:19:29.863837 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:19:29.899033 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:29.899442 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:29.900318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:19:29.900318 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:29.900318 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:19:29.900318 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:19:29.900318 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:29.900318 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:19:29.900318 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:29.900318 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:19:29.901214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:29.901214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:19:29.901214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:29.901214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:19:29.901214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:19:29.901214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:19:29.901214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:19:29.901214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:29.901214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:29.901214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:29.901214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:19:29.901214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:29.901859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:29.901859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:29.901859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:19:29.901859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:29.901859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:19:29.901859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:19:29.901859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:29.901859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:19:29.901859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:29.901859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:19:29.901859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:19:29.901859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:29.901859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:19:29.902508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:29.902508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:19:29.902508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:29.902508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:19:29.902508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:19:29.902508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:19:29.902508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:19:29.902508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:19:29.902508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:29.902508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:19:29.902508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:19:29.902508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:19:29.902508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:19:29.902508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:19:29.902508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:29.903307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:29.903307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:19:29.903307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:19:29.903307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:19:29.903307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:19:29.903307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:29.903307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:29.903307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:29.903307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:19:29.903307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:29.903307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:29.903307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:29.903307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:19:29.903307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:29.903307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:19:29.903307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:19:29.903307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:29.903307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:19:29.903307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:29.904245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:19:29.904245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:19:29.904245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:29.904245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:19:29.904245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:29.904245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:19:30.109175 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:19:30.109618 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:19:30.109807 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:19:30.110753 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:30.111067 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:30.111356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:30.112503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:19:30.112503 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:30.112503 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:19:30.112503 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:19:30.112503 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:30.112503 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:19:30.112503 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:30.112503 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:19:30.113962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:30.113962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:19:30.113962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:30.113962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:19:30.113962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:19:30.113962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:19:30.113962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:19:30.113962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:30.113962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:30.113962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:30.113962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:19:30.113962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:30.114701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:30.114701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:30.114701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:19:30.114701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:30.114701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:19:30.114701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:19:30.114701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:30.114701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:19:30.114701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:30.114701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:19:30.114701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:19:30.114701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:30.114701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:19:30.115271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:30.115271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:19:30.115856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:30.115856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:19:30.115856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:19:30.115856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:19:30.115856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:19:30.115856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:19:30.115856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:30.115856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:19:30.115856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:19:30.115856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:19:30.115856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:19:30.115856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:19:30.115856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:30.115856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:30.115856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:19:30.115856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:19:30.116436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:19:30.116436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:19:30.116436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:30.116436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:30.116436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:30.116436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:19:30.116436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:30.116436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:30.116436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:30.116436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:19:30.116436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:30.116436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:19:30.116436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:19:30.116436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:30.116436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:19:30.116436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:30.116436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:19:30.116436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:19:30.116436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:30.116966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:19:30.116966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:30.116966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:19:30.117267 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:19:30.117573 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:19:30.117676 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:19:30.117793 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:19:30.118106 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:30.118339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:30.118817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:19:30.118817 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:30.118817 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:19:30.118817 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:19:30.118817 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:30.118817 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:19:30.118817 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:30.118817 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:19:30.119539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:30.119539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:19:30.119539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:19:30.119539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:19:30.119539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:30.119539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:19:30.119539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:30.119539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:19:30.119539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:30.119539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:19:30.119539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:30.119539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:19:30.119539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:30.119539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:19:30.119539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:19:30.120109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:30.120109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:19:30.120109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:30.120109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:19:30.120109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:19:30.120109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:30.120109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:19:30.120109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:30.120109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:19:30.120513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:30.120513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:19:30.120513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:30.120513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:19:30.120513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:19:30.120513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:19:30.120513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:19:30.120513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:19:30.120513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:19:30.120513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:19:30.120513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:30.120513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:30.120868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:30.120868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:19:30.120868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:30.120868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:30.120868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:30.120868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:19:30.120868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:30.120868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:19:30.120868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:19:30.120868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:30.120868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:19:30.120868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:30.120868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:19:30.120868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:19:30.121250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:30.121250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:19:30.121250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:30.121250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:19:30.121385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:30.121385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:19:30.121385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:19:30.121385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:19:30.121385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:19:30.121385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:19:30.121385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:30.121385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:19:30.121385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:19:30.121385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:19:30.121385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:19:30.121385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:19:30.121385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:30.121385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:30.121385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:19:30.121385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:19:30.121906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:19:30.121906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:19:30.121906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:19:30.121906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:19:30.121906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:19:30.121906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:30.121906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:30.121906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:30.121906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:19:30.121906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:30.121906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:30.121906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:30.121906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:19:30.121906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:30.121906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:19:30.121906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:19:30.121906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:30.121906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:19:30.122491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:30.122491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:19:30.122491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:19:30.122491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:30.122491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:19:30.122491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:30.122491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:19:30.160122 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:19:30.160600 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:19:30.160790 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:19:30.160991 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:19:30.161556 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:30.161898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:30.162671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:19:30.162671 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:30.162671 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:19:30.162671 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:19:30.162671 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:30.162671 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:19:30.162671 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:30.162671 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:19:30.163843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:30.163843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:19:30.163843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:19:30.163843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:19:30.163843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:30.163843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:19:30.163843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:30.163843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:19:30.163843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:30.163843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:19:30.163843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:30.163843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:19:30.163843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:30.163843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:19:30.163843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:19:30.164235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:30.164235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:19:30.164235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:30.164235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:19:30.164235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:19:30.164235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:30.164235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:19:30.164235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:30.164235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:19:30.165254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:30.165254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:19:30.165254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:30.165254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:19:30.165254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:19:30.165254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:19:30.165254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:19:30.165254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:19:30.165254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:19:30.165254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:30.165254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:30.165595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:30.165595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:19:30.165595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:30.165595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:30.165595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:30.165595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:19:30.165595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:30.165595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:19:30.165595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:19:30.165595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:30.165595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:19:30.165595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:30.165595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:19:30.165595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:19:30.165952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:30.165952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:19:30.165952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:30.165952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:19:30.166376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:30.166376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:19:30.166376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:19:30.166376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:19:30.166376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:19:30.166376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:19:30.166376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:30.166376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:19:30.166376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:19:30.166376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:19:30.166376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:19:30.166376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:19:30.166376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:30.166376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:30.166376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:19:30.166376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:19:30.166376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:19:30.167022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:19:30.167022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:19:30.167022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:19:30.167022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:30.167022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:30.167022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:30.167022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:19:30.167022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:30.167022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:30.167022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:30.167022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:19:30.167022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:30.167022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:19:30.167022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:19:30.167022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:30.167022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:19:30.167022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:30.167022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:19:30.167022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:19:30.167659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:30.167659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:19:30.167659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:30.167659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:19:30.168893 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:19:30.169113 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:19:30.169224 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:19:30.169370 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:19:30.169646 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:30.169850 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:30.169986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:30.170460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:19:30.170460 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:30.170460 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:19:30.170460 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:19:30.170460 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:30.170460 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:19:30.170460 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:30.170460 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:19:30.171278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:30.171278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:19:30.171278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:30.171278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:19:30.171278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:19:30.171278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:19:30.171278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:19:30.171278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:30.171278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:30.171278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:30.171278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:19:30.171278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:30.171765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:30.171765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:30.171765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:19:30.171765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:30.171765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:19:30.171765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:19:30.171765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:30.171765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:19:30.171765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:30.171765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:19:30.171765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:19:30.171765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:30.171765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:19:30.172265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:30.172265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:19:30.172265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:30.172265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:19:30.172265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:19:30.172265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:19:30.172265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:19:30.172265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:19:30.172265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:30.172265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:19:30.172265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:19:30.172265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:19:30.172265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:19:30.172265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:19:30.172265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:30.172771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:30.172771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:19:30.172771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:19:30.172771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:19:30.172771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:19:30.172771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:30.172771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:30.172771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:30.172771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:19:30.172771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:30.172771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:30.172771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:30.172771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:19:30.172771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:30.172771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:19:30.172771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:19:30.172771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:30.172771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:19:30.172771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:30.173353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:19:30.173353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:19:30.173353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:30.173353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:19:30.173353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:30.173353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:19:34.270023 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:19:34.271583 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_for_inconsistencies {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:19:34.271583 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Checking Neutron and OVN revision consistency. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:19:34.280517 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Synchronizing Neutron and OVN databases started {{(pid=105581) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:440}}
May 09 14:19:34.280680 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Number of inconsistencies found at create/update: networks=4, routers=1, router_ports=1 {{(pid=105581) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:419}}
May 09 14:19:34.280680 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Fixing resource 0b4a5724-2518-4e33-a6d3-0d92c749f303 (type: networks) at create/update {{(pid=105581) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 14:19:34.312013 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-0b4a5724-2518-4e33-a6d3-0d92c749f303, resource={'id': '0b4a5724-2518-4e33-a6d3-0d92c749f303', 'name': 'private', 'tenant_id': '5796380176014a32af8ff613b146faed', 'project_id': '5796380176014a32af8ff613b146faed', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['23eb2e2e-ec2f-4f98-bc47-7125a2f92952', 'db2636e9-0880-488d-9aa3-e100ce5eeb9f'], 'standard_attr_id': 12, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-09T14:04:20Z', 'updated_at': '2026-05-09T14:04:23Z', 'revision_number': 3, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 52278}, resource_type=networks, if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:34.312459 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-0b4a5724-2518-4e33-a6d3-0d92c749f303, col_values=(('external_ids', {'neutron:network_name': 'private', 'neutron:mtu': '1372', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:34.313219 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-f71b8cb1-fe70-438c-84f9-bbb7008a596d, external_ids_update=None, columns={'tag': 52278}, if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:34.313219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:34.314157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:19:34.314157 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:34.314157 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:19:34.314157 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:19:34.314157 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:34.314157 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:19:34.314157 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:34.314157 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:19:34.315435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Failed to fix resource 0b4a5724-2518-4e33-a6d3-0d92c749f303 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:34.315435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:19:34.315435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 14:19:34.315435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 14:19:34.315435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 14:19:34.315435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 14:19:34.315435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 09 14:19:34.315435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:19:34.315435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:34.315435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:34.315435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:34.315992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:19:34.315992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:34.315992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:34.315992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:34.315992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:19:34.315992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:34.315992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:19:34.315992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:19:34.315992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:34.315992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:19:34.315992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:34.315992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:19:34.315992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:19:34.315992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:34.316774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:19:34.316774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:34.316774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:19:34.316774 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Fixing resource a7d094cd-23f3-4e0c-9e88-24be6ee17b2c (type: networks) at create/update {{(pid=105581) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 14:19:34.345640 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-a7d094cd-23f3-4e0c-9e88-24be6ee17b2c, resource={'id': 'a7d094cd-23f3-4e0c-9e88-24be6ee17b2c', 'name': 'public', 'tenant_id': '5796380176014a32af8ff613b146faed', 'project_id': '5796380176014a32af8ff613b146faed', 'admin_state_up': True, 'mtu': 1430, 'status': 'ACTIVE', 'subnets': ['2f0821e3-c6a2-49d6-80ee-3480a17ba8d2', '644269b7-6b84-4d8a-ad28-d59a7c6b7c5e'], 'standard_attr_id': 18, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': True, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'is_default': True, 'tags': [], 'created_at': '2026-05-09T14:04:26Z', 'updated_at': '2026-05-09T14:04:34Z', 'revision_number': 3, 'provider:network_type': 'flat', 'provider:physical_network': 'public', 'provider:segmentation_id': None}, resource_type=networks, if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:34.345890 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-a7d094cd-23f3-4e0c-9e88-24be6ee17b2c, col_values=(('external_ids', {'neutron:network_name': 'public', 'neutron:mtu': '1430', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'flat', 'neutron:provnet-physical-network': 'public'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false', 'fdb_age_threshold': '300', 'broadcast-arps-to-all-routers': 'true'})), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:34.346282 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-08fea115-9b05-4667-a0d0-487c303084a5, external_ids_update=None, columns={'tag': []}, if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:34.346540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:34.347154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:19:34.347154 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:34.347154 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:19:34.347154 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:19:34.347154 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:34.347154 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:19:34.347154 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:34.347154 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:19:34.347959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Failed to fix resource a7d094cd-23f3-4e0c-9e88-24be6ee17b2c (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:34.347959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:19:34.347959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 14:19:34.347959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 14:19:34.347959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 14:19:34.347959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 14:19:34.347959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 09 14:19:34.347959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:19:34.347959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:34.347959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:34.347959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:34.348527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:19:34.348527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:34.348527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:34.348527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:34.348527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:19:34.348527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:34.348527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:19:34.348527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:19:34.348527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:34.348527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:19:34.348527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:34.348527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:19:34.348527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:19:34.348527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:34.349233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:19:34.349233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:34.349233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:19:34.349233 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Fixing resource 3e80d551-6de5-4370-806b-36d1db5f6fda (type: networks) at create/update {{(pid=105581) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 14:19:34.376048 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-3e80d551-6de5-4370-806b-36d1db5f6fda, resource={'id': '3e80d551-6de5-4370-806b-36d1db5f6fda', 'name': 'admin_net', 'tenant_id': '590c644b03ae4bed862ecf492c6b2eb2', 'project_id': '590c644b03ae4bed862ecf492c6b2eb2', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['453849e5-22a7-43ca-911e-9401ed2b0d74'], 'standard_attr_id': 42, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-09T14:07:41Z', 'updated_at': '2026-05-09T14:07:42Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 35952}, resource_type=networks, if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:34.376436 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-3e80d551-6de5-4370-806b-36d1db5f6fda, col_values=(('external_ids', {'neutron:network_name': 'admin_net', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:34.376941 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-f84050fb-ff65-46ce-b744-016e1ff15440, external_ids_update=None, columns={'tag': 35952}, if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:34.377115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:34.377914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:19:34.377914 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:34.377914 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:19:34.377914 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:19:34.377914 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:34.377914 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:19:34.377914 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:34.377914 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:19:34.378893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Failed to fix resource 3e80d551-6de5-4370-806b-36d1db5f6fda (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:34.378893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:19:34.378893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 14:19:34.378893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 14:19:34.378893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 14:19:34.378893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 14:19:34.378893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 09 14:19:34.378893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:19:34.378893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:34.378893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:34.378893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:34.379484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:19:34.379484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:34.379484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:34.379484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:34.379484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:19:34.379484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:34.379484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:19:34.379484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:19:34.379484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:34.379484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:19:34.379484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:34.379484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:19:34.379484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:19:34.379484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:34.380053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:19:34.380053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:34.380053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:19:34.380053 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Fixing resource 632a2ddb-5634-4bb3-aad0-f05d97abd45e (type: networks) at create/update {{(pid=105581) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 14:19:34.405474 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-632a2ddb-5634-4bb3-aad0-f05d97abd45e, resource={'id': '632a2ddb-5634-4bb3-aad0-f05d97abd45e', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-5urlz', 'tenant_id': '1f17f4a2a7ac4543a1253522e61993d2', 'project_id': '1f17f4a2a7ac4543a1253522e61993d2', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['32b523fe-135f-4f9e-a98b-fc0537fcdb35'], 'standard_attr_id': 52, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-09T14:08:15Z', 'updated_at': '2026-05-09T14:08:16Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 39245}, resource_type=networks, if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:34.405921 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-632a2ddb-5634-4bb3-aad0-f05d97abd45e, col_values=(('external_ids', {'neutron:network_name': 'k8s-clusterapi-cluster-magnum-system-kube-5urlz', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:34.406483 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-bca44f95-fe96-432e-84b8-1edc05e10ce0, external_ids_update=None, columns={'tag': 39245}, if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:34.406667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:34.407475 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:19:34.407475 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:34.407475 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:19:34.407475 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:19:34.407475 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:34.407475 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:19:34.407475 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:34.407475 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:19:34.408262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Failed to fix resource 632a2ddb-5634-4bb3-aad0-f05d97abd45e (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:34.408262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:19:34.408262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 14:19:34.408262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 14:19:34.408262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 14:19:34.408262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 14:19:34.408262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 09 14:19:34.408262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:19:34.408262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:34.408262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:34.408262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:34.408833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:19:34.408833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:34.408833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:34.408833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:34.408833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:19:34.408833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:34.408833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:19:34.408833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:19:34.408833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:34.408833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:19:34.408833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:34.408833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:19:34.408833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:19:34.408833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:34.409491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:19:34.409491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:34.409491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:19:34.409491 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Fixing resource 09e042cf-39a9-42c8-b683-b29ffc42b049 (type: routers) at create/update {{(pid=105581) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 14:19:34.487252 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, resource={'id': '09e042cf-39a9-42c8-b683-b29ffc42b049', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-5urlz', 'project_id': '1f17f4a2a7ac4543a1253522e61993d2', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': 'a7d094cd-23f3-4e0c-9e88-24be6ee17b2c', 'external_fixed_ips': [{'subnet_id': '2f0821e3-c6a2-49d6-80ee-3480a17ba8d2', 'ip_address': '172.24.5.175'}, {'subnet_id': '644269b7-6b84-4d8a-ad28-d59a7c6b7c5e', 'ip_address': '2001:db8::d8'}], 'enable_snat': True}, 'gw_port_id': '71ded037-e812-40c0-8d8f-70c141c74323', 'description': 'Created by cluster-api-provider-openstack cluster magnum-system-kube-5urlz', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': 'a7d094cd-23f3-4e0c-9e88-24be6ee17b2c', 'external_fixed_ips': [{'ip_address': '172.24.5.175', 'subnet_id': '2f0821e3-c6a2-49d6-80ee-3480a17ba8d2'}, {'ip_address': '2001:db8::d8', 'subnet_id': '644269b7-6b84-4d8a-ad28-d59a7c6b7c5e'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2026-05-09T14:08:17Z', 'updated_at': '2026-05-09T14:08:19Z', 'revision_number': 3, 'tenant_id': '1f17f4a2a7ac4543a1253522e61993d2'}, resource_type=routers, if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:34.487541 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, columns={'external_ids': {'neutron:router_name': 'k8s-clusterapi-cluster-magnum-system-kube-5urlz', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:34.487787 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-a7d094cd-23f3-4e0c-9e88-24be6ee17b2c, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "71ded037-e812-40c0-8d8f-70c141c74323"')], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:34.487931 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-a7d094cd-23f3-4e0c-9e88-24be6ee17b2c, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "71ded037-e812-40c0-8d8f-70c141c74323"')], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:34.488062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:34.488773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:19:34.488773 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:34.488773 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:19:34.488773 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:19:34.488773 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:34.488773 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:19:34.488773 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:34.488773 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:19:34.489173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Unable to update router 09e042cf-39a9-42c8-b683-b29ffc42b049. Error: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:34.490489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Failed to fix resource 09e042cf-39a9-42c8-b683-b29ffc42b049 (type: routers): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:34.490489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:19:34.490489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 14:19:34.490489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 14:19:34.490489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 14:19:34.490489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 14:19:34.490489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1635, in update_router
May 09 14:19:34.490489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with excutils.save_and_reraise_exception():
May 09 14:19:34.490489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 271, in __exit__
May 09 14:19:34.490489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.force_reraise()
May 09 14:19:34.490489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 233, in force_reraise
May 09 14:19:34.491102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise self.value
May 09 14:19:34.491102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1569, in update_router
May 09 14:19:34.491102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:19:34.491102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:34.491102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:34.491102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:34.491102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:19:34.491102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:34.491102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:34.491102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:34.491102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:19:34.491102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:34.491102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:19:34.491102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:19:34.491577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:34.491577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:19:34.491577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:34.491577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:19:34.491577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:19:34.491577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:34.491577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:19:34.491577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:34.491577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:19:34.491577 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Fixing resource 71ded037-e812-40c0-8d8f-70c141c74323 (type: router_ports) at create/update {{(pid=105581) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 14:19:34.609272 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-71ded037-e812-40c0-8d8f-70c141c74323, resource={'id': '71ded037-e812-40c0-8d8f-70c141c74323', 'name': '', 'network_id': 'a7d094cd-23f3-4e0c-9e88-24be6ee17b2c', 'project_id': '', 'mac_address': 'fa:16:3e:c7:86:37', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '09e042cf-39a9-42c8-b683-b29ffc42b049', 'device_owner': 'network:router_gateway', 'standard_attr_id': 57, 'fixed_ips': [{'subnet_id': '2f0821e3-c6a2-49d6-80ee-3480a17ba8d2', 'ip_address': '172.24.5.175'}, {'subnet_id': '644269b7-6b84-4d8a-ad28-d59a7c6b7c5e', 'ip_address': '2001:db8::d8'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0000173183', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'ovs_create_tap': False, 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2026-05-09T14:08:17Z', 'updated_at': '2026-05-09T14:08:19Z', 'revision_number': 5, 'tenant_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:34.609608 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-71ded037-e812-40c0-8d8f-70c141c74323, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': '2f0821e3-c6a2-49d6-80ee-3480a17ba8d2 644269b7-6b84-4d8a-ad28-d59a7c6b7c5e', 'neutron:network_name': 'neutron-a7d094cd-23f3-4e0c-9e88-24be6ee17b2c', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-09e042cf-39a9-42c8-b683-b29ffc42b049'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['2001:db8::d8/64', '172.24.5.175/24'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:34.609910 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=71ded037-e812-40c0-8d8f-70c141c74323, lrouter_port=lrp-71ded037-e812-40c0-8d8f-70c141c74323, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:34.610139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:34.610914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:19:34.610914 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:34.610914 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:19:34.610914 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:19:34.610914 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:34.610914 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:19:34.610914 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:34.610914 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:19:34.611827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Failed to fix resource 71ded037-e812-40c0-8d8f-70c141c74323 (type: router_ports): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:34.611827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:19:34.611827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 14:19:34.611827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 14:19:34.611827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 14:19:34.611827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 14:19:34.611827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1976, in update_router_port
May 09 14:19:34.611827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:19:34.611827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:34.611827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:34.611827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:34.612196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:19:34.612196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:34.612196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:34.612196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:34.612196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:19:34.612196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:34.612196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:19:34.612196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:19:34.612196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:34.612196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:19:34.612196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:34.612196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:19:34.612196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:19:34.612196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:34.613031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:19:34.613031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:34.613031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:19:34.613031 np0000173183 neutron-ovn-maintenance-worker[105581]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_for_inconsistencies finished in 0.341 seconds
May 09 14:19:34.862881 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:19:34.863427 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:19:34.863612 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:19:34.864918 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:19:34.899713 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:34.900111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:34.901119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:19:34.901119 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:34.901119 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:19:34.901119 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:19:34.901119 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:34.901119 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:19:34.901119 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:34.901119 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:19:34.901865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:34.901865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:19:34.901865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:34.901865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:19:34.901865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:19:34.901865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:19:34.901865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:19:34.901865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:34.901865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:34.901865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:34.901865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:19:34.901865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:34.902569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:34.902569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:34.902569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:19:34.902569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:34.902569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:19:34.902569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:19:34.902569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:34.902569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:19:34.902569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:34.902569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:19:34.902569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:19:34.902569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:34.902569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:19:34.903146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:34.903146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:19:34.903146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:34.903146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:19:34.903146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:19:34.903146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:19:34.903146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:19:34.903146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:19:34.903146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:34.903146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:19:34.903146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:19:34.903146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:19:34.903146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:19:34.903146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:19:34.903146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:34.903924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:34.903924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:19:34.903924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:19:34.903924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:19:34.903924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:19:34.903924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:34.903924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:34.903924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:34.903924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:19:34.903924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:34.903924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:34.903924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:34.903924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:19:34.903924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:34.903924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:19:34.903924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:19:34.903924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:34.903924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:19:34.903924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:34.904830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:19:34.904830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:19:34.904830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:34.904830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:19:34.904830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:34.904830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:19:35.109894 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:19:35.110580 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:19:35.110788 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:19:35.111863 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:35.112204 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:35.112484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:35.113436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:19:35.113436 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:35.113436 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:19:35.113436 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:19:35.113436 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:35.113436 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:19:35.113436 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:35.113436 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:19:35.115176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:35.115176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:19:35.115176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:35.115176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:19:35.115176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:19:35.115176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:19:35.115176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:19:35.115176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:35.115176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:35.115176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:35.115176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:19:35.115176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:35.115809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:35.115809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:35.115809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:19:35.115809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:35.115809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:19:35.115809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:19:35.115809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:35.115809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:19:35.115809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:35.115809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:19:35.115809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:19:35.115809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:35.115809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:19:35.116366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:35.116366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:19:35.117041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:35.117041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:19:35.117041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:19:35.117041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:19:35.117041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:19:35.117041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:19:35.117041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:35.117041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:19:35.117041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:19:35.117041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:19:35.117041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:19:35.117041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:19:35.117041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:35.117041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:35.117041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:19:35.117041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:19:35.117762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:19:35.117762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:19:35.117762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:35.117762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:35.117762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:35.117762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:19:35.117762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:35.117762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:35.117762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:35.117762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:19:35.117762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:35.117762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:19:35.117762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:19:35.117762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:35.117762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:19:35.117762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:35.117762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:19:35.117762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:19:35.117762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:35.118698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:19:35.118698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:35.118698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:19:35.118698 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:19:35.118698 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:19:35.118698 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:19:35.118698 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:19:35.119075 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:35.119075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:35.119797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:19:35.119797 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:35.119797 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:19:35.119797 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:19:35.119797 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:35.119797 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:19:35.119797 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:35.119797 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:19:35.120601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:35.120601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:19:35.120601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:19:35.120601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:19:35.120601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:35.120601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:19:35.120601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:35.120601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:19:35.120601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:35.120601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:19:35.120601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:35.120601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:19:35.120601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:35.120601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:19:35.120601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:19:35.121273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:35.121273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:19:35.121273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:35.121273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:19:35.121273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:19:35.121273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:35.121273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:19:35.121273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:35.121273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:19:35.121705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:35.121705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:19:35.121705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:35.121705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:19:35.121705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:19:35.121705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:19:35.121705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:19:35.121705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:19:35.121705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:19:35.121705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:19:35.121705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:35.121705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:35.122237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:35.122237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:19:35.122237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:35.122237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:35.122237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:35.122237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:19:35.122237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:35.122237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:19:35.122237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:19:35.122237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:35.122237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:19:35.122237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:35.122237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:19:35.122237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:19:35.122836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:35.122836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:19:35.122836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:35.122836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:19:35.122836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:35.122836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:19:35.122836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:19:35.122836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:19:35.122836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:19:35.122836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:19:35.122836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:35.122836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:19:35.122836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:19:35.122836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:19:35.123456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:19:35.123456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:19:35.123456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:35.123456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:35.123456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:19:35.123456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:19:35.123456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:19:35.123456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:19:35.123456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:19:35.123456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:19:35.123456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:19:35.123456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:35.123456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:35.123456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:35.123456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:19:35.123456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:35.123456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:35.123456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:35.123456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:19:35.124276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:35.124276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:19:35.124276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:19:35.124276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:35.124276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:19:35.124276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:35.124276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:19:35.124276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:19:35.124276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:35.124276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:19:35.124276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:35.124276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:19:35.160788 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:19:35.161369 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:19:35.161619 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:19:35.161874 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:19:35.162398 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:35.162827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:35.163621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:19:35.163621 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:35.163621 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:19:35.163621 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:19:35.163621 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:35.163621 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:19:35.163621 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:35.163621 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:19:35.165106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:35.165106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:19:35.165106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:19:35.165106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:19:35.165106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:35.165106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:19:35.165106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:35.165106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:19:35.165106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:35.165106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:19:35.165106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:35.165106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:19:35.165106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:35.165106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:19:35.165106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:19:35.165860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:35.165860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:19:35.165860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:35.165860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:19:35.165860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:19:35.165860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:35.165860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:19:35.165860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:35.165860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:19:35.166669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:35.166669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:19:35.166669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:35.166669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:19:35.166669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:19:35.166669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:19:35.166669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:19:35.166669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:19:35.166669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:19:35.166669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:35.166669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:35.167236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:35.167236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:19:35.167236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:35.167236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:35.167236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:35.167236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:19:35.167236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:35.167236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:19:35.167236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:19:35.167236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:35.167236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:19:35.167236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:35.167236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:19:35.167236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:19:35.167930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:35.167930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:19:35.167930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:35.167930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:19:35.168682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:35.168682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:19:35.168682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:19:35.168682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:19:35.168682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:19:35.168682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:19:35.168682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:35.168682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:19:35.168682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:19:35.168682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:19:35.168682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:19:35.168682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:19:35.168682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:35.168682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:35.168682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:19:35.168682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:19:35.168682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:19:35.169751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:19:35.169751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:19:35.169751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:19:35.169751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:35.169751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:35.169751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:35.169751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:19:35.169751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:35.169751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:35.169751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:35.169751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:19:35.169751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:35.169751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:19:35.169751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:19:35.169751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:35.169751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:19:35.169751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:35.169751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:19:35.169751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:19:35.170630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:35.170630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:19:35.170630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:35.170630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:19:35.170630 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:19:35.170630 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:19:35.170630 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:19:35.170630 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:19:35.171081 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:35.171081 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:35.171081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:35.171565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:19:35.171565 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:35.171565 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:19:35.171565 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:19:35.171565 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:35.171565 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:19:35.171565 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:35.171565 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:19:35.172483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:35.172483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:19:35.172483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:35.172483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:19:35.172483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:19:35.172483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:19:35.172483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:19:35.172483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:35.172483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:35.172483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:35.172483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:19:35.172483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:35.173105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:35.173105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:35.173105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:19:35.173105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:35.173105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:19:35.173105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:19:35.173105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:35.173105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:19:35.173105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:35.173105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:19:35.173105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:19:35.173105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:35.173105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:19:35.173550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:35.173550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:19:35.173550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:35.173550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:19:35.173550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:19:35.173550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:19:35.173550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:19:35.173550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:19:35.173550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:35.173550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:19:35.173550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:19:35.173550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:19:35.173550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:19:35.173550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:19:35.173550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:35.173970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:35.173970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:19:35.173970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:19:35.173970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:19:35.173970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:19:35.173970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:35.173970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:35.173970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:35.173970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:19:35.173970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:35.173970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:35.173970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:35.173970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:19:35.173970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:35.173970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:19:35.173970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:19:35.173970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:35.173970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:19:35.173970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:35.174474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:19:35.174474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:19:35.174474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:35.174474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:19:35.174474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:35.174474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:19:39.863374 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:19:39.864088 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:19:39.864088 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:19:39.865301 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:19:39.897533 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:39.897759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:39.898278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:19:39.898278 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:39.898278 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:19:39.898278 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:19:39.898278 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:39.898278 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:19:39.898278 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:39.898278 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:19:39.898958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:39.898958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:19:39.898958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:39.898958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:19:39.898958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:19:39.898958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:19:39.898958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:19:39.898958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:39.898958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:39.898958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:39.898958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:19:39.898958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:39.899529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:39.899529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:39.899529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:19:39.899529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:39.899529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:19:39.899529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:19:39.899529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:39.899529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:19:39.899529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:39.899529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:19:39.899529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:19:39.899529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:39.899529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:19:39.899873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:39.899873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:19:39.899873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:39.899873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:19:39.899873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:19:39.899873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:19:39.899873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:19:39.899873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:19:39.899873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:39.899873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:19:39.899873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:19:39.899873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:19:39.899873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:19:39.899873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:19:39.899873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:39.900377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:39.900377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:19:39.900377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:19:39.900377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:19:39.900377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:19:39.900377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:39.900377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:39.900377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:39.900377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:19:39.900377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:39.900377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:39.900377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:39.900377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:19:39.900377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:39.900377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:19:39.900377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:19:39.900377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:39.900377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:19:39.900377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:39.900881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:19:39.900881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:19:39.900881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:39.900881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:19:39.900881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:39.900881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:19:40.110581 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:19:40.110982 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:19:40.111166 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:19:40.112106 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:40.112397 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:40.112668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:40.113509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:19:40.113509 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:40.113509 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:19:40.113509 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:19:40.113509 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:40.113509 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:19:40.113509 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:40.113509 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:19:40.114877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:40.114877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:19:40.114877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:40.114877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:19:40.114877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:19:40.114877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:19:40.114877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:19:40.114877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:40.114877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:40.114877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:40.114877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:19:40.114877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:40.115583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:40.115583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:40.115583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:19:40.115583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:40.115583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:19:40.115583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:19:40.115583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:40.115583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:19:40.115583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:40.115583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:19:40.115583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:19:40.115583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:40.115583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:19:40.116254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:40.116254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:19:40.116567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:40.116567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:19:40.116567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:19:40.116567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:19:40.116567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:19:40.116567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:19:40.116567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:40.116567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:19:40.116567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:19:40.116567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:19:40.116567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:19:40.116567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:19:40.116567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:40.116567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:40.116567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:19:40.116567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:19:40.117011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:19:40.117011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:19:40.117011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:40.117011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:40.117011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:40.117011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:19:40.117011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:40.117011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:40.117011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:40.117011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:19:40.117011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:40.117011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:19:40.117011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:19:40.117011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:40.117011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:19:40.117011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:40.117011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:19:40.117011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:19:40.117011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:40.117548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:19:40.117548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:40.117548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:19:40.117978 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:19:40.118275 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:19:40.118487 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:19:40.118679 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:19:40.118983 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:40.119164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:40.119677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:19:40.119677 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:40.119677 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:19:40.119677 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:19:40.119677 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:40.119677 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:19:40.119677 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:40.119677 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:19:40.120921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:40.120921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:19:40.120921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:19:40.120921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:19:40.120921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:40.120921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:19:40.120921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:40.120921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:19:40.120921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:40.120921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:19:40.120921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:40.120921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:19:40.120921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:40.120921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:19:40.120921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:19:40.121362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:40.121362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:19:40.121362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:40.121362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:19:40.121362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:19:40.121362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:40.121362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:19:40.121362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:40.121362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:19:40.122280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:40.122280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:19:40.122280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:40.122280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:19:40.122280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:19:40.122280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:19:40.122280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:19:40.122280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:19:40.122280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:19:40.122280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:19:40.122280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:40.122280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:40.122636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:40.122636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:19:40.122636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:40.122636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:40.122636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:40.122636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:19:40.122636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:40.122636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:19:40.122636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:19:40.122636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:40.122636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:19:40.122636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:40.122636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:19:40.122636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:19:40.122954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:40.122954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:19:40.122954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:40.122954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:19:40.124008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:40.124008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:19:40.124008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:19:40.124008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:19:40.124008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:19:40.124008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:19:40.124008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:40.124008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:19:40.124008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:19:40.124008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:19:40.124008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:19:40.124008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:19:40.124008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:40.124008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:40.124008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:19:40.124008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:19:40.124376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:19:40.124376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:19:40.124376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:19:40.124376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:19:40.124376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:19:40.124376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:40.124376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:40.124376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:40.124376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:19:40.124376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:40.124376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:40.124376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:40.124376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:19:40.124376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:40.124376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:19:40.124376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:19:40.124376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:40.124376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:19:40.124881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:40.124881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:19:40.124881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:19:40.124881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:40.124881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:19:40.124881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:40.124881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:19:40.161338 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:19:40.161687 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:19:40.161862 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:19:40.162058 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:19:40.162671 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:40.163076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:40.163882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:19:40.163882 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:40.163882 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:19:40.163882 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:19:40.163882 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:40.163882 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:19:40.163882 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:40.163882 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:19:40.165034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:40.165034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:19:40.165034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:19:40.165034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:19:40.165034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:40.165034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:19:40.165034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:40.165034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:19:40.165034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:40.165034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:19:40.165034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:40.165034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:19:40.165034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:40.165034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:19:40.165034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:19:40.165518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:40.165518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:19:40.165518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:40.165518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:19:40.165518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:19:40.165518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:40.165518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:19:40.165518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:40.165518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:19:40.166513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:40.166513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:19:40.166513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:40.166513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:19:40.166513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:19:40.166513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:19:40.166513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:19:40.166513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:19:40.166513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:19:40.166513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:40.166513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:40.166782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:40.166782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:19:40.166782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:40.166782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:40.166782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:40.166782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:19:40.166782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:40.166782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:19:40.166782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:19:40.166782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:40.166782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:19:40.166782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:40.166782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:19:40.166782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:19:40.167079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:40.167079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:19:40.167079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:40.167079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:19:40.168278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:40.168278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:19:40.168278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:19:40.168278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:19:40.168278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:19:40.168278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:19:40.168278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:40.168278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:19:40.168278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:19:40.168278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:19:40.168278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:19:40.168278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:19:40.168278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:40.168278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:40.168278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:19:40.168278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:19:40.168278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:19:40.168824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:19:40.168824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:19:40.168824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:19:40.168824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:40.168824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:40.168824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:40.168824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:19:40.168824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:40.168824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:40.168824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:40.168824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:19:40.168824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:40.168824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:19:40.168824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:19:40.168824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:40.168824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:19:40.168824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:40.168824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:19:40.168824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:19:40.169351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:40.169351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:19:40.169351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:40.169351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:19:40.169705 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:19:40.170018 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:19:40.170191 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:19:40.170420 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:19:40.170699 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:40.170913 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:40.171050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:40.171543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:19:40.171543 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:40.171543 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:19:40.171543 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:19:40.171543 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:40.171543 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:19:40.171543 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:40.171543 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:19:40.172292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:40.172292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:19:40.172292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:40.172292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:19:40.172292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:19:40.172292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:19:40.172292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:19:40.172292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:40.172292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:40.172292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:40.172292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:19:40.172292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:40.172749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:40.172749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:40.172749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:19:40.172749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:40.172749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:19:40.172749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:19:40.172749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:40.172749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:19:40.172749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:40.172749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:19:40.172749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:19:40.172749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:40.172749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:19:40.173097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:40.173097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:19:40.173248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:40.173248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:19:40.173248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:19:40.173248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:19:40.173248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:19:40.173248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:19:40.173248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:40.173248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:19:40.173248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:19:40.173248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:19:40.173248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:19:40.173248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:19:40.173248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:40.173248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:40.173248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:19:40.173248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:19:40.173691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:19:40.173691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:19:40.173691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:40.173691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:40.173691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:40.173691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:19:40.173691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:40.173691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:40.173691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:40.173691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:19:40.173691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:40.173691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:19:40.173691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:19:40.173691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:40.173691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:19:40.173691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:40.173691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:19:40.173691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:19:40.173691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:40.174185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:19:40.174185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:40.174185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:19:44.864109 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:19:44.865091 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:19:44.865091 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:19:44.865547 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:19:44.891515 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:44.891946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:44.892778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:19:44.892778 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:44.892778 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:19:44.892778 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:19:44.892778 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:44.892778 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:19:44.892778 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:44.892778 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:19:44.893548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.028 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:44.893548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:19:44.893548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:44.893548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:19:44.893548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:19:44.893548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:19:44.893548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:19:44.893548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:44.893548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:44.893548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:44.893548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:19:44.893548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:44.894304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:44.894304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:44.894304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:19:44.894304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:44.894304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:19:44.894304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:19:44.894304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:44.894304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:19:44.894304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:44.894304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:19:44.894304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:19:44.894304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:44.894304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:19:44.894956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:44.894956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:19:44.894956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:44.894956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:19:44.894956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:19:44.894956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:19:44.894956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:19:44.894956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:19:44.894956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:44.894956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:19:44.894956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:19:44.894956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:19:44.894956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:19:44.894956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:19:44.894956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:44.895699 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:44.895699 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:19:44.895699 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:19:44.895699 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:19:44.895699 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:19:44.895699 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:44.895699 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:44.895699 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:44.895699 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:19:44.895699 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:44.895699 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:44.895699 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:44.895699 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:19:44.895699 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:44.895699 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:19:44.895699 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:19:44.895699 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:44.895699 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:19:44.895699 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:44.896583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:19:44.896583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:19:44.896583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:44.896583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:19:44.896583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:44.896583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:19:45.111173 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:19:45.111779 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:19:45.111973 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:19:45.112939 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:45.113501 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:45.113759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:45.114724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:19:45.114724 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:45.114724 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:19:45.114724 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:19:45.114724 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:45.114724 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:19:45.114724 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:45.114724 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:19:45.116215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:45.116215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:19:45.116215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:45.116215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:19:45.116215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:19:45.116215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:19:45.116215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:19:45.116215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:45.116215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:45.116215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:45.116215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:19:45.116215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:45.116857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:45.116857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:45.116857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:19:45.116857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:45.116857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:19:45.116857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:19:45.116857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:45.116857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:19:45.116857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:45.116857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:19:45.116857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:19:45.116857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:45.116857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:19:45.117908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:45.117908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:19:45.117908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:45.117908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:19:45.117908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:19:45.117908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:19:45.117908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:19:45.117908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:19:45.117908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:45.117908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:19:45.117908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:19:45.117908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:19:45.117908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:19:45.117908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:19:45.117908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:45.118669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:45.118669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:19:45.118669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:19:45.118669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:19:45.118669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:19:45.118669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:45.118669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:45.118669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:45.118669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:19:45.118669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:45.118669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:45.118669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:45.118669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:19:45.118669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:45.118669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:19:45.118669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:19:45.118669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:45.118669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:19:45.118669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:45.120100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:19:45.120100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:19:45.120100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:45.120100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:19:45.120100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:45.120100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:19:45.120100 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:19:45.120100 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:19:45.120100 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:19:45.120100 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:19:45.121464 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:45.121464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:45.121464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:19:45.121464 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:45.121464 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:19:45.121464 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:19:45.121464 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:45.121464 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:19:45.121464 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:45.121464 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:19:45.121464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:45.121464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:19:45.121464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:19:45.121464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:19:45.122145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:45.122145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:19:45.122145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:45.122145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:19:45.122145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:45.122145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:19:45.122145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:45.122145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:19:45.122145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:45.122145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:19:45.122145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:19:45.122145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:45.122145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:19:45.122145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:45.122145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:19:45.122145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:19:45.122145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:45.122145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:19:45.122858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:45.122858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:19:45.122858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:45.122858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:19:45.122858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:45.122858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:19:45.122858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:19:45.122858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:19:45.122858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:19:45.122858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:19:45.122858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:19:45.122858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:19:45.123121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:45.123121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:45.123121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:45.123121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:19:45.123121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:45.123121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:45.123121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:45.123121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:19:45.123121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:45.123121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:19:45.123121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:19:45.123121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:45.123121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:19:45.123121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:45.123384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:19:45.123384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:19:45.123384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:45.123384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:19:45.123384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:45.123384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:19:45.123384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:45.123384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:19:45.123384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:19:45.123384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:19:45.123384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:19:45.123384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:19:45.123384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:45.123384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:19:45.123682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:19:45.123682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:19:45.123682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:19:45.123682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:19:45.123682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:45.123682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:45.123682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:19:45.123682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:19:45.123682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:19:45.123682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:19:45.123682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:19:45.123682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:19:45.123682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:19:45.123682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:45.123682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:45.123682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:45.123682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:19:45.123682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:45.123682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:45.124029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:45.124029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:19:45.124029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:45.124029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:19:45.124029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:19:45.124029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:45.124029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:19:45.124029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:45.124029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:19:45.124029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:19:45.124029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:45.124029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:19:45.124029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:45.124029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:19:45.162702 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:19:45.162702 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:19:45.162702 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:19:45.162702 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:19:45.163264 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:45.163756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:45.164610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:19:45.164610 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:45.164610 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:19:45.164610 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:19:45.164610 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:45.164610 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:19:45.164610 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:45.164610 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:19:45.165777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:45.165777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:19:45.165777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:19:45.165777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:19:45.165777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:45.165777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:19:45.165777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:45.165777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:19:45.165777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:45.165777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:19:45.165777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:45.165777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:19:45.165777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:45.165777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:19:45.165777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:19:45.166109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:45.166109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:19:45.166109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:45.166109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:19:45.166109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:19:45.166109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:45.166109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:19:45.166109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:45.166109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:19:45.167167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:45.167167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:19:45.167167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:45.167167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:19:45.167167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:19:45.167167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:19:45.167167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:19:45.167167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:19:45.167167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:19:45.167167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:45.167167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:45.168110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:45.168110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:19:45.168110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:45.168110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:45.168110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:45.168110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:19:45.168110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:45.168110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:19:45.168110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:19:45.168110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:45.168110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:19:45.168110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:45.168110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:19:45.168110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:19:45.168357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:45.168357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:19:45.168357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:45.168357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:19:45.168803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:45.168803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:19:45.168803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:19:45.168803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:19:45.168803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:19:45.168803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:19:45.168803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:45.168803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:19:45.168803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:19:45.168803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:19:45.168803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:19:45.168803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:19:45.168803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:45.168803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:45.168803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:19:45.168803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:19:45.168803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:19:45.169509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:19:45.169509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:19:45.169509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:19:45.169509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:45.169509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:45.169509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:45.169509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:19:45.169509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:45.169509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:45.169509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:45.169509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:19:45.169509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:45.169509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:19:45.169509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:19:45.169509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:45.169509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:19:45.169509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:45.169509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:19:45.169509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:19:45.169865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:45.169865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:19:45.169865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:45.169865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:19:45.170207 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:19:45.170397 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:19:45.170621 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:19:45.170934 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:19:45.171065 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:45.171291 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:45.171481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:45.171958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:19:45.171958 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:45.171958 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:19:45.171958 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:19:45.171958 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:45.171958 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:19:45.171958 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:45.171958 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:19:45.172805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:45.172805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:19:45.172805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:45.172805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:19:45.172805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:19:45.172805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:19:45.172805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:19:45.172805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:45.172805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:45.172805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:45.172805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:19:45.172805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:45.173294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:45.173294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:45.173294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:19:45.173294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:45.173294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:19:45.173294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:19:45.173294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:45.173294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:19:45.173294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:45.173294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:19:45.173294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:19:45.173294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:45.173294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:19:45.173937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:45.173937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:19:45.173937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:45.173937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:19:45.173937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:19:45.173937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:19:45.173937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:19:45.173937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:19:45.173937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:45.173937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:19:45.173937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:19:45.173937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:19:45.173937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:19:45.173937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:19:45.173937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:45.174365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:45.174365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:19:45.174365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:19:45.174365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:19:45.174365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:19:45.174365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:45.174365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:45.174365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:45.174365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:19:45.174365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:45.174365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:45.174365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:45.174365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:19:45.174365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:45.174365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:19:45.174365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:19:45.174365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:45.174365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:19:45.174365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:45.175060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:19:45.175060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:19:45.175060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:45.175060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:19:45.175060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:45.175060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:19:49.865033 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:19:49.865676 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:19:49.865676 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:19:49.866075 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:19:49.901546 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:49.901953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:49.902373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:19:49.902373 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:49.902373 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:19:49.902373 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:19:49.902373 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:49.902373 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:19:49.902373 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:49.902373 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:19:49.903087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:49.903087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:19:49.903087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:49.903087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:19:49.903087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:19:49.903087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:19:49.903087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:19:49.903087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:49.903087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:49.903087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:49.903087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:19:49.903087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:49.903507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:49.903507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:49.903507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:19:49.903507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:49.903507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:19:49.903507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:19:49.903507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:49.903507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:19:49.903507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:49.903507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:19:49.903507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:19:49.903507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:49.903507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:19:49.903897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:49.903897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:19:49.903897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:49.903897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:19:49.903897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:19:49.903897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:19:49.903897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:19:49.903897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:19:49.903897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:49.903897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:19:49.903897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:19:49.903897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:19:49.903897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:19:49.903897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:19:49.903897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:49.904356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:49.904356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:19:49.904356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:19:49.904356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:19:49.904356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:19:49.904356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:49.904356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:49.904356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:49.904356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:19:49.904356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:49.904356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:49.904356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:49.904356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:19:49.904356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:49.904356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:19:49.904356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:19:49.904356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:49.904356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:19:49.904356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:49.904933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:19:49.904933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:19:49.904933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:49.904933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:19:49.904933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:49.904933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:19:50.111875 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:19:50.112582 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:19:50.112770 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:19:50.113806 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:50.114218 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:50.114596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:50.115754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:19:50.115754 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:50.115754 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:19:50.115754 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:19:50.115754 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:50.115754 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:19:50.115754 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:50.115754 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:19:50.117359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:50.117359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:19:50.117359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:50.117359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:19:50.117359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:19:50.117359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:19:50.117359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:19:50.117359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:50.117359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:50.117359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:50.117359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:19:50.117359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:50.117987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:50.117987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:50.117987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:19:50.117987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:50.117987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:19:50.117987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:19:50.117987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:50.117987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:19:50.117987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:50.117987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:19:50.117987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:19:50.117987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:50.117987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:19:50.118552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:50.118552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:19:50.119951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:50.119951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:19:50.119951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:19:50.119951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:19:50.119951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:19:50.119951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:19:50.119951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:50.119951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:19:50.119951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:19:50.119951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:19:50.119951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:19:50.119951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:19:50.119951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:50.119951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:50.119951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:19:50.119951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:19:50.120581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:19:50.120581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:19:50.120581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:50.120581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:50.120581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:50.120581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:19:50.120581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:50.120581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:50.120581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:50.120581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:19:50.120581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:50.120581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:19:50.120581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:19:50.120581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:50.120581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:19:50.120581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:50.120581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:19:50.120581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:19:50.120581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:50.121207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:19:50.121207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:50.121207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:19:50.121207 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:19:50.121207 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:19:50.121207 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:19:50.121207 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:19:50.121597 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:50.121597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:50.122125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:19:50.122125 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:50.122125 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:19:50.122125 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:19:50.122125 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:50.122125 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:19:50.122125 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:50.122125 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:19:50.123074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:50.123074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:19:50.123074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:19:50.123074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:19:50.123074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:50.123074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:19:50.123074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:50.123074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:19:50.123074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:50.123074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:19:50.123074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:50.123074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:19:50.123074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:50.123074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:19:50.123074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:19:50.125160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:50.125160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:19:50.125160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:50.125160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:19:50.125160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:19:50.125160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:50.125160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:19:50.125160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:50.125160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:19:50.125160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:50.125160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:19:50.125160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:50.125160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:19:50.125160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:19:50.125806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:19:50.125806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:19:50.125806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:19:50.125806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:19:50.125806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:19:50.125806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:50.125806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:50.125806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:50.125806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:19:50.125806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:50.125806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:50.125806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:50.125806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:19:50.126234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:50.126234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:19:50.126234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:19:50.126234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:50.126234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:19:50.126234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:50.126234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:19:50.126234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:19:50.126234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:50.126234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:19:50.126234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:50.126234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:19:50.126603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:50.126603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:19:50.126603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:19:50.126603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:19:50.126603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:19:50.126603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:19:50.126603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:50.126603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:19:50.126603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:19:50.126603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:19:50.126603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:19:50.126603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:19:50.126603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:50.126603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:50.126603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:19:50.126603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:19:50.127111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:19:50.127111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:19:50.127111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:19:50.127111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:19:50.127111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:19:50.127111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:50.127111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:50.127111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:50.127111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:19:50.127111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:50.127111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:50.127111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:50.127111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:19:50.127111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:50.127111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:19:50.127111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:19:50.127111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:50.127111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:19:50.127590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:50.127590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:19:50.127590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:19:50.127590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:50.127590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:19:50.127590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:50.127590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:19:50.162429 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:19:50.162857 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:19:50.163022 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:19:50.163814 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:19:50.163814 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:50.164314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:50.164658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:19:50.164658 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:50.164658 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:19:50.164658 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:19:50.164658 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:50.164658 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:19:50.164658 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:50.164658 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:19:50.166044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:50.166044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:19:50.166044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:19:50.166044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:19:50.166044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:50.166044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:19:50.166044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:50.166044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:19:50.166044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:50.166044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:19:50.166044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:50.166044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:19:50.166044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:50.166044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:19:50.166044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:19:50.167155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:50.167155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:19:50.167155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:50.167155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:19:50.167155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:19:50.167155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:50.167155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:19:50.167155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:50.167155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:19:50.168040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:50.168040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:19:50.168040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:50.168040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:19:50.168040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:19:50.168040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:19:50.168040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:19:50.168040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:19:50.168040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:19:50.168040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:50.168040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:50.168583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:50.168583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:19:50.168583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:50.168583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:50.168583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:50.168583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:19:50.168583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:50.168583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:19:50.168583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:19:50.168583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:50.168583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:19:50.168583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:50.168583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:19:50.168583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:19:50.168977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:50.168977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:19:50.168977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:50.168977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:19:50.169151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:50.169151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:19:50.169151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:19:50.169151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:19:50.169151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:19:50.169151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:19:50.169151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:50.169151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:19:50.169151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:19:50.169151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:19:50.169151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:19:50.169151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:19:50.169151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:50.169151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:50.169151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:19:50.169151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:19:50.169151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:19:50.169783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:19:50.169783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:19:50.169783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:19:50.169783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:50.169783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:50.169783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:50.169783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:19:50.169783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:50.169783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:50.169783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:50.169783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:19:50.169783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:50.169783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:19:50.169783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:19:50.169783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:50.169783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:19:50.169783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:50.169783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:19:50.169783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:19:50.170502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:50.170502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:19:50.170502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:50.170502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:19:50.170628 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:19:50.170886 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:19:50.171041 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:19:50.171193 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:19:50.171668 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:50.172069 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:50.172317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:50.173511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:19:50.173511 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:50.173511 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:19:50.173511 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:19:50.173511 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:50.173511 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:19:50.173511 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:50.173511 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:19:50.174203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:50.174203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:19:50.174203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:50.174203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:19:50.174203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:19:50.174203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:19:50.174203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:19:50.174203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:50.174203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:50.174203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:50.174203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:19:50.174203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:50.174950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:50.174950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:50.174950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:19:50.174950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:50.174950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:19:50.174950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:19:50.174950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:50.174950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:19:50.174950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:50.174950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:19:50.174950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:19:50.174950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:50.174950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:19:50.175732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:50.175732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:19:50.175732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:50.175732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:19:50.175732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:19:50.175732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:19:50.175732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:19:50.175732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:19:50.175732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:50.175732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:19:50.175732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:19:50.175732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:19:50.175732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:19:50.175732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:19:50.175732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:50.176502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:50.176502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:19:50.176502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:19:50.176502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:19:50.176502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:19:50.176502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:50.176502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:50.176502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:50.176502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:19:50.176502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:50.176502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:50.176502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:50.176502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:19:50.176502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:50.176502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:19:50.176502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:19:50.176502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:50.176502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:19:50.176502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:50.177394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:19:50.177394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:19:50.177394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:50.177394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:19:50.177394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:50.177394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:19:54.865709 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:19:54.866711 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:19:54.866711 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:19:54.867241 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:19:54.902338 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:54.902661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:54.903318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:19:54.903318 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:54.903318 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:19:54.903318 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:19:54.903318 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:54.903318 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:19:54.903318 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:54.903318 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:19:54.904050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:54.904050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:19:54.904050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:54.904050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:19:54.904050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:19:54.904050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:19:54.904050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:19:54.904050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:54.904050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:54.904050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:54.904050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:19:54.904050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:54.904643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:54.904643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:54.904643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:19:54.904643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:54.904643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:19:54.904643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:19:54.904643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:54.904643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:19:54.904643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:54.904643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:19:54.904643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:19:54.904643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:54.904643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:19:54.905258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:54.905258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:19:54.905258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:54.905258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:19:54.905258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:19:54.905258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:19:54.905258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:19:54.905258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:19:54.905258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:54.905258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:19:54.905258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:19:54.905258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:19:54.905258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:19:54.905258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:19:54.905258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:54.905904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:54.905904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:19:54.905904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:19:54.905904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:19:54.905904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:19:54.905904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:54.905904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:54.905904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:54.905904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:19:54.905904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:54.905904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:54.905904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:54.905904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:19:54.905904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:54.905904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:19:54.905904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:19:54.905904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:54.905904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:19:54.905904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:54.906702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:19:54.906702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:19:54.906702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:54.906702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:19:54.906702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:54.906702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:19:55.112612 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:19:55.113237 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:19:55.113453 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:19:55.114645 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:55.115066 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:55.115332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:55.116297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:19:55.116297 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:55.116297 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:19:55.116297 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:19:55.116297 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:55.116297 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:19:55.116297 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:55.116297 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:19:55.117871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:55.117871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:19:55.117871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:55.117871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:19:55.117871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:19:55.117871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:19:55.117871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:19:55.117871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:55.117871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:55.117871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:55.117871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:19:55.117871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:55.118258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:55.118258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:55.118258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:19:55.118258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:55.118258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:19:55.118258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:19:55.118258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:55.118258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:19:55.118258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:55.118258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:19:55.118258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:19:55.118258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:55.118258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:19:55.118701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:55.118701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:19:55.119606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:55.119606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:19:55.119606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:19:55.119606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:19:55.119606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:19:55.119606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:19:55.119606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:55.119606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:19:55.119606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:19:55.119606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:19:55.119606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:19:55.119606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:19:55.119606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:55.119606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:55.119606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:19:55.119606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:19:55.120116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:19:55.120116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:19:55.120116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:55.120116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:55.120116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:55.120116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:19:55.120116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:55.120116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:55.120116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:55.120116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:19:55.120116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:55.120116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:19:55.120116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:19:55.120116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:55.120116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:19:55.120116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:55.120116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:19:55.120116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:19:55.120116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:55.120714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:19:55.120714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:55.120714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:19:55.120811 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:19:55.121108 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:19:55.121293 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:19:55.121512 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:19:55.121999 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:55.122505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:55.123136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:19:55.123136 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:55.123136 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:19:55.123136 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:19:55.123136 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:55.123136 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:19:55.123136 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:55.123136 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:19:55.123886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:55.123886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:19:55.123886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:19:55.123886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:19:55.123886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:55.123886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:19:55.123886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:55.123886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:19:55.123886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:55.123886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:19:55.123886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:55.123886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:19:55.123886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:55.123886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:19:55.123886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:19:55.124420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:55.124420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:19:55.124420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:55.124420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:19:55.124420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:19:55.124420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:55.124420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:19:55.124420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:55.124420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:19:55.124693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:55.124693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:19:55.124693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:55.124693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:19:55.124693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:19:55.124693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:19:55.124693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:19:55.124693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:19:55.124693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:19:55.124693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:19:55.124693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:55.124693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:55.125154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:55.125154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:19:55.125154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:55.125154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:55.125154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:55.125154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:19:55.125154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:55.125154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:19:55.125154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:19:55.125154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:55.125154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:19:55.125154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:55.125154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:19:55.125154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:19:55.125538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:55.125538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:19:55.125538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:55.125538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:19:55.125695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:55.125695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:19:55.125695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:19:55.125695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:19:55.125695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:19:55.125695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:19:55.125695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:55.125695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:19:55.125695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:19:55.125695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:19:55.125695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:19:55.125695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:19:55.125695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:55.125695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:55.125695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:19:55.125695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:19:55.126124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:19:55.126124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:19:55.126124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:19:55.126124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:19:55.126124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:19:55.126124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:55.126124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:55.126124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:55.126124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:19:55.126124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:55.126124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:55.126124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:55.126124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:19:55.126124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:55.126124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:19:55.126124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:19:55.126124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:55.126124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:19:55.126654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:55.126654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:19:55.126654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:19:55.126654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:55.126654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:19:55.126654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:55.126654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:19:55.162930 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:19:55.163377 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:19:55.163540 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:19:55.163761 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:19:55.164291 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:55.164677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:55.165437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:19:55.165437 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:55.165437 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:19:55.165437 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:19:55.165437 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:55.165437 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:19:55.165437 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:55.165437 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:19:55.166675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:55.166675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:19:55.166675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:19:55.166675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:19:55.166675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:55.166675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:19:55.166675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:55.166675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:19:55.166675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:55.166675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:19:55.166675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:55.166675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:19:55.166675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:55.166675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:19:55.166675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:19:55.167553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:55.167553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:19:55.167553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:55.167553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:19:55.167553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:19:55.167553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:55.167553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:19:55.167553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:55.167553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:19:55.168068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:55.168068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:19:55.168068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:55.168068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:19:55.168068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:19:55.168068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:19:55.168068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:19:55.168068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:19:55.168068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:19:55.168068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:55.168068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:55.169010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:55.169010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:19:55.169010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:55.169010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:55.169010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:55.169010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:19:55.169010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:55.169010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:19:55.169010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:19:55.169010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:55.169010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:19:55.169010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:55.169010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:19:55.169010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:19:55.169696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:55.169696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:19:55.169696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:55.169696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:19:55.169961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:55.169961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:19:55.169961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:19:55.169961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:19:55.169961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:19:55.169961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:19:55.169961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:55.169961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:19:55.169961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:19:55.169961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:19:55.169961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:19:55.169961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:19:55.169961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:55.169961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:55.169961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:19:55.169961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:19:55.169961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:19:55.170855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:19:55.170855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:19:55.170855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:19:55.170855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:55.170855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:55.170855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:55.170855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:19:55.170855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:55.170855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:55.170855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:55.170855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:19:55.170855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:55.170855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:19:55.170855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:19:55.170855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:55.170855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:19:55.170855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:55.170855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:19:55.170855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:19:55.171678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:55.171678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:19:55.171678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:55.171678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:19:55.171678 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:19:55.171678 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:19:55.171678 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:19:55.171678 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:19:55.171985 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:55.172321 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:55.172601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:55.173264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:19:55.173264 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:55.173264 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:19:55.173264 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:19:55.173264 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:55.173264 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:19:55.173264 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:55.173264 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:19:55.174150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:55.174150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:19:55.174150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:55.174150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:19:55.174150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:19:55.174150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:19:55.174150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:19:55.174150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:55.174150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:55.174150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:55.174150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:19:55.174150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:55.174551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:55.174551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:55.174551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:19:55.174551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:55.174551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:19:55.174551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:19:55.174551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:55.174551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:19:55.174551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:55.174551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:19:55.174551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:19:55.174551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:55.174551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:19:55.174927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:55.174927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:19:55.175113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:55.175113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:19:55.175113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:19:55.175113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:19:55.175113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:19:55.175113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:19:55.175113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:55.175113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:19:55.175113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:19:55.175113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:19:55.175113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:19:55.175113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:19:55.175113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:55.175113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:55.175113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:19:55.175113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:19:55.175572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:19:55.175572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:19:55.175572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:55.175572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:55.175572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:55.175572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:19:55.175572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:55.175572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:55.175572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:55.175572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:19:55.175572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:55.175572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:19:55.175572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:19:55.175572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:55.175572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:19:55.175572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:55.175572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:19:55.175572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:19:55.175572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:55.176088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:19:55.176088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:55.176088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:19:59.866472 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:19:59.867327 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:19:59.867327 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:19:59.867519 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:19:59.904453 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:19:59.904577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:59.905106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:19:59.905106 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:59.905106 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:19:59.905106 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:19:59.905106 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:59.905106 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:19:59.905106 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:59.905106 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:19:59.906072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:59.906072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:19:59.906072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:59.906072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:19:59.906072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:19:59.906072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:19:59.906072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:19:59.906072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:59.906072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:59.906072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:59.906072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:19:59.906072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:59.906854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:19:59.906854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:59.906854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:19:59.906854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:59.906854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:19:59.906854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:19:59.906854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:59.906854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:19:59.906854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:59.906854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:19:59.906854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:19:59.906854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:59.906854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:19:59.907442 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:59.907442 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:19:59.907561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:59.907561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:19:59.907561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:19:59.907561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:19:59.907561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:19:59.907561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:19:59.907561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:59.907561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:19:59.907561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:19:59.907561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:19:59.907561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:19:59.907561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:19:59.907561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:19:59.907561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:19:59.907561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:19:59.907561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:19:59.908250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:19:59.908250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:19:59.908250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:59.908250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:59.908250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:19:59.908250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:19:59.908250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:19:59.908250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:19:59.908250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:19:59.908250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:19:59.908250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:19:59.908250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:19:59.908250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:19:59.908250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:19:59.908250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:19:59.908250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:19:59.908250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:19:59.908250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:19:59.908250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:19:59.909055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:19:59.909055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:19:59.909055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:20:00.113288 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:20:00.113786 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:20:00.113897 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:20:00.114661 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:20:00.114892 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:20:00.115100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:00.115960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:20:00.115960 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:00.115960 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:20:00.115960 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:20:00.115960 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:00.115960 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:20:00.115960 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:00.115960 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:20:00.117289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:00.117289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:20:00.117289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:00.117289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:20:00.117289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:20:00.117289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:20:00.117289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:20:00.117289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:00.117289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:00.117289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:00.117289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:20:00.117289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:00.117930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:00.117930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:00.117930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:20:00.117930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:00.117930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:20:00.117930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:20:00.117930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:00.117930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:20:00.117930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:00.117930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:20:00.117930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:20:00.117930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:00.117930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:20:00.118527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:00.118527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:20:00.119064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:00.119064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:20:00.119064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:20:00.119064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:20:00.119064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:20:00.119064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:20:00.119064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:00.119064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:20:00.119064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:20:00.119064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:20:00.119064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:20:00.119064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:20:00.119064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:00.119064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:00.119064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:20:00.119064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:20:00.119806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:20:00.119806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:20:00.119806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:00.119806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:00.119806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:00.119806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:20:00.119806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:00.119806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:00.119806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:00.119806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:20:00.119806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:00.119806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:20:00.119806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:20:00.119806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:00.119806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:20:00.119806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:00.119806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:20:00.119806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:20:00.119806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:00.120650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:20:00.120650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:00.120650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:20:00.121141 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:20:00.121534 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:20:00.121707 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:20:00.121888 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:20:00.122238 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:20:00.122531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:00.123322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:20:00.123322 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:00.123322 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:20:00.123322 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:20:00.123322 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:00.123322 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:20:00.123322 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:00.123322 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:20:00.124029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:00.124029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:20:00.124029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:20:00.124029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:20:00.124029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:00.124029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:20:00.124029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:00.124029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:20:00.124029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:00.124029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:20:00.124029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:00.124029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:20:00.124029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:00.124029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:20:00.124029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:20:00.124476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:00.124476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:20:00.124476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:00.124476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:20:00.124476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:20:00.124476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:00.124476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:20:00.124476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:00.124476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:20:00.124877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:00.124877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:20:00.124877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:00.124877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:20:00.124877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:20:00.124877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:20:00.124877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:20:00.124877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:20:00.124877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:20:00.124877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:20:00.124877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:00.124877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:00.125225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:00.125225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:20:00.125225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:00.125225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:00.125225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:00.125225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:20:00.125225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:00.125225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:20:00.125225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:20:00.125225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:00.125225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:20:00.125225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:00.125225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:20:00.125225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:20:00.125607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:00.125607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:20:00.125607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:00.125607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:20:00.125897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:00.125897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:20:00.125897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:20:00.125897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:20:00.125897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:20:00.125897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:20:00.125897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:00.125897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:20:00.125897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:20:00.125897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:20:00.125897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:20:00.125897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:20:00.125897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:00.125897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:00.125897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:20:00.125897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:20:00.126340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:20:00.126340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:20:00.126340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:20:00.126340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:20:00.126340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:20:00.126340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:00.126340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:00.126340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:00.126340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:20:00.126340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:00.126340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:00.126340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:00.126340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:20:00.126340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:00.126340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:20:00.126340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:20:00.126340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:00.126340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:20:00.126844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:00.126844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:20:00.126844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:20:00.126844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:00.126844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:20:00.126844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:00.126844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:20:00.163519 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:20:00.163874 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:20:00.164083 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:20:00.164257 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:20:00.164685 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:20:00.165071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:00.165836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:20:00.165836 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:00.165836 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:20:00.165836 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:20:00.165836 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:00.165836 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:20:00.165836 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:00.165836 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:20:00.167095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:00.167095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:20:00.167095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:20:00.167095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:20:00.167095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:00.167095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:20:00.167095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:00.167095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:20:00.167095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:00.167095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:20:00.167095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:00.167095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:20:00.167095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:00.167095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:20:00.167095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:20:00.167899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:00.167899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:20:00.167899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:00.167899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:20:00.167899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:20:00.167899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:00.167899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:20:00.167899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:00.167899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:20:00.168349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:00.168349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:20:00.168349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:00.168349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:20:00.168349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:20:00.168349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:20:00.168349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:20:00.168349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:20:00.168349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:20:00.168349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:00.168349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:00.168988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:00.168988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:20:00.168988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:00.168988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:00.168988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:00.168988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:20:00.168988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:00.168988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:20:00.168988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:20:00.168988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:00.168988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:20:00.168988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:00.168988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:20:00.168988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:20:00.169905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:00.169905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:20:00.169905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:00.169905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:20:00.169905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:00.169905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:20:00.169905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:20:00.169905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:20:00.169905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:20:00.169905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:20:00.169905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:00.169905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:20:00.169905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:20:00.169905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:20:00.171004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:20:00.171004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:20:00.171004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:00.171004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:00.171004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:20:00.171004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:20:00.171004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:20:00.171004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:20:00.171004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:20:00.171004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:20:00.171004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:00.171004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:00.171004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:00.171004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:20:00.171004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:00.171004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:00.171004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:00.171004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:20:00.171004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:00.172059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:20:00.172059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:20:00.172059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:00.172059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:20:00.172059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:00.172059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:20:00.172059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:20:00.172059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:00.172059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:20:00.172059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:00.172059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:20:00.172059 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:20:00.172059 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:20:00.173074 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:20:00.173074 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:20:00.173074 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:20:00.173074 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:20:00.173074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:00.173074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:20:00.173074 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:00.173074 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:20:00.173074 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:20:00.173587 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:00.173587 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:20:00.173587 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:00.173587 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:20:00.173587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:00.173587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:20:00.173587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:00.173587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:20:00.173587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:20:00.173587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:20:00.173587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:20:00.173587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:00.173587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:00.173587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:00.174268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:20:00.174268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:00.174268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:00.174268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:00.174268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:20:00.174268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:00.174268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:20:00.174268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:20:00.174268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:00.174268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:20:00.174268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:00.174268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:20:00.174268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:20:00.174268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:00.174893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:20:00.174893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:00.174893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:20:00.174893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:00.174893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:20:00.174893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:20:00.174893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:20:00.174893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:20:00.174893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:20:00.174893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:00.174893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:20:00.174893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:20:00.174893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:20:00.174893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:20:00.174893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:20:00.174893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:00.175520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:00.175520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:20:00.175520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:20:00.175520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:20:00.175520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:20:00.175520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:00.175520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:00.175520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:00.175520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:20:00.175520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:00.175520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:00.175520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:00.175520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:20:00.175520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:00.175520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:20:00.175520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:20:00.175520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:00.175520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:20:00.175520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:00.175975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:20:00.175975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:20:00.175975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:00.175975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:20:00.175975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:00.175975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:20:04.867126 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:20:04.867690 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:20:04.867733 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:20:04.868375 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:20:04.900758 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:20:04.900902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:04.901350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:20:04.901350 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:04.901350 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:20:04.901350 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:20:04.901350 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:04.901350 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:20:04.901350 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:04.901350 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:20:04.902017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:04.902017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:20:04.902017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:04.902017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:20:04.902017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:20:04.902017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:20:04.902017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:20:04.902017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:04.902017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:04.902017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:04.902017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:20:04.902017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:04.902613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:04.902613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:04.902613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:20:04.902613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:04.902613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:20:04.902613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:20:04.902613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:04.902613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:20:04.902613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:04.902613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:20:04.902613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:20:04.902613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:04.902613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:20:04.903272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:04.903272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:20:04.903272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:04.903272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:20:04.903272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:20:04.903272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:20:04.903272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:20:04.903272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:20:04.903272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:04.903272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:20:04.903272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:20:04.903272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:20:04.903272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:20:04.903272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:20:04.903272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:04.904024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:04.904024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:20:04.904024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:20:04.904024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:20:04.904024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:20:04.904024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:04.904024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:04.904024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:04.904024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:20:04.904024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:04.904024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:04.904024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:04.904024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:20:04.904024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:04.904024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:20:04.904024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:20:04.904024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:04.904024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:20:04.904024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:04.904968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:20:04.904968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:20:04.904968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:04.904968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:20:04.904968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:04.904968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:20:05.114022 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:20:05.114689 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:20:05.115240 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:20:05.116005 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:20:05.116335 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:20:05.116617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:05.117668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:20:05.117668 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:05.117668 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:20:05.117668 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:20:05.117668 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:05.117668 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:20:05.117668 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:05.117668 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:20:05.119234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:05.119234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:20:05.119234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:05.119234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:20:05.119234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:20:05.119234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:20:05.119234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:20:05.119234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:05.119234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:05.119234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:05.119234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:20:05.119234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:05.119808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:05.119808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:05.119808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:20:05.119808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:05.119808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:20:05.119808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:20:05.119808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:05.119808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:20:05.119808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:05.119808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:20:05.119808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:20:05.119808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:05.119808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:20:05.120359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:05.120359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:20:05.121645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:05.121645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:20:05.121645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:20:05.121645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:20:05.121645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:20:05.121645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:20:05.121645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:05.121645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:20:05.121645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:20:05.121645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:20:05.121645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:20:05.121645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:20:05.121645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:05.121645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:05.121645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:20:05.121645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:20:05.122482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:20:05.122482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:20:05.122482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:05.122482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:05.122482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:05.122482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:20:05.122482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:05.122482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:05.122482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:05.122482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:20:05.122482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:05.122482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:20:05.122482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:20:05.122482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:05.122482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:20:05.122482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:05.122482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:20:05.122482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:20:05.122482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:05.123332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:20:05.123332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:05.123332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:20:05.123332 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:20:05.123332 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:20:05.123332 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:20:05.123332 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:20:05.123332 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:20:05.123874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:05.124045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:20:05.124045 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:05.124045 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:20:05.124045 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:20:05.124045 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:05.124045 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:20:05.124045 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:05.124045 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:20:05.124985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:05.124985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:20:05.124985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:20:05.124985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:20:05.124985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:05.124985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:20:05.124985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:05.124985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:20:05.124985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:05.124985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:20:05.124985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:05.124985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:20:05.124985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:05.124985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:20:05.124985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:20:05.125499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:05.125499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:20:05.125499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:05.125499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:20:05.125499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:20:05.125499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:05.125499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:20:05.125499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:05.125499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:20:05.126129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:05.126129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:20:05.126129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:05.126129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:20:05.126129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:20:05.126129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:20:05.126129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:20:05.126129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:20:05.126129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:20:05.126129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:20:05.126129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:05.126129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:05.126942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:05.126942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:20:05.126942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:05.126942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:05.126942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:05.126942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:20:05.126942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:05.126942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:20:05.126942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:20:05.126942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:05.126942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:20:05.126942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:05.126942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:20:05.126942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:20:05.127505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:05.127505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:20:05.127505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:05.127505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:20:05.128173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:05.128173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:20:05.128173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:20:05.128173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:20:05.128173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:20:05.128173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:20:05.128173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:05.128173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:20:05.128173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:20:05.128173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:20:05.128173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:20:05.128173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:20:05.128173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:05.128173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:05.128173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:20:05.128173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:20:05.129047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:20:05.129047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:20:05.129047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:20:05.129047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:20:05.129047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:20:05.129047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:05.129047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:05.129047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:05.129047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:20:05.129047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:05.129047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:05.129047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:05.129047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:20:05.129047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:05.129047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:20:05.129047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:20:05.129047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:05.129047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:20:05.129574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:05.129574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:20:05.129574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:20:05.129574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:05.129574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:20:05.129574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:05.129574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:20:05.163986 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:20:05.164503 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:20:05.164652 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:20:05.164827 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:20:05.165247 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:20:05.165620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:05.166757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:20:05.166757 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:05.166757 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:20:05.166757 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:20:05.166757 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:05.166757 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:20:05.166757 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:05.166757 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:20:05.167923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:05.167923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:20:05.167923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:20:05.167923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:20:05.167923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:05.167923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:20:05.167923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:05.167923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:20:05.167923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:05.167923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:20:05.167923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:05.167923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:20:05.167923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:05.167923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:20:05.167923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:20:05.168334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:05.168334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:20:05.168334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:05.168334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:20:05.168334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:20:05.168334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:05.168334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:20:05.168334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:05.168334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:20:05.168845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:05.168845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:20:05.168845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:05.168845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:20:05.168845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:20:05.168845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:20:05.168845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:20:05.168845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:20:05.168845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:20:05.168845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:05.168845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:05.169174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:05.169174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:20:05.169174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:05.169174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:05.169174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:05.169174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:20:05.169174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:05.169174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:20:05.169174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:20:05.169174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:05.169174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:20:05.169174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:05.169174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:20:05.169174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:20:05.169547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:05.169547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:20:05.169547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:05.169547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:20:05.169661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:05.169661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:20:05.169661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:20:05.169661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:20:05.169661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:20:05.169661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:20:05.169661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:05.169661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:20:05.169661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:20:05.169661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:20:05.169661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:20:05.169661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:20:05.169661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:05.169661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:05.169661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:20:05.169661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:20:05.169661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:20:05.170135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:20:05.170135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:20:05.170135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:20:05.170135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:05.170135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:05.170135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:05.170135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:20:05.170135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:05.170135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:05.170135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:05.170135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:20:05.170135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:05.170135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:20:05.170135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:20:05.170135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:05.170135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:20:05.170135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:05.170135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:20:05.170135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:20:05.170684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:05.170684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:20:05.170684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:05.170684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:20:05.171722 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:20:05.171944 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:20:05.172067 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:20:05.172248 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:20:05.172574 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:20:05.172787 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:20:05.172930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:05.173419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:20:05.173419 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:05.173419 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:20:05.173419 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:20:05.173419 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:05.173419 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:20:05.173419 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:05.173419 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:20:05.174263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:05.174263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:20:05.174263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:05.174263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:20:05.174263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:20:05.174263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:20:05.174263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:20:05.174263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:05.174263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:05.174263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:05.174263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:20:05.174263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:05.174616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:05.174616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:05.174616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:20:05.174616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:05.174616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:20:05.174616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:20:05.174616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:05.174616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:20:05.174616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:05.174616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:20:05.174616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:20:05.174616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:05.174616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:20:05.174961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:05.174961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:20:05.175212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:05.175212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:20:05.175212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:20:05.175212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:20:05.175212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:20:05.175212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:20:05.175212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:05.175212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:20:05.175212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:20:05.175212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:20:05.175212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:20:05.175212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:20:05.175212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:05.175212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:05.175212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:20:05.175212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:20:05.175663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:20:05.175663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:20:05.175663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:05.175663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:05.175663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:05.175663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:20:05.175663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:05.175663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:05.175663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:05.175663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:20:05.175663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:05.175663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:20:05.175663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:20:05.175663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:05.175663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:20:05.175663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:05.175663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:20:05.175663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:20:05.175663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:05.176190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:20:05.176190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:05.176190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:20:09.867922 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:20:09.868601 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:20:09.868685 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:20:09.869934 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:20:09.903657 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:20:09.903937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:09.904551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:20:09.904551 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:09.904551 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:20:09.904551 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:20:09.904551 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:09.904551 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:20:09.904551 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:09.904551 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:20:09.905280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:09.905280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:20:09.905280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:09.905280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:20:09.905280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:20:09.905280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:20:09.905280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:20:09.905280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:09.905280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:09.905280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:09.905280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:20:09.905280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:09.906083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:09.906083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:09.906083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:20:09.906083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:09.906083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:20:09.906083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:20:09.906083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:09.906083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:20:09.906083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:09.906083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:20:09.906083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:20:09.906083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:09.906083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:20:09.906799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:09.906799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:20:09.906799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:09.906799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:20:09.906799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:20:09.906799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:20:09.906799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:20:09.906799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:20:09.906799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:09.906799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:20:09.906799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:20:09.906799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:20:09.906799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:20:09.906799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:20:09.906799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:09.907647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:09.907647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:20:09.907647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:20:09.907647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:20:09.907647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:20:09.907647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:09.907647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:09.907647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:09.907647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:20:09.907647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:09.907647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:09.907647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:09.907647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:20:09.907647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:09.907647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:20:09.907647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:20:09.907647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:09.907647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:20:09.907647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:09.908614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:20:09.908614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:20:09.908614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:09.908614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:20:09.908614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:09.908614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:20:10.114695 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:20:10.115295 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:20:10.115588 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:20:10.116914 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:20:10.117256 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:20:10.117536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:10.118634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:20:10.118634 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:10.118634 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:20:10.118634 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:20:10.118634 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:10.118634 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:20:10.118634 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:10.118634 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:20:10.120441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:10.120441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:20:10.120441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:10.120441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:20:10.120441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:20:10.120441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:20:10.120441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:20:10.120441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:10.120441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:10.120441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:10.120441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:20:10.120441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:10.121166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:10.121166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:10.121166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:20:10.121166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:10.121166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:20:10.121166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:20:10.121166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:10.121166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:20:10.121166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:10.121166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:20:10.121166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:20:10.121166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:10.121166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:20:10.121995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:10.121995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:20:10.122937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:10.122937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:20:10.122937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:20:10.122937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:20:10.122937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:20:10.122937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:20:10.122937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:10.122937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:20:10.122937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:20:10.122937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:20:10.122937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:20:10.122937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:20:10.122937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:10.122937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:10.122937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:20:10.122937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:20:10.123434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:20:10.123434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:20:10.123434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:10.123434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:10.123434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:10.123434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:20:10.123434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:10.123434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:10.123434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:10.123434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:20:10.123434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:10.123434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:20:10.123434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:20:10.123434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:10.123434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:20:10.123434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:10.123434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:20:10.123434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:20:10.123434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:10.123957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:20:10.123957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:10.123957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:20:10.123957 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:20:10.123957 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:20:10.123957 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:20:10.123957 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:20:10.124190 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:20:10.124397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:10.125068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:20:10.125068 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:10.125068 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:20:10.125068 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:20:10.125068 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:10.125068 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:20:10.125068 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:10.125068 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:20:10.125816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:10.125816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:20:10.125816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:20:10.125816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:20:10.125816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:10.125816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:20:10.125816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:10.125816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:20:10.125816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:10.125816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:20:10.125816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:10.125816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:20:10.125816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:10.125816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:20:10.125816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:20:10.126311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:10.126311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:20:10.126311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:10.126311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:20:10.126311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:20:10.126311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:10.126311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:20:10.126311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:10.126311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:20:10.126641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:10.126641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:20:10.126641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:10.126641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:20:10.126641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:20:10.126641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:20:10.126641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:20:10.126641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:20:10.126641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:20:10.126641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:20:10.126641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:10.126641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:10.127003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:10.127003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:20:10.127003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:10.127003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:10.127003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:10.127003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:20:10.127003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:10.127003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:20:10.127003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:20:10.127003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:10.127003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:20:10.127003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:10.127003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:20:10.127003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:20:10.127367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:10.127367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:20:10.127367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:10.127367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:20:10.127617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:10.127617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:20:10.127617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:20:10.127617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:20:10.127617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:20:10.127617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:20:10.127617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:10.127617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:20:10.127617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:20:10.127617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:20:10.127617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:20:10.127617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:20:10.127617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:10.127617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:10.127617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:20:10.127617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:20:10.128046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:20:10.128046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:20:10.128046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:20:10.128046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:20:10.128046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:20:10.128046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:10.128046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:10.128046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:10.128046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:20:10.128046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:10.128046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:10.128046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:10.128046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:20:10.128046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:10.128046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:20:10.128046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:20:10.128046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:10.128046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:20:10.128543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:10.128543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:20:10.128543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:20:10.128543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:10.128543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:20:10.128543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:10.128543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:20:10.164674 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:20:10.165194 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:20:10.165385 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:20:10.165625 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:20:10.166304 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:20:10.166698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:10.167495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:20:10.167495 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:10.167495 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:20:10.167495 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:20:10.167495 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:10.167495 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:20:10.167495 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:10.167495 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:20:10.168820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:10.168820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:20:10.168820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:20:10.168820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:20:10.168820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:10.168820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:20:10.168820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:10.168820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:20:10.168820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:10.168820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:20:10.168820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:10.168820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:20:10.168820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:10.168820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:20:10.168820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:20:10.169500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:10.169500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:20:10.169500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:10.169500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:20:10.169500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:20:10.169500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:10.169500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:20:10.169500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:10.169500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:20:10.170315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:10.170315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:20:10.170315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:10.170315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:20:10.170315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:20:10.170315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:20:10.170315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:20:10.170315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:20:10.170315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:20:10.170315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:10.170315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:10.170918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:10.170918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:20:10.170918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:10.170918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:10.170918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:10.170918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:20:10.170918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:10.170918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:20:10.170918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:20:10.170918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:10.170918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:20:10.170918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:10.170918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:20:10.170918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:20:10.171689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:10.171689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:20:10.171689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:10.171689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:20:10.172087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:10.172087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:20:10.172087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:20:10.172087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:20:10.172087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:20:10.172087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:20:10.172087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:10.172087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:20:10.172087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:20:10.172087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:20:10.172087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:20:10.172087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:20:10.172087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:10.172087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:10.172087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:20:10.172087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:20:10.172087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:20:10.172919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:20:10.172919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:20:10.172919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:20:10.172919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:10.172919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:10.172919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:10.172919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:20:10.172919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:10.172919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:10.172919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:10.172919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:20:10.172919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:10.172919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:20:10.172919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:20:10.172919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:10.172919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:20:10.172919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:10.172919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:20:10.172919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:20:10.173863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:10.173863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:20:10.173863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:10.173863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:20:10.173863 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:20:10.173863 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:20:10.173863 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:20:10.173863 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:20:10.174358 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:20:10.174358 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:20:10.174358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:10.174910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:20:10.174910 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:10.174910 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:20:10.174910 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:20:10.174910 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:10.174910 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:20:10.174910 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:10.174910 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:20:10.175768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:10.175768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:20:10.175768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:10.175768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:20:10.175768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:20:10.175768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:20:10.175768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:20:10.175768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:10.175768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:10.175768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:10.175768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:20:10.175768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:10.176118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:10.176118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:10.176118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:20:10.176118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:10.176118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:20:10.176118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:20:10.176118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:10.176118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:20:10.176118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:10.176118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:20:10.176118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:20:10.176118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:10.176118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:20:10.176469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:10.176469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:20:10.176759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:10.176759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:20:10.176759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:20:10.176759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:20:10.176759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:20:10.176759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:20:10.176759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:10.176759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:20:10.176759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:20:10.176759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:20:10.176759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:20:10.176759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:20:10.176759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:10.176759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:10.176759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:20:10.176759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:20:10.177199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:20:10.177199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:20:10.177199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:10.177199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:10.177199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:10.177199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:20:10.177199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:10.177199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:10.177199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:10.177199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:20:10.177199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:10.177199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:20:10.177199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:20:10.177199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:10.177199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:20:10.177199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:10.177199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:20:10.177199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:20:10.177199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:10.177759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:20:10.177759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:10.177759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:20:14.868477 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:20:14.869493 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:20:14.869493 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:20:14.871177 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:20:14.902187 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:20:14.902678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:14.903600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:20:14.903600 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:14.903600 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:20:14.903600 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:20:14.903600 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:14.903600 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:20:14.903600 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:14.903600 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:20:14.904315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:14.904315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:20:14.904315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:14.904315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:20:14.904315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:20:14.904315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:20:14.904315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:20:14.904315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:14.904315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:14.904315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:14.904315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:20:14.904315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:14.904979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:14.904979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:14.904979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:20:14.904979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:14.904979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:20:14.904979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:20:14.904979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:14.904979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:20:14.904979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:14.904979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:20:14.904979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:20:14.904979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:14.904979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:20:14.905608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:14.905608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:20:14.905608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:14.905608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:20:14.905608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:20:14.905608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:20:14.905608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:20:14.905608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:20:14.905608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:14.905608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:20:14.905608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:20:14.905608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:20:14.905608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:20:14.905608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:20:14.905608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:14.906321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:14.906321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:20:14.906321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:20:14.906321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:20:14.906321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:20:14.906321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:14.906321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:14.906321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:14.906321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:20:14.906321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:14.906321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:14.906321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:14.906321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:20:14.906321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:14.906321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:20:14.906321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:20:14.906321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:14.906321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:20:14.906321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:14.907244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:20:14.907244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:20:14.907244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:14.907244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:20:14.907244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:14.907244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:20:15.115338 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:20:15.115853 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:20:15.116053 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:20:15.117243 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:20:15.117682 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:20:15.117904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:15.118873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:20:15.118873 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:15.118873 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:20:15.118873 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:20:15.118873 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:15.118873 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:20:15.118873 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:15.118873 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:20:15.120173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:15.120173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:20:15.120173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:15.120173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:20:15.120173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:20:15.120173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:20:15.120173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:20:15.120173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:15.120173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:15.120173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:15.120173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:20:15.120173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:15.120768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:15.120768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:15.120768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:20:15.120768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:15.120768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:20:15.120768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:20:15.120768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:15.120768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:20:15.120768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:15.120768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:20:15.120768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:20:15.120768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:15.120768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:20:15.121364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:15.121364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:20:15.121815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:15.121815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:20:15.121815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:20:15.121815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:20:15.121815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:20:15.121815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:20:15.121815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:15.121815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:20:15.121815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:20:15.121815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:20:15.121815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:20:15.121815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:20:15.121815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:15.121815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:15.121815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:20:15.121815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:20:15.122345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:20:15.122345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:20:15.122345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:15.122345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:15.122345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:15.122345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:20:15.122345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:15.122345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:15.122345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:15.122345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:20:15.122345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:15.122345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:20:15.122345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:20:15.122345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:15.122345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:20:15.122345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:15.122345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:20:15.122345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:20:15.122345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:15.122870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:20:15.122870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:15.122870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:20:15.123497 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:20:15.123751 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:20:15.123863 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:20:15.124018 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:20:15.124422 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:20:15.124752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:15.125501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:20:15.125501 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:15.125501 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:20:15.125501 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:20:15.125501 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:15.125501 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:20:15.125501 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:15.125501 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:20:15.126300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:15.126300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:20:15.126300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:20:15.126300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:20:15.126300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:15.126300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:20:15.126300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:15.126300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:20:15.126300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:15.126300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:20:15.126300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:15.126300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:20:15.126300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:15.126300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:20:15.126300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:20:15.126779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:15.126779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:20:15.126779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:15.126779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:20:15.126779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:20:15.126779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:15.126779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:20:15.126779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:15.126779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:20:15.127050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:15.127050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:20:15.127050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:15.127050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:20:15.127050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:20:15.127050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:20:15.127050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:20:15.127050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:20:15.127050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:20:15.127050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:20:15.127050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:15.127050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:15.127433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:15.127433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:20:15.127433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:15.127433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:15.127433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:15.127433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:20:15.127433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:15.127433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:20:15.127433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:20:15.127433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:15.127433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:20:15.127433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:15.127433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:20:15.127433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:20:15.127790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:15.127790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:20:15.127790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:15.127790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:20:15.128079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:15.128079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:20:15.128079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:20:15.128079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:20:15.128079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:20:15.128079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:20:15.128079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:15.128079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:20:15.128079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:20:15.128079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:20:15.128079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:20:15.128079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:20:15.128079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:15.128079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:15.128079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:20:15.128079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:20:15.128519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:20:15.128519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:20:15.128519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:20:15.128519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:20:15.128519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:20:15.128519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:15.128519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:15.128519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:15.128519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:20:15.128519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:15.128519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:15.128519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:15.128519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:20:15.128519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:15.128519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:20:15.128519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:20:15.128519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:15.128519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:20:15.129063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:15.129063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:20:15.129063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:20:15.129063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:15.129063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:20:15.129063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:15.129063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:20:15.165191 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:20:15.165543 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:20:15.165660 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:20:15.165783 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:20:15.166227 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:20:15.166529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:15.167099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:20:15.167099 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:15.167099 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:20:15.167099 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:20:15.167099 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:15.167099 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:20:15.167099 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:15.167099 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:20:15.167851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:15.167851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:20:15.167851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:20:15.167851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:20:15.167851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:15.167851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:20:15.167851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:15.167851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:20:15.167851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:15.167851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:20:15.167851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:15.167851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:20:15.167851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:15.167851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:20:15.167851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:20:15.168260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:15.168260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:20:15.168260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:15.168260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:20:15.168260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:20:15.168260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:15.168260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:20:15.168260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:15.168260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:20:15.168695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:15.168695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:20:15.168695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:15.168695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:20:15.168695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:20:15.168695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:20:15.168695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:20:15.168695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:20:15.168695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:20:15.168695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:15.168695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:15.169060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:15.169060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:20:15.169060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:15.169060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:15.169060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:15.169060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:20:15.169060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:15.169060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:20:15.169060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:20:15.169060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:15.169060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:20:15.169060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:15.169060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:20:15.169060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:20:15.169429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:15.169429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:20:15.169429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:15.169429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:20:15.169760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:15.169760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:20:15.169760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:20:15.169760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:20:15.169760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:20:15.169760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:20:15.169760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:15.169760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:20:15.169760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:20:15.169760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:20:15.169760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:20:15.169760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:20:15.169760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:15.169760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:15.169760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:20:15.169760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:20:15.169760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:20:15.170201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:20:15.170201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:20:15.170201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:20:15.170201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:15.170201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:15.170201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:15.170201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:20:15.170201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:15.170201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:15.170201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:15.170201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:20:15.170201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:15.170201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:20:15.170201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:20:15.170201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:15.170201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:20:15.170201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:15.170201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:20:15.170201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:20:15.170685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:15.170685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:20:15.170685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:15.170685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:20:15.172945 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:20:15.173265 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:20:15.173371 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:20:15.173532 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:20:15.173835 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:20:15.174063 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:20:15.174215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:15.174732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:20:15.174732 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:15.174732 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:20:15.174732 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:20:15.174732 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:15.174732 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:20:15.174732 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:15.174732 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:20:15.175546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:15.175546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:20:15.175546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:15.175546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:20:15.175546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:20:15.175546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:20:15.175546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:20:15.175546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:15.175546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:15.175546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:15.175546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:20:15.175546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:15.175911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:15.175911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:15.175911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:20:15.175911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:15.175911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:20:15.175911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:20:15.175911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:15.175911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:20:15.175911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:15.175911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:20:15.175911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:20:15.175911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:15.175911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:20:15.176243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:15.176243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:20:15.176847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:15.176847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:20:15.176847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:20:15.176847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:20:15.176847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:20:15.176847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:20:15.176847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:15.176847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:20:15.176847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:20:15.176847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:20:15.176847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:20:15.176847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:20:15.176847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:15.176847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:15.176847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:20:15.176847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:20:15.177349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:20:15.177349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:20:15.177349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:15.177349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:15.177349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:15.177349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:20:15.177349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:15.177349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:15.177349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:15.177349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:20:15.177349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:15.177349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:20:15.177349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:20:15.177349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:15.177349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:20:15.177349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:15.177349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:20:15.177349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:20:15.177349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:15.177965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:20:15.177965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:15.177965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:20:19.869080 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:20:19.869556 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:20:19.869688 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:20:19.870566 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:20:19.900546 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:20:19.900921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:19.901792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:20:19.901792 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:19.901792 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:20:19.901792 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:20:19.901792 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:19.901792 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:20:19.901792 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:19.901792 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:20:19.903043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:19.903043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:20:19.903043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:19.903043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:20:19.903043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:20:19.903043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:20:19.903043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:20:19.903043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:19.903043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:19.903043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:19.903043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:20:19.903043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:19.903764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:19.903764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:19.903764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:20:19.903764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:19.903764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:20:19.903764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:20:19.903764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:19.903764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:20:19.903764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:19.903764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:20:19.903764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:20:19.903764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:19.903764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:20:19.904625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:19.904625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:20:19.904625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:19.904625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:20:19.904625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:20:19.904625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:20:19.904625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:20:19.904625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:20:19.904625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:19.904625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:20:19.904625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:20:19.904625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:20:19.904625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:20:19.904625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:20:19.904625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:19.905383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:19.905383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:20:19.905383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:20:19.905383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:20:19.905383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:20:19.905383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:19.905383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:19.905383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:19.905383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:20:19.905383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:19.905383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:19.905383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:19.905383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:20:19.905383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:19.905383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:20:19.905383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:20:19.905383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:19.905383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:20:19.905383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:19.906315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:20:19.906315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:20:19.906315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:19.906315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:20:19.906315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:19.906315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:20:20.116028 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:20:20.116665 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:20:20.116852 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:20:20.117879 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:20:20.118234 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:20:20.118522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:20.119495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:20:20.119495 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:20.119495 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:20:20.119495 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:20:20.119495 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:20.119495 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:20:20.119495 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:20.119495 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:20:20.121005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:20.121005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:20:20.121005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:20.121005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:20:20.121005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:20:20.121005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:20:20.121005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:20:20.121005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:20.121005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:20.121005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:20.121005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:20:20.121005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:20.121871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:20.121871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:20.121871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:20:20.121871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:20.121871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:20:20.121871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:20:20.121871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:20.121871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:20:20.121871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:20.121871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:20:20.121871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:20:20.121871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:20.121871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:20:20.122724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:20.122724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:20:20.122724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:20.122724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:20:20.122724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:20:20.122724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:20:20.122724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:20:20.122724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:20:20.122724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:20.122724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:20:20.122724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:20:20.122724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:20:20.122724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:20:20.122724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:20:20.122724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:20.123317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:20.123317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:20:20.123317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:20:20.123317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:20:20.123317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:20:20.123317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:20.123317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:20.123317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:20.123317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:20:20.123317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:20.123317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:20.123317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:20.123317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:20:20.123317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:20.123317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:20:20.123317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:20:20.123317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:20.123317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:20:20.123317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:20.123901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:20:20.123901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:20:20.123901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:20.123901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:20:20.123901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:20.123901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:20:20.123901 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:20:20.124145 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:20:20.124190 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:20:20.124376 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:20:20.124804 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:20:20.125101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:20.125845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:20:20.125845 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:20.125845 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:20:20.125845 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:20:20.125845 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:20.125845 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:20:20.125845 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:20.125845 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:20:20.126573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:20.126573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:20:20.126573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:20:20.126573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:20:20.126573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:20.126573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:20:20.126573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:20.126573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:20:20.126573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:20.126573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:20:20.126573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:20.126573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:20:20.126573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:20.126573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:20:20.126573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:20:20.127069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:20.127069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:20:20.127069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:20.127069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:20:20.127069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:20:20.127069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:20.127069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:20:20.127069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:20.127069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:20:20.127419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:20.127419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:20:20.127419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:20.127419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:20:20.127419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:20:20.127419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:20:20.127419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:20:20.127419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:20:20.127419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:20:20.127419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:20:20.127419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:20.127419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:20.127757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:20.127757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:20:20.127757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:20.127757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:20.127757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:20.127757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:20:20.127757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:20.127757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:20:20.127757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:20:20.127757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:20.127757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:20:20.127757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:20.127757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:20:20.127757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:20:20.128172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:20.128172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:20:20.128172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:20.128172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:20:20.128491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:20.128491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:20:20.128491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:20:20.128491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:20:20.128491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:20:20.128491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:20:20.128491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:20.128491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:20:20.128491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:20:20.128491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:20:20.128491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:20:20.128491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:20:20.128491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:20.128491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:20.128491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:20:20.128491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:20:20.128916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:20:20.128916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:20:20.128916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:20:20.128916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:20:20.128916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:20:20.128916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:20.128916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:20.128916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:20.128916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:20:20.128916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:20.128916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:20.128916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:20.128916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:20:20.128916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:20.128916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:20:20.128916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:20:20.128916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:20.128916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:20:20.129370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:20.129370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:20:20.129370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:20:20.129370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:20.129370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:20:20.129370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:20.129370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:20:20.165648 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:20:20.165953 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:20:20.166070 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:20:20.166197 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:20:20.166765 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:20:20.167228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:20.168284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:20:20.168284 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:20.168284 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:20:20.168284 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:20:20.168284 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:20.168284 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:20:20.168284 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:20.168284 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:20:20.169479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:20.169479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:20:20.169479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:20:20.169479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:20:20.169479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:20.169479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:20:20.169479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:20.169479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:20:20.169479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:20.169479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:20:20.169479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:20.169479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:20:20.169479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:20.169479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:20:20.169479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:20:20.170309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:20.170309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:20:20.170309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:20.170309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:20:20.170309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:20:20.170309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:20.170309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:20:20.170309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:20.170309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:20:20.170309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:20.170309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:20:20.170309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:20.170309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:20:20.170309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:20:20.171136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:20:20.171136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:20:20.171136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:20:20.171136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:20:20.171136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:20.171136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:20.171136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:20.171136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:20:20.171136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:20.171136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:20.171136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:20.171136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:20:20.171136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:20.171771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:20:20.171771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:20:20.171771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:20.171771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:20:20.171771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:20.171771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:20:20.171771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:20:20.171771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:20.171771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:20:20.171771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:20.171771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:20:20.171771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:20.172307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:20:20.172307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:20:20.172307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:20:20.172307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:20:20.172307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:20:20.172307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:20.172307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:20:20.172307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:20:20.172307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:20:20.172307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:20:20.172307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:20:20.172307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:20.172307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:20.172307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:20:20.172307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:20:20.172307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:20:20.172307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:20:20.172307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:20:20.172307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:20:20.173352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:20.173352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:20.173352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:20.173352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:20:20.173352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:20.173352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:20.173352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:20.173352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:20:20.173352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:20.173352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:20:20.173352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:20:20.173352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:20.173352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:20:20.173352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:20.173352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:20:20.173352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:20:20.173352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:20.173352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:20:20.173352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:20.173352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:20:20.174162 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:20:20.174220 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:20:20.174498 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:20:20.174893 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:20:20.175334 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:20:20.175677 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:20:20.175890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:20.176576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:20:20.176576 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:20.176576 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:20:20.176576 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:20:20.176576 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:20.176576 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:20:20.176576 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:20.176576 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:20:20.177369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:20.177369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:20:20.177369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:20.177369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:20:20.177369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:20:20.177369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:20:20.177369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:20:20.177369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:20.177369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:20.177369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:20.177369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:20:20.177369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:20.177637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:20.177637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:20.177637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:20:20.177637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:20.177637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:20:20.177637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:20:20.177637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:20.177637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:20:20.177637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:20.177637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:20:20.177637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:20:20.177637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:20.177637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:20:20.177877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:20.177877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:20:20.178385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:20.178385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:20:20.178385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:20:20.178385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:20:20.178385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:20:20.178385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:20:20.178385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:20.178385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:20:20.178385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:20:20.178385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:20:20.178385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:20:20.178385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:20:20.178385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:20.178385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:20.178385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:20:20.178385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:20:20.178706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:20:20.178706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:20:20.178706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:20.178706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:20.178706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:20.178706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:20:20.178706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:20.178706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:20.178706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:20.178706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:20:20.178706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:20.178706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:20:20.178706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:20:20.178706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:20.178706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:20:20.178706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:20.178706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:20:20.178706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:20:20.178706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:20.179079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:20:20.179079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:20.179079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:20:24.870002 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:20:24.870917 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:20:24.870917 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:20:24.872383 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:20:24.874305 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG dbcounter [-] [105581] Writing DB stats neutron:SELECT=434 {{(pid=105581) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 14:20:24.901180 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:20:24.901397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:24.901941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:20:24.901941 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:24.901941 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:20:24.901941 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:20:24.901941 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:24.901941 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:20:24.901941 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:24.901941 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:20:24.902611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.031 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:24.902611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:20:24.902611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:24.902611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:20:24.902611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:20:24.902611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:20:24.902611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:20:24.902611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:24.902611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:24.902611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:24.902611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:20:24.902611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:24.903444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:24.903444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:24.903444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:20:24.903444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:24.903444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:20:24.903444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:20:24.903444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:24.903444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:20:24.903444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:24.903444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:20:24.903444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:20:24.903444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:24.903444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:20:24.904069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:24.904069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:20:24.904069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:24.904069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:20:24.904069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:20:24.904069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:20:24.904069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:20:24.904069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:20:24.904069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:24.904069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:20:24.904069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:20:24.904069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:20:24.904069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:20:24.904069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:20:24.904069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:24.904805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:24.904805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:20:24.904805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:20:24.904805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:20:24.904805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:20:24.904805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:24.904805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:24.904805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:24.904805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:20:24.904805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:24.904805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:24.904805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:24.904805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:20:24.904805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:24.904805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:20:24.904805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:20:24.904805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:24.904805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:20:24.904805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:24.905704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:20:24.905704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:20:24.905704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:24.905704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:20:24.905704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:24.905704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:20:25.116789 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:20:25.117220 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:20:25.117452 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:20:25.118426 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:20:25.118797 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:20:25.119073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:25.120205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:20:25.120205 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:25.120205 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:20:25.120205 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:20:25.120205 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:25.120205 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:20:25.120205 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:25.120205 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:20:25.121616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:25.121616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:20:25.121616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:25.121616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:20:25.121616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:20:25.121616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:20:25.121616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:20:25.121616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:25.121616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:25.121616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:25.121616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:20:25.121616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:25.121968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:25.121968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:25.121968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:20:25.121968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:25.121968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:20:25.121968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:20:25.121968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:25.121968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:20:25.121968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:25.121968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:20:25.121968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:20:25.121968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:25.121968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:20:25.122289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:25.122289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:20:25.123680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:25.123680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:20:25.123680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:20:25.123680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:20:25.123680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:20:25.123680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:20:25.123680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:25.123680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:20:25.123680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:20:25.123680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:20:25.123680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:20:25.123680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:20:25.123680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:25.123680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:25.123680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:20:25.123680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:20:25.124172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:20:25.124172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:20:25.124172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:25.124172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:25.124172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:25.124172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:20:25.124172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:25.124172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:25.124172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:25.124172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:20:25.124172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:25.124172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:20:25.124172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:20:25.124172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:25.124172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:20:25.124172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:25.124172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:20:25.124172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:20:25.124172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:25.124729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:20:25.124729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:25.124729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:20:25.124857 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:20:25.125285 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:20:25.125502 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:20:25.125693 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:20:25.126000 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:20:25.126209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:25.126817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:20:25.126817 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:25.126817 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:20:25.126817 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:20:25.126817 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:25.126817 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:20:25.126817 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:25.126817 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:20:25.128511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:25.128511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:20:25.128511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:20:25.128511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:20:25.128511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:25.128511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:20:25.128511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:25.128511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:20:25.128511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:25.128511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:20:25.128511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:25.128511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:20:25.128511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:25.128511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:20:25.128511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:20:25.129109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:25.129109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:20:25.129109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:25.129109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:20:25.129109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:20:25.129109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:25.129109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:20:25.129109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:25.129109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:20:25.129109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:25.129109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:20:25.129109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:25.129109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:20:25.129109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:20:25.129554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:20:25.129554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:20:25.129554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:20:25.129554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:20:25.129554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:20:25.129554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:25.129554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:25.129554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:25.129554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:20:25.129554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:25.129554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:25.129554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:25.129554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:20:25.129981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:25.129981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:20:25.129981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:20:25.129981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:25.129981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:20:25.129981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:25.129981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:20:25.129981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:20:25.129981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:25.129981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:20:25.129981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:25.129981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:20:25.130308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:25.130308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:20:25.130308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:20:25.130308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:20:25.130308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:20:25.130308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:20:25.130308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:25.130308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:20:25.130308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:20:25.130308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:20:25.130308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:20:25.130308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:20:25.130308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:25.130308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:25.130308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:20:25.130308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:20:25.130812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:20:25.130812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:20:25.130812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:20:25.130812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:20:25.130812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:20:25.130812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:25.130812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:25.130812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:25.130812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:20:25.130812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:25.130812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:25.130812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:25.130812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:20:25.130812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:25.130812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:20:25.130812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:20:25.130812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:25.130812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:20:25.131344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:25.131344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:20:25.131344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:20:25.131344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:25.131344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:20:25.131344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:25.131344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:20:25.166131 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:20:25.166633 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:20:25.166741 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:20:25.166887 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:20:25.167433 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:20:25.167954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:25.168802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:20:25.168802 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:25.168802 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:20:25.168802 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:20:25.168802 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:25.168802 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:20:25.168802 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:25.168802 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:20:25.169735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:25.169735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:20:25.169735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:20:25.169735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:20:25.169735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:25.169735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:20:25.169735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:25.169735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:20:25.169735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:25.169735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:20:25.169735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:25.169735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:20:25.169735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:25.169735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:20:25.169735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:20:25.170501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:25.170501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:20:25.170501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:25.170501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:20:25.170501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:20:25.170501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:25.170501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:20:25.170501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:25.170501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:20:25.170904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:25.170904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:20:25.170904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:25.170904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:20:25.170904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:20:25.170904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:20:25.170904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:20:25.170904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:20:25.170904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:20:25.170904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:25.170904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:25.171677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:25.171677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:20:25.171677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:25.171677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:25.171677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:25.171677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:20:25.171677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:25.171677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:20:25.171677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:20:25.171677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:25.171677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:20:25.171677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:25.171677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:20:25.171677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:20:25.172333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:25.172333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:20:25.172333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:25.172333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:20:25.172333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:25.172333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:20:25.172333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:20:25.172333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:20:25.172333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:20:25.172333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:20:25.172333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:25.172333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:20:25.172333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:20:25.172333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:20:25.173216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:20:25.173216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:20:25.173216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:25.173216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:25.173216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:20:25.173216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:20:25.173216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:20:25.173216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:20:25.173216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:20:25.173216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:20:25.173216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:25.173216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:25.173216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:25.173216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:20:25.173216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:25.173216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:25.173216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:25.173216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:20:25.173216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:25.174148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:20:25.174148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:20:25.174148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:25.174148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:20:25.174148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:25.174148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:20:25.174148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:20:25.174148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:25.174148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:20:25.174148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:25.174148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:20:25.174787 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:20:25.174787 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:20:25.174921 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:20:25.175030 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:20:25.175465 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:20:25.175779 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:20:25.175964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:25.176651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:20:25.176651 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:25.176651 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:20:25.176651 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:20:25.176651 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:25.176651 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:20:25.176651 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:25.176651 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:20:25.178154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:25.178154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:20:25.178154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:25.178154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:20:25.178154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:20:25.178154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:20:25.178154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:20:25.178154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:25.178154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:25.178154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:25.178154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:20:25.178154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:25.178538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:25.178538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:25.178538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:20:25.178538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:25.178538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:20:25.178538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:20:25.178538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:25.178538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:20:25.178538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:25.178538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:20:25.178538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:20:25.178538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:25.178538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:20:25.178877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:25.178877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:20:25.179889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:25.179889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:20:25.179889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:20:25.179889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:20:25.179889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:20:25.179889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:20:25.179889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:25.179889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:20:25.179889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:20:25.179889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:20:25.179889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:20:25.179889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:20:25.179889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:25.179889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:25.179889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:20:25.179889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:20:25.180391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:20:25.180391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:20:25.180391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:25.180391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:25.180391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:25.180391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:20:25.180391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:25.180391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:25.180391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:25.180391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:20:25.180391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:25.180391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:20:25.180391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:20:25.180391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:25.180391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:20:25.180391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:25.180391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:20:25.180391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:20:25.180391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:25.180946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:20:25.180946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:25.180946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:20:29.870703 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:20:29.871278 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:20:29.871278 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:20:29.872729 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:20:29.904103 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:20:29.904570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:29.905453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:20:29.905453 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:29.905453 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:20:29.905453 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:20:29.905453 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:29.905453 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:20:29.905453 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:29.905453 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:20:29.906175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:29.906175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:20:29.906175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:29.906175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:20:29.906175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:20:29.906175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:20:29.906175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:20:29.906175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:29.906175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:29.906175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:29.906175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:20:29.906175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:29.906820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:29.906820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:29.906820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:20:29.906820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:29.906820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:20:29.906820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:20:29.906820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:29.906820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:20:29.906820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:29.906820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:20:29.906820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:20:29.906820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:29.906820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:20:29.907451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:29.907451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:20:29.907451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:29.907451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:20:29.907451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:20:29.907451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:20:29.907451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:20:29.907451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:20:29.907451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:29.907451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:20:29.907451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:20:29.907451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:20:29.907451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:20:29.907451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:20:29.907451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:29.908155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:29.908155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:20:29.908155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:20:29.908155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:20:29.908155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:20:29.908155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:29.908155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:29.908155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:29.908155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:20:29.908155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:29.908155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:29.908155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:29.908155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:20:29.908155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:29.908155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:20:29.908155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:20:29.908155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:29.908155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:20:29.908155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:29.909011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:20:29.909011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:20:29.909011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:29.909011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:20:29.909011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:29.909011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:20:30.117443 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:20:30.117891 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:20:30.118120 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:20:30.119114 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:20:30.119395 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:20:30.119667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:30.120568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:20:30.120568 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:30.120568 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:20:30.120568 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:20:30.120568 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:30.120568 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:20:30.120568 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:30.120568 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:20:30.122056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:30.122056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:20:30.122056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:30.122056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:20:30.122056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:20:30.122056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:20:30.122056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:20:30.122056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:30.122056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:30.122056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:30.122056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:20:30.122056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:30.122453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:30.122453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:30.122453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:20:30.122453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:30.122453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:20:30.122453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:20:30.122453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:30.122453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:20:30.122453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:30.122453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:20:30.122453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:20:30.122453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:30.122453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:20:30.122818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:30.122818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:20:30.123858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:30.123858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:20:30.123858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:20:30.123858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:20:30.123858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:20:30.123858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:20:30.123858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:30.123858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:20:30.123858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:20:30.123858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:20:30.123858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:20:30.123858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:20:30.123858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:30.123858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:30.123858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:20:30.123858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:20:30.124328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:20:30.124328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:20:30.124328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:30.124328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:30.124328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:30.124328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:20:30.124328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:30.124328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:30.124328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:30.124328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:20:30.124328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:30.124328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:20:30.124328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:20:30.124328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:30.124328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:20:30.124328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:30.124328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:20:30.124328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:20:30.124328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:30.124928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:20:30.124928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:30.124928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:20:30.125245 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:20:30.125534 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:20:30.125678 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:20:30.125804 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:20:30.126064 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:20:30.126258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:30.126688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:20:30.126688 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:30.126688 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:20:30.126688 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:20:30.126688 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:30.126688 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:20:30.126688 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:30.126688 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:20:30.127511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:30.127511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:20:30.127511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:20:30.127511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:20:30.127511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:30.127511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:20:30.127511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:30.127511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:20:30.127511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:30.127511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:20:30.127511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:30.127511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:20:30.127511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:30.127511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:20:30.127511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:20:30.128006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:30.128006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:20:30.128006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:30.128006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:20:30.128006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:20:30.128006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:30.128006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:20:30.128006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:30.128006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:20:30.128369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:30.128369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:20:30.128369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:30.128369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:20:30.128369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:20:30.128369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:20:30.128369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:20:30.128369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:20:30.128369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:20:30.128369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:20:30.128369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:30.128369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:30.128742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:30.128742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:20:30.128742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:30.128742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:30.128742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:30.128742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:20:30.128742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:30.128742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:20:30.128742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:20:30.128742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:30.128742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:20:30.128742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:30.128742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:20:30.128742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:20:30.129202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:30.129202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:20:30.129202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:30.129202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:20:30.129397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:30.129397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:20:30.129397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:20:30.129397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:20:30.129397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:20:30.129397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:20:30.129397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:30.129397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:20:30.129397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:20:30.129397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:20:30.129397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:20:30.129397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:20:30.129397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:30.129397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:30.129397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:20:30.129397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:20:30.129920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:20:30.129920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:20:30.129920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:20:30.129920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:20:30.129920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:20:30.129920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:30.129920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:30.129920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:30.129920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:20:30.129920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:30.129920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:30.129920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:30.129920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:20:30.129920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:30.129920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:20:30.129920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:20:30.129920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:30.129920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:20:30.130593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:30.130593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:20:30.130593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:20:30.130593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:30.130593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:20:30.130593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:30.130593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:20:30.166623 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:20:30.167001 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:20:30.167195 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:20:30.167423 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:20:30.167838 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:20:30.168119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:30.168708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:20:30.168708 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:30.168708 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:20:30.168708 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:20:30.168708 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:30.168708 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:20:30.168708 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:30.168708 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:20:30.169435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:30.169435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:20:30.169435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:20:30.169435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:20:30.169435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:30.169435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:20:30.169435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:30.169435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:20:30.169435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:30.169435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:20:30.169435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:30.169435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:20:30.169435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:30.169435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:20:30.169435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:20:30.170178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:30.170178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:20:30.170178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:30.170178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:20:30.170178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:20:30.170178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:30.170178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:20:30.170178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:30.170178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:20:30.170702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:30.170702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:20:30.170702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:30.170702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:20:30.170702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:20:30.170702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:20:30.170702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:20:30.170702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:20:30.170702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:20:30.170702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:30.170702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:30.171290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:30.171290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:20:30.171290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:30.171290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:30.171290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:30.171290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:20:30.171290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:30.171290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:20:30.171290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:20:30.171290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:30.171290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:20:30.171290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:30.171290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:20:30.171290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:20:30.172039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:30.172039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:20:30.172039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:30.172039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:20:30.172039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:30.172039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:20:30.172039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:20:30.172039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:20:30.172039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:20:30.172039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:20:30.172039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:30.172039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:20:30.172039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:20:30.172039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:20:30.172710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:20:30.172710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:20:30.172710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:30.172710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:30.172710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:20:30.172710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:20:30.172710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:20:30.172710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:20:30.172710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:20:30.172710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:20:30.172710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:30.172710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:30.172710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:30.172710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:20:30.172710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:30.172710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:30.172710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:30.172710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:20:30.172710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:30.173645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:20:30.173645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:20:30.173645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:30.173645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:20:30.173645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:30.173645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:20:30.173645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:20:30.173645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:30.173645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:20:30.173645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:30.173645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:20:30.174922 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:20:30.175180 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:20:30.175279 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:20:30.175433 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:20:30.175955 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:20:30.176369 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:20:30.176653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:30.177425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:20:30.177425 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:30.177425 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:20:30.177425 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:20:30.177425 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:30.177425 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:20:30.177425 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:30.177425 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:20:30.178266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:30.178266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:20:30.178266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:30.178266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:20:30.178266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:20:30.178266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:20:30.178266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:20:30.178266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:30.178266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:30.178266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:30.178266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:20:30.178266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:30.178619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:30.178619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:30.178619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:20:30.178619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:30.178619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:20:30.178619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:20:30.178619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:30.178619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:20:30.178619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:30.178619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:20:30.178619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:20:30.178619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:30.178619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:20:30.178982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:30.178982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:20:30.179222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:30.179222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:20:30.179222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:20:30.179222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:20:30.179222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:20:30.179222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:20:30.179222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:30.179222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:20:30.179222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:20:30.179222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:20:30.179222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:20:30.179222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:20:30.179222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:30.179222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:30.179222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:20:30.179222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:20:30.179673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:20:30.179673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:20:30.179673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:30.179673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:30.179673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:30.179673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:20:30.179673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:30.179673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:30.179673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:30.179673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:20:30.179673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:30.179673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:20:30.179673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:20:30.179673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:30.179673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:20:30.179673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:30.179673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:20:30.179673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:20:30.179673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:30.180184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:20:30.180184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:30.180184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:20:34.871471 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:20:34.872656 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:20:34.872656 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:20:34.873916 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:20:34.904353 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:20:34.904617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:34.905194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:20:34.905194 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:34.905194 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:20:34.905194 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:20:34.905194 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:34.905194 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:20:34.905194 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:34.905194 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:20:34.906187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:34.906187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:20:34.906187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:34.906187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:20:34.906187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:20:34.906187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:20:34.906187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:20:34.906187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:34.906187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:34.906187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:34.906187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:20:34.906187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:34.906900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:34.906900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:34.906900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:20:34.906900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:34.906900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:20:34.906900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:20:34.906900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:34.906900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:20:34.906900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:34.906900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:20:34.906900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:20:34.906900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:34.906900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:20:34.907634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:34.907634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:20:34.907634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:34.907634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:20:34.907634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:20:34.907634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:20:34.907634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:20:34.907634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:20:34.907634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:34.907634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:20:34.907634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:20:34.907634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:20:34.907634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:20:34.907634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:20:34.907634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:34.908477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:34.908477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:20:34.908477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:20:34.908477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:20:34.908477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:20:34.908477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:34.908477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:34.908477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:34.908477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:20:34.908477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:34.908477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:34.908477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:34.908477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:20:34.908477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:34.908477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:20:34.908477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:20:34.908477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:34.908477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:20:34.908477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:34.909482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:20:34.909482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:20:34.909482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:34.909482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:20:34.909482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:34.909482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:20:35.118059 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:20:35.118580 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:20:35.118789 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:20:35.119866 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:20:35.120144 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:20:35.120367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:35.121244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:20:35.121244 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:35.121244 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:20:35.121244 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:20:35.121244 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:35.121244 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:20:35.121244 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:35.121244 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:20:35.122959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:35.122959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:20:35.122959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:35.122959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:20:35.122959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:20:35.122959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:20:35.122959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:20:35.122959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:35.122959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:35.122959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:35.122959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:20:35.122959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:35.123623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:35.123623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:35.123623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:20:35.123623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:35.123623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:20:35.123623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:20:35.123623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:35.123623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:20:35.123623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:35.123623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:20:35.123623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:20:35.123623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:35.123623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:20:35.124235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:35.124235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:20:35.124965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:35.124965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:20:35.124965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:20:35.124965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:20:35.124965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:20:35.124965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:20:35.124965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:35.124965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:20:35.124965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:20:35.124965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:20:35.124965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:20:35.124965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:20:35.124965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:35.124965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:35.124965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:20:35.124965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:20:35.125733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:20:35.125733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:20:35.125733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:35.125733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:35.125733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:35.125733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:20:35.125733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:35.125733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:35.125733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:35.125733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:20:35.125733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:35.125733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:20:35.125733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:20:35.125733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:35.125733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:20:35.125733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:35.125733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:20:35.125733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:20:35.125733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:35.126582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:20:35.126582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:35.126582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:20:35.126582 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:20:35.126582 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:20:35.126582 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:20:35.126582 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:20:35.126999 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:20:35.126999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:35.127511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:20:35.127511 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:35.127511 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:20:35.127511 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:20:35.127511 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:35.127511 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:20:35.127511 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:35.127511 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:20:35.128325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:35.128325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:20:35.128325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:20:35.128325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:20:35.128325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:35.128325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:20:35.128325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:35.128325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:20:35.128325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:35.128325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:20:35.128325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:35.128325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:20:35.128325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:35.128325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:20:35.128325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:20:35.128953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:35.128953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:20:35.128953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:35.128953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:20:35.128953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:20:35.128953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:35.128953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:20:35.128953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:35.128953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:20:35.129397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:35.129397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:20:35.129397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:35.129397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:20:35.129397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:20:35.129397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:20:35.129397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:20:35.129397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:20:35.129397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:20:35.129397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:20:35.129397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:35.129397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:35.129802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:35.129802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:20:35.129802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:35.129802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:35.129802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:35.129802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:20:35.129802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:35.129802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:20:35.129802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:20:35.129802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:35.129802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:20:35.129802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:35.129802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:20:35.129802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:20:35.130290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:35.130290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:20:35.130290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:35.130290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:20:35.130668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:35.130668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:20:35.130668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:20:35.130668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:20:35.130668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:20:35.130668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:20:35.130668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:35.130668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:20:35.130668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:20:35.130668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:20:35.130668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:20:35.130668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:20:35.130668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:35.130668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:35.130668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:20:35.130668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:20:35.131119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:20:35.131119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:20:35.131119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:20:35.131119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:20:35.131119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:20:35.131119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:35.131119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:35.131119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:35.131119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:20:35.131119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:35.131119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:35.131119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:35.131119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:20:35.131119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:35.131119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:20:35.131119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:20:35.131119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:35.131119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:20:35.131673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:35.131673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:20:35.131673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:20:35.131673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:35.131673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:20:35.131673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:35.131673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:20:35.167020 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:20:35.167253 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:20:35.167356 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:20:35.167577 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:20:35.167898 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:20:35.168151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:35.168664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:20:35.168664 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:35.168664 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:20:35.168664 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:20:35.168664 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:35.168664 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:20:35.168664 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:35.168664 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:20:35.169428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:35.169428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:20:35.169428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:20:35.169428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:20:35.169428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:35.169428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:20:35.169428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:35.169428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:20:35.169428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:35.169428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:20:35.169428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:35.169428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:20:35.169428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:35.169428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:20:35.169428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:20:35.170111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:35.170111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:20:35.170111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:35.170111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:20:35.170111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:20:35.170111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:35.170111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:20:35.170111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:35.170111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:20:35.170622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:35.170622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:20:35.170622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:35.170622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:20:35.170622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:20:35.170622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:20:35.170622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:20:35.170622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:20:35.170622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:20:35.170622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:35.170622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:35.171186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:35.171186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:20:35.171186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:35.171186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:35.171186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:35.171186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:20:35.171186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:35.171186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:20:35.171186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:20:35.171186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:35.171186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:20:35.171186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:35.171186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:20:35.171186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:20:35.172074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:35.172074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:20:35.172074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:35.172074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:20:35.172074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:35.172074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:20:35.172074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:20:35.172074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:20:35.172074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:20:35.172074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:20:35.172074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:35.172074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:20:35.172074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:20:35.172074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:20:35.172716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:20:35.172716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:20:35.172716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:35.172716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:35.172716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:20:35.172716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:20:35.172716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:20:35.172716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:20:35.172716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:20:35.172716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:20:35.172716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:35.172716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:35.172716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:35.172716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:20:35.172716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:35.172716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:35.172716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:35.172716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:20:35.172716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:35.173239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:20:35.173239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:20:35.173239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:35.173239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:20:35.173239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:35.173239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:20:35.173239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:20:35.173239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:35.173239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:20:35.173239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:35.173239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:20:35.175231 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:20:35.175474 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:20:35.175584 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:20:35.175766 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:20:35.176073 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:20:35.176285 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:20:35.176447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:35.176923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:20:35.176923 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:35.176923 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:20:35.176923 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:20:35.176923 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:35.176923 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:20:35.176923 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:35.176923 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:20:35.177730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:35.177730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:20:35.177730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:35.177730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:20:35.177730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:20:35.177730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:20:35.177730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:20:35.177730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:35.177730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:35.177730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:35.177730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:20:35.177730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:35.178067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:35.178067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:35.178067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:20:35.178067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:35.178067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:20:35.178067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:20:35.178067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:35.178067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:20:35.178067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:35.178067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:20:35.178067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:20:35.178067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:35.178067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:20:35.178455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:35.178455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:20:35.178668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:35.178668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:20:35.178668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:20:35.178668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:20:35.178668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:20:35.178668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:20:35.178668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:35.178668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:20:35.178668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:20:35.178668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:20:35.178668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:20:35.178668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:20:35.178668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:35.178668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:35.178668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:20:35.178668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:20:35.179115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:20:35.179115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:20:35.179115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:35.179115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:35.179115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:35.179115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:20:35.179115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:35.179115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:35.179115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:35.179115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:20:35.179115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:35.179115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:20:35.179115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:20:35.179115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:35.179115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:20:35.179115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:35.179115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:20:35.179115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:20:35.179115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:35.179713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:20:35.179713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:35.179713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:20:39.872097 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:20:39.873011 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:20:39.873011 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:20:39.874155 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:20:39.905737 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:20:39.905926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:39.906474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:20:39.906474 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:39.906474 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:20:39.906474 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:20:39.906474 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:39.906474 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:20:39.906474 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:39.906474 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:20:39.907548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:39.907548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:20:39.907548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:39.907548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:20:39.907548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:20:39.907548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:20:39.907548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:20:39.907548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:39.907548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:39.907548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:39.907548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:20:39.907548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:39.908202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:39.908202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:39.908202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:20:39.908202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:39.908202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:20:39.908202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:20:39.908202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:39.908202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:20:39.908202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:39.908202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:20:39.908202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:20:39.908202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:39.908202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:20:39.908817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:39.908817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:20:39.908817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:39.908817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:20:39.908817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:20:39.908817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:20:39.908817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:20:39.908817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:20:39.908817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:39.908817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:20:39.908817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:20:39.908817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:20:39.908817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:20:39.908817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:20:39.908817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:39.909564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:39.909564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:20:39.909564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:20:39.909564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:20:39.909564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:20:39.909564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:39.909564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:39.909564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:39.909564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:20:39.909564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:39.909564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:39.909564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:39.909564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:20:39.909564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:39.909564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:20:39.909564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:20:39.909564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:39.909564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:20:39.909564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:39.910909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:20:39.910909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:20:39.910909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:39.910909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:20:39.910909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:39.910909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:20:40.118792 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:20:40.119431 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:20:40.119593 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:20:40.120618 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:20:40.120982 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:20:40.121200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:40.122103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:20:40.122103 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:40.122103 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:20:40.122103 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:20:40.122103 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:40.122103 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:20:40.122103 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:40.122103 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:20:40.123715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:40.123715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:20:40.123715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:40.123715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:20:40.123715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:20:40.123715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:20:40.123715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:20:40.123715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:40.123715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:40.123715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:40.123715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:20:40.123715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:40.125533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:40.125533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:40.125533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:20:40.125533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:40.125533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:20:40.125533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:20:40.125533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:40.125533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:20:40.125533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:40.125533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:20:40.125533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:20:40.125533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:40.125533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:20:40.126575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:40.126575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:20:40.126575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:40.126575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:20:40.126575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:20:40.126575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:20:40.126575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:20:40.126575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:20:40.126575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:40.126575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:20:40.126575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:20:40.126575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:20:40.126575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:20:40.126575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:20:40.126575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:40.127338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:40.127338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:20:40.127338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:20:40.127338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:20:40.127338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:20:40.127338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:40.127338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:40.127338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:40.127338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:20:40.127338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:40.127338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:40.127338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:40.127338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:20:40.127338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:40.127338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:20:40.127338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:20:40.127338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:40.127338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:20:40.127338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:40.128171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:20:40.128171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:20:40.128171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:40.128171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:20:40.128171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:40.128171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:20:40.128171 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:20:40.128171 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:20:40.128171 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:20:40.128171 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:20:40.129181 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:20:40.129181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:40.129181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:20:40.129181 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:40.129181 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:20:40.129181 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:20:40.129181 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:40.129181 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:20:40.129181 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:40.129181 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:20:40.129181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:40.129181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:20:40.129181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:20:40.129181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:20:40.130523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:40.130523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:20:40.130523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:40.130523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:20:40.130523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:40.130523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:20:40.130523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:40.130523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:20:40.130523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:40.130523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:20:40.130523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:20:40.130523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:40.130523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:20:40.130523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:40.130523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:20:40.130523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:20:40.130523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:40.130523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:20:40.132246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:40.132246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:20:40.132246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:40.132246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:20:40.132246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:40.132246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:20:40.132246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:20:40.132246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:20:40.132246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:20:40.132246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:20:40.132246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:20:40.132246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:20:40.134112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:40.134112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:40.134112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:40.134112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:20:40.134112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:40.134112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:40.134112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:40.134112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:20:40.134112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:40.134112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:20:40.134112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:20:40.134112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:40.134112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:20:40.134112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:40.135331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:20:40.135331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:20:40.135331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:40.135331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:20:40.135331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:40.135331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:20:40.135331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:40.135331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:20:40.135331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:20:40.135331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:20:40.135331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:20:40.135331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:20:40.135331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:40.135331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:20:40.136034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:20:40.136034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:20:40.136034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:20:40.136034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:20:40.136034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:40.136034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:40.136034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:20:40.136034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:20:40.136034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:20:40.136034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:20:40.136034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:20:40.136034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:20:40.136034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:20:40.136034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:40.136034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:40.136034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:40.136034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:20:40.136034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:40.136034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:40.136930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:40.136930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:20:40.136930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:40.136930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:20:40.136930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:20:40.136930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:40.136930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:20:40.136930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:40.136930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:20:40.136930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:20:40.136930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:40.136930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:20:40.136930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:40.136930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:20:40.167656 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:20:40.167997 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:20:40.168135 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:20:40.168296 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:20:40.168845 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:20:40.169272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:40.170077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:20:40.170077 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:40.170077 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:20:40.170077 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:20:40.170077 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:40.170077 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:20:40.170077 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:40.170077 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:20:40.170836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:40.170836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:20:40.170836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:20:40.170836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:20:40.170836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:40.170836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:20:40.170836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:40.170836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:20:40.170836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:40.170836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:20:40.170836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:40.170836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:20:40.170836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:40.170836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:20:40.170836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:20:40.171319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:40.171319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:20:40.171319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:40.171319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:20:40.171319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:20:40.171319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:40.171319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:20:40.171319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:40.171319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:20:40.171819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:40.171819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:20:40.171819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:40.171819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:20:40.171819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:20:40.171819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:20:40.171819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:20:40.171819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:20:40.171819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:20:40.171819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:40.171819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:40.172158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:40.172158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:20:40.172158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:40.172158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:40.172158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:40.172158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:20:40.172158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:40.172158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:20:40.172158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:20:40.172158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:40.172158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:20:40.172158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:40.172158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:20:40.172158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:20:40.172541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:40.172541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:20:40.172541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:40.172541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:20:40.172905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:40.172905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:20:40.172905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:20:40.172905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:20:40.172905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:20:40.172905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:20:40.172905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:40.172905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:20:40.172905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:20:40.172905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:20:40.172905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:20:40.172905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:20:40.172905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:40.172905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:40.172905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:20:40.172905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:20:40.172905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:20:40.173423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:20:40.173423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:20:40.173423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:20:40.173423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:40.173423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:40.173423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:40.173423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:20:40.173423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:40.173423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:40.173423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:40.173423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:20:40.173423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:40.173423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:20:40.173423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:20:40.173423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:40.173423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:20:40.173423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:40.173423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:20:40.173423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:20:40.173920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:40.173920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:20:40.173920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:40.173920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:20:40.175543 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:20:40.175813 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:20:40.175920 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:20:40.176064 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:20:40.176319 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:20:40.176548 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:20:40.176688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:40.177163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:20:40.177163 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:40.177163 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:20:40.177163 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:20:40.177163 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:40.177163 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:20:40.177163 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:40.177163 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:20:40.178053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:40.178053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:20:40.178053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:40.178053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:20:40.178053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:20:40.178053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:20:40.178053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:20:40.178053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:40.178053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:40.178053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:40.178053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:20:40.178053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:40.178436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:40.178436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:40.178436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:20:40.178436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:40.178436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:20:40.178436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:20:40.178436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:40.178436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:20:40.178436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:40.178436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:20:40.178436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:20:40.178436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:40.178436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:20:40.178806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:40.178806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:20:40.179046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:40.179046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:20:40.179046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:20:40.179046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:20:40.179046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:20:40.179046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:20:40.179046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:40.179046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:20:40.179046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:20:40.179046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:20:40.179046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:20:40.179046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:20:40.179046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:40.179046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:40.179046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:20:40.179046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:20:40.179521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:20:40.179521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:20:40.179521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:40.179521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:40.179521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:40.179521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:20:40.179521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:40.179521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:40.179521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:40.179521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:20:40.179521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:40.179521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:20:40.179521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:20:40.179521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:40.179521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:20:40.179521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:40.179521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:20:40.179521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:20:40.179521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:40.180024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:20:40.180024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:40.180024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:20:44.872871 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:20:44.873771 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:20:44.873771 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:20:44.875886 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:20:44.905153 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:20:44.905606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:44.906477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:20:44.906477 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:44.906477 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:20:44.906477 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:20:44.906477 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:44.906477 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:20:44.906477 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:44.906477 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:20:44.907491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:44.907491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:20:44.907491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:44.907491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:20:44.907491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:20:44.907491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:20:44.907491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:20:44.907491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:44.907491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:44.907491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:44.907491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:20:44.907491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:44.908134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:44.908134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:44.908134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:20:44.908134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:44.908134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:20:44.908134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:20:44.908134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:44.908134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:20:44.908134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:44.908134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:20:44.908134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:20:44.908134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:44.908134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:20:44.908871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:44.908871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:20:44.908871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:44.908871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:20:44.908871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:20:44.908871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:20:44.908871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:20:44.908871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:20:44.908871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:44.908871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:20:44.908871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:20:44.908871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:20:44.908871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:20:44.908871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:20:44.908871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:44.909652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:44.909652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:20:44.909652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:20:44.909652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:20:44.909652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:20:44.909652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:44.909652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:44.909652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:44.909652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:20:44.909652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:44.909652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:44.909652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:44.909652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:20:44.909652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:44.909652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:20:44.909652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:20:44.909652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:44.909652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:20:44.909652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:44.910932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:20:44.910932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:20:44.910932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:44.910932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:20:44.910932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:44.910932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:20:45.119622 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:20:45.120018 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:20:45.120198 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:20:45.121127 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:20:45.121503 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:20:45.121835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:45.122831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:20:45.122831 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:45.122831 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:20:45.122831 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:20:45.122831 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:45.122831 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:20:45.122831 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:45.122831 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:20:45.123739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:45.123739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:20:45.123739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:45.123739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:20:45.123739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:20:45.123739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:20:45.123739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:20:45.123739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:45.123739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:45.123739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:45.123739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:20:45.123739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:45.124244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:45.124244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:45.124244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:20:45.124244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:45.124244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:20:45.124244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:20:45.124244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:45.124244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:20:45.124244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:45.124244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:20:45.124244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:20:45.124244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:45.124244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:20:45.124685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:45.124685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:20:45.124685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:45.124685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:20:45.124685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:20:45.124685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:20:45.124685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:20:45.124685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:20:45.124685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:45.124685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:20:45.124685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:20:45.124685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:20:45.124685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:20:45.124685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:20:45.124685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:45.125096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:45.125096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:20:45.125096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:20:45.125096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:20:45.125096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:20:45.125096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:45.125096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:45.125096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:45.125096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:20:45.125096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:45.125096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:45.125096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:45.125096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:20:45.125096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:45.125096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:20:45.125096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:20:45.125096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:45.125096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:20:45.125096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:45.125643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:20:45.125643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:20:45.125643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:45.125643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:20:45.125643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:45.125643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:20:45.126232 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:20:45.126561 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:20:45.126651 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:20:45.126773 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:20:45.127223 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:20:45.127539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:45.128328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:20:45.128328 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:45.128328 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:20:45.128328 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:20:45.128328 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:45.128328 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:20:45.128328 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:45.128328 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:20:45.129351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:45.129351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:20:45.129351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:20:45.129351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:20:45.129351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:45.129351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:20:45.129351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:45.129351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:20:45.129351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:45.129351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:20:45.129351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:45.129351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:20:45.129351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:45.129351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:20:45.129351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:20:45.129994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:45.129994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:20:45.129994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:45.129994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:20:45.129994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:20:45.129994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:45.129994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:20:45.129994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:45.129994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:20:45.129994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:45.129994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:20:45.129994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:45.129994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:20:45.129994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:20:45.130446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:20:45.130446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:20:45.130446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:20:45.130446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:20:45.130446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:20:45.130446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:45.130446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:45.130446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:45.130446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:20:45.130446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:45.130446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:45.130446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:45.130446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:20:45.130852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:45.130852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:20:45.130852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:20:45.130852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:45.130852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:20:45.130852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:45.130852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:20:45.130852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:20:45.130852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:45.130852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:20:45.130852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:45.130852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:20:45.131200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:45.131200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:20:45.131200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:20:45.131200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:20:45.131200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:20:45.131200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:20:45.131200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:45.131200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:20:45.131200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:20:45.131200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:20:45.131200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:20:45.131200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:20:45.131200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:45.131200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:45.131200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:20:45.131200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:20:45.131666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:20:45.131666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:20:45.131666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:20:45.131666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:20:45.131666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:20:45.131666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:45.131666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:45.131666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:45.131666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:20:45.131666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:45.131666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:45.131666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:45.131666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:20:45.131666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:45.131666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:20:45.131666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:20:45.131666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:45.131666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:20:45.132178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:45.132178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:20:45.132178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:20:45.132178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:45.132178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:20:45.132178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:45.132178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:20:45.168270 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:20:45.168559 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:20:45.168671 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:20:45.168791 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:20:45.169226 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:20:45.169546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:45.170140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:20:45.170140 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:45.170140 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:20:45.170140 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:20:45.170140 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:45.170140 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:20:45.170140 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:45.170140 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:20:45.170875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:45.170875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:20:45.170875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:20:45.170875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:20:45.170875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:45.170875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:20:45.170875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:45.170875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:20:45.170875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:45.170875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:20:45.170875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:45.170875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:20:45.170875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:45.170875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:20:45.170875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:20:45.171380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:45.171380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:20:45.171380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:45.171380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:20:45.171380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:20:45.171380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:45.171380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:20:45.171380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:45.171380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:20:45.171748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:45.171748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:20:45.171748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:45.171748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:20:45.171748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:20:45.171748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:20:45.171748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:20:45.171748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:20:45.171748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:20:45.171748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:45.171748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:45.172114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:45.172114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:20:45.172114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:45.172114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:45.172114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:45.172114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:20:45.172114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:45.172114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:20:45.172114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:20:45.172114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:45.172114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:20:45.172114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:45.172114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:20:45.172114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:20:45.172500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:45.172500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:20:45.172500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:45.172500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:20:45.172817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:45.172817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:20:45.172817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:20:45.172817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:20:45.172817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:20:45.172817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:20:45.172817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:45.172817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:20:45.172817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:20:45.172817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:20:45.172817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:20:45.172817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:20:45.172817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:45.172817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:45.172817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:20:45.172817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:20:45.172817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:20:45.173293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:20:45.173293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:20:45.173293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:20:45.173293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:45.173293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:45.173293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:45.173293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:20:45.173293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:45.173293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:45.173293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:45.173293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:20:45.173293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:45.173293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:20:45.173293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:20:45.173293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:45.173293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:20:45.173293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:45.173293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:20:45.173293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:20:45.173830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:45.173830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:20:45.173830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:45.173830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:20:45.175966 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:20:45.176256 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:20:45.176333 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:20:45.176543 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:20:45.176944 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:20:45.177340 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:20:45.177644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:45.178377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:20:45.178377 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:45.178377 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:20:45.178377 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:20:45.178377 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:45.178377 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:20:45.178377 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:45.178377 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:20:45.179214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:45.179214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:20:45.179214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:45.179214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:20:45.179214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:20:45.179214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:20:45.179214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:20:45.179214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:45.179214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:45.179214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:45.179214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:20:45.179214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:45.179895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:45.179895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:45.179895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:20:45.179895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:45.179895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:20:45.179895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:20:45.179895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:45.179895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:20:45.179895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:45.179895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:20:45.179895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:20:45.179895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:45.179895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:20:45.180585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:45.180585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:20:45.180585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:45.180585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:20:45.180585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:20:45.180585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:20:45.180585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:20:45.180585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:20:45.180585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:45.180585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:20:45.180585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:20:45.180585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:20:45.180585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:20:45.180585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:20:45.180585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:45.181284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:45.181284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:20:45.181284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:20:45.181284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:20:45.181284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:20:45.181284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:45.181284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:45.181284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:45.181284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:20:45.181284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:45.181284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:45.181284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:45.181284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:20:45.181284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:45.181284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:20:45.181284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:20:45.181284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:45.181284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:20:45.181284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:45.182163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:20:45.182163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:20:45.182163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:45.182163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:20:45.182163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:45.182163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:20:49.902108 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:20:49.906163 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:20:49.906163 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:20:49.906163 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:20:49.940032 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:20:49.940481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:49.942931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:20:49.942931 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:49.942931 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:20:49.942931 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:20:49.942931 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:49.942931 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:20:49.942931 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:49.942931 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:20:49.942931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:49.942931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:20:49.942931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:49.942931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:20:49.942931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:20:49.942931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:20:49.942931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:20:49.943422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:49.943422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:49.943422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:49.943422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:20:49.943422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:49.943422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:49.943422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:49.943422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:20:49.943422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:49.943422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:20:49.943422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:20:49.943422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:49.943422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:20:49.943422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:49.943745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:20:49.943745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:20:49.943745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:49.943745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:20:49.943745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:49.943745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:20:49.943745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:49.943745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:20:49.943745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:20:49.943745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:20:49.943745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:20:49.943745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:20:49.943745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:49.943745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:20:49.944061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:20:49.944061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:20:49.944061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:20:49.944061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:20:49.944061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:49.944061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:49.944061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:20:49.944061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:20:49.944061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:20:49.944061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:20:49.944061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:49.944061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:49.944061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:49.944061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:20:49.944061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:49.944061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:49.944061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:49.944061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:20:49.944061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:49.944061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:20:49.944061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:20:49.944548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:49.944548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:20:49.944548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:49.944548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:20:49.944548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:20:49.944548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:49.944548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:20:49.944548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:49.944548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:20:50.120315 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:20:50.121172 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:20:50.121358 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:20:50.122367 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:20:50.122725 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:20:50.122971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:50.123793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:20:50.123793 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:50.123793 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:20:50.123793 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:20:50.123793 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:50.123793 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:20:50.123793 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:50.123793 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:20:50.124813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:50.124813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:20:50.124813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:50.124813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:20:50.124813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:20:50.124813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:20:50.124813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:20:50.124813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:50.124813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:50.124813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:50.124813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:20:50.124813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:50.125506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:50.125506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:50.125506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:20:50.125506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:50.125506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:20:50.125506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:20:50.125506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:50.125506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:20:50.125506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:50.125506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:20:50.125506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:20:50.125506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:50.125506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:20:50.126127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:50.126127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:20:50.126127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:50.126127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:20:50.126127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:20:50.126127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:20:50.126127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:20:50.126127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:20:50.126127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:50.126127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:20:50.126127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:20:50.126127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:20:50.126127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:20:50.126127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:20:50.126127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:50.126899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:50.126899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:20:50.126899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:20:50.126899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:20:50.126899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:20:50.126899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:50.126899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:50.126899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:50.126899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:20:50.126899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:50.126899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:50.126899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:50.126899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:20:50.126899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:50.126899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:20:50.126899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:20:50.126899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:50.126899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:20:50.126899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:50.127882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:20:50.127882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:20:50.127882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:50.127882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:20:50.127882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:50.127882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:20:50.127882 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:20:50.127882 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:20:50.127882 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:20:50.127882 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:20:50.128433 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:20:50.128433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:50.128638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:20:50.128638 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:50.128638 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:20:50.128638 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:20:50.128638 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:50.128638 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:20:50.128638 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:50.128638 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:20:50.129386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:50.129386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:20:50.129386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:20:50.129386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:20:50.129386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:50.129386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:20:50.129386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:50.129386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:20:50.129386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:50.129386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:20:50.129386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:50.129386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:20:50.129386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:50.129386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:20:50.129386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:20:50.129944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:50.129944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:20:50.129944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:50.129944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:20:50.129944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:20:50.129944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:50.129944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:20:50.129944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:50.129944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:20:50.130242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:50.130242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:20:50.130242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:50.130242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:20:50.130242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:20:50.130242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:20:50.130242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:20:50.130242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:20:50.130242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:20:50.130242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:20:50.130242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:50.130242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:50.130697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:50.130697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:20:50.130697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:50.130697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:50.130697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:50.130697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:20:50.130697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:50.130697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:20:50.130697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:20:50.130697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:50.130697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:20:50.130697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:50.130697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:20:50.130697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:20:50.131109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:50.131109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:20:50.131109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:50.131109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:20:50.131253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:50.131253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:20:50.131253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:20:50.131253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:20:50.131253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:20:50.131253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:20:50.131253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:50.131253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:20:50.131253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:20:50.131253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:20:50.131253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:20:50.131253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:20:50.131253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:50.131253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:50.131253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:20:50.131253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:20:50.131751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:20:50.131751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:20:50.131751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:20:50.131751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:20:50.131751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:20:50.131751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:50.131751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:50.131751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:50.131751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:20:50.131751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:50.131751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:50.131751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:50.131751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:20:50.131751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:50.131751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:20:50.131751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:20:50.131751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:50.131751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:20:50.132470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:50.132470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:20:50.132470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:20:50.132470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:50.132470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:20:50.132470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:50.132470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:20:50.168673 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:20:50.169153 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:20:50.169320 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:20:50.169578 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:20:50.169988 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:20:50.170276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:50.170864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:20:50.170864 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:50.170864 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:20:50.170864 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:20:50.170864 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:50.170864 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:20:50.170864 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:50.170864 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:20:50.171980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:50.171980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:20:50.171980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:20:50.171980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:20:50.171980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:50.171980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:20:50.171980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:50.171980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:20:50.171980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:50.171980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:20:50.171980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:50.171980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:20:50.171980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:50.171980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:20:50.171980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:20:50.172743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:50.172743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:20:50.172743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:50.172743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:20:50.172743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:20:50.172743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:50.172743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:20:50.172743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:50.172743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:20:50.173314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:50.173314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:20:50.173314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:50.173314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:20:50.173314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:20:50.173314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:20:50.173314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:20:50.173314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:20:50.173314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:20:50.173314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:50.173314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:50.173718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:50.173718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:20:50.173718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:50.173718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:50.173718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:50.173718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:20:50.173718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:50.173718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:20:50.173718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:20:50.173718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:50.173718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:20:50.173718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:50.173718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:20:50.173718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:20:50.174088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:50.174088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:20:50.174088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:50.174088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:20:50.174834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:50.174834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:20:50.174834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:20:50.174834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:20:50.174834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:20:50.174834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:20:50.174834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:50.174834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:20:50.174834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:20:50.174834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:20:50.174834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:20:50.174834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:20:50.174834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:50.174834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:50.174834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:20:50.174834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:20:50.174834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:20:50.175305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:20:50.175305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:20:50.175305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:20:50.175305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:50.175305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:50.175305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:50.175305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:20:50.175305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:50.175305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:50.175305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:50.175305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:20:50.175305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:50.175305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:20:50.175305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:20:50.175305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:50.175305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:20:50.175305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:50.175305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:20:50.175305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:20:50.175818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:50.175818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:20:50.175818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:50.175818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:20:50.176413 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:20:50.176714 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:20:50.176821 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:20:50.176968 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:20:50.177353 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:20:50.177849 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:20:50.178187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:50.178968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:20:50.178968 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:50.178968 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:20:50.178968 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:20:50.178968 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:50.178968 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:20:50.178968 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:50.178968 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:20:50.179810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:50.179810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:20:50.179810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:50.179810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:20:50.179810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:20:50.179810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:20:50.179810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:20:50.179810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:50.179810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:50.179810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:50.179810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:20:50.179810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:50.180151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:50.180151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:50.180151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:20:50.180151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:50.180151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:20:50.180151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:20:50.180151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:50.180151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:20:50.180151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:50.180151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:20:50.180151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:20:50.180151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:50.180151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:20:50.180497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:50.180497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:20:50.180789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:50.180789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:20:50.180789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:20:50.180789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:20:50.180789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:20:50.180789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:20:50.180789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:50.180789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:20:50.180789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:20:50.180789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:20:50.180789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:20:50.180789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:20:50.180789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:50.180789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:50.180789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:20:50.180789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:20:50.181211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:20:50.181211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:20:50.181211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:50.181211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:50.181211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:50.181211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:20:50.181211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:50.181211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:50.181211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:50.181211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:20:50.181211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:50.181211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:20:50.181211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:20:50.181211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:50.181211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:20:50.181211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:50.181211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:20:50.181211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:20:50.181211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:50.181757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:20:50.181757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:50.181757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:20:54.905101 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:20:54.906295 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:20:54.906295 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:20:54.907563 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:20:54.939844 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:20:54.940099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:54.940775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:20:54.940775 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:54.940775 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:20:54.940775 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:20:54.940775 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:54.940775 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:20:54.940775 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:54.940775 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:20:54.941540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:54.941540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:20:54.941540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:54.941540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:20:54.941540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:20:54.941540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:20:54.941540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:20:54.941540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:54.941540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:54.941540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:54.941540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:20:54.941540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:54.942312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:54.942312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:54.942312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:20:54.942312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:54.942312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:20:54.942312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:20:54.942312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:54.942312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:20:54.942312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:54.942312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:20:54.942312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:20:54.942312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:54.942312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:20:54.943266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:54.943266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:20:54.943266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:54.943266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:20:54.943266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:20:54.943266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:20:54.943266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:20:54.943266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:20:54.943266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:54.943266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:20:54.943266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:20:54.943266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:20:54.943266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:20:54.943266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:20:54.943266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:54.944158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:54.944158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:20:54.944158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:20:54.944158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:20:54.944158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:20:54.944158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:54.944158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:54.944158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:54.944158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:20:54.944158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:54.944158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:54.944158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:54.944158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:20:54.944158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:54.944158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:20:54.944158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:20:54.944158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:54.944158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:20:54.944158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:54.945051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:20:54.945051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:20:54.945051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:54.945051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:20:54.945051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:54.945051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:20:55.121239 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:20:55.122433 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:20:55.122650 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:20:55.123661 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:20:55.123987 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:20:55.124233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:55.125262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:20:55.125262 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:55.125262 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:20:55.125262 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:20:55.125262 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:55.125262 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:20:55.125262 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:55.125262 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:20:55.128303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:55.128303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:20:55.128303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:55.128303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:20:55.128303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:20:55.128303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:20:55.128303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:20:55.128303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:55.128303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:55.128303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:55.128303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:20:55.128303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:55.129152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:55.129152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:55.129152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:20:55.129152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:55.129152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:20:55.129152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:20:55.129152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:55.129152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:20:55.129152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:55.129152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:20:55.129152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:20:55.129152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:55.129152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:20:55.129856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:55.129856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:20:55.131282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:55.131282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:20:55.131282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:20:55.131282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:20:55.131282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:20:55.131282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:20:55.131282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:55.131282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:20:55.131282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:20:55.131282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:20:55.131282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:20:55.131282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:20:55.131282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:55.131282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:55.131282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:20:55.131282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:20:55.131827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:20:55.131827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:20:55.131827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:55.131827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:55.131827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:55.131827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:20:55.131827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:55.131827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:55.131827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:55.131827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:20:55.131827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:55.131827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:20:55.131827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:20:55.131827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:55.131827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:20:55.131827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:55.131827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:20:55.131827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:20:55.131827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:55.132341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:20:55.132341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:55.132341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:20:55.132341 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:20:55.132341 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:20:55.132341 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:20:55.132599 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:20:55.133061 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:20:55.133348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:55.134011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:20:55.134011 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:55.134011 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:20:55.134011 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:20:55.134011 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:55.134011 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:20:55.134011 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:55.134011 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:20:55.134883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:55.134883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:20:55.134883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:20:55.134883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:20:55.134883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:55.134883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:20:55.134883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:55.134883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:20:55.134883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:55.134883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:20:55.134883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:55.134883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:20:55.134883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:55.134883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:20:55.134883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:20:55.135457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:55.135457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:20:55.135457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:55.135457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:20:55.135457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:20:55.135457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:55.135457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:20:55.135457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:55.135457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:20:55.135737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:55.135737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:20:55.135737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:55.135737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:20:55.135737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:20:55.135737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:20:55.135737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:20:55.135737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:20:55.135737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:20:55.135737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:20:55.135737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:55.135737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:55.136105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:55.136105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:20:55.136105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:55.136105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:55.136105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:55.136105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:20:55.136105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:55.136105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:20:55.136105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:20:55.136105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:55.136105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:20:55.136105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:55.136105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:20:55.136105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:20:55.136583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:55.136583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:20:55.136583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:55.136583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:20:55.136766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:55.136766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:20:55.136766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:20:55.136766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:20:55.136766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:20:55.136766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:20:55.136766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:55.136766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:20:55.136766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:20:55.136766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:20:55.136766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:20:55.136766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:20:55.136766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:55.136766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:55.136766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:20:55.136766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:20:55.137169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:20:55.137169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:20:55.137169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:20:55.137169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:20:55.137169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:20:55.137169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:55.137169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:55.137169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:55.137169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:20:55.137169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:55.137169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:55.137169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:55.137169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:20:55.137169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:55.137169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:20:55.137169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:20:55.137169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:55.137169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:20:55.137952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:55.137952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:20:55.137952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:20:55.137952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:55.137952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:20:55.137952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:55.137952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:20:55.169484 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:20:55.169865 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:20:55.170052 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:20:55.170255 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:20:55.170794 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:20:55.171161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:55.171989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:20:55.171989 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:55.171989 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:20:55.171989 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:20:55.171989 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:55.171989 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:20:55.171989 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:55.171989 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:20:55.173398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:55.173398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:20:55.173398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:20:55.173398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:20:55.173398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:55.173398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:20:55.173398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:55.173398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:20:55.173398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:55.173398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:20:55.173398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:55.173398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:20:55.173398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:55.173398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:20:55.173398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:20:55.173777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:55.173777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:20:55.173777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:55.173777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:20:55.173777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:20:55.173777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:55.173777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:20:55.173777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:55.173777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:20:55.175044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:55.175044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:20:55.175044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:55.175044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:20:55.175044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:20:55.175044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:20:55.175044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:20:55.175044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:20:55.175044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:20:55.175044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:55.175044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:55.175438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:55.175438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:20:55.175438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:55.175438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:55.175438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:55.175438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:20:55.175438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:55.175438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:20:55.175438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:20:55.175438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:55.175438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:20:55.175438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:55.175438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:20:55.175438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:20:55.175802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:55.175802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:20:55.175802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:55.175802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:20:55.177122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:55.177122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:20:55.177122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:20:55.177122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:20:55.177122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:20:55.177122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:20:55.177122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:55.177122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:20:55.177122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:20:55.177122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:20:55.177122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:20:55.177122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:20:55.177122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:55.177122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:55.177122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:20:55.177122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:20:55.177122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:20:55.177631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:20:55.177631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:20:55.177631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:20:55.177631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:55.177631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:55.177631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:55.177631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:20:55.177631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:55.177631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:55.177631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:55.177631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:20:55.177631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:55.177631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:20:55.177631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:20:55.177631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:55.177631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:20:55.177631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:55.177631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:20:55.177631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:20:55.178234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:55.178234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:20:55.178234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:55.178234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:20:55.178234 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:20:55.178234 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:20:55.178234 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:20:55.178234 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:20:55.178604 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:20:55.178951 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:20:55.179210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:55.179934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:20:55.179934 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:55.179934 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:20:55.179934 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:20:55.179934 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:55.179934 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:20:55.179934 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:55.179934 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:20:55.180789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:55.180789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:20:55.180789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:55.180789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:20:55.180789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:20:55.180789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:20:55.180789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:20:55.180789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:55.180789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:55.180789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:55.180789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:20:55.180789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:55.181199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:55.181199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:55.181199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:20:55.181199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:55.181199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:20:55.181199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:20:55.181199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:55.181199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:20:55.181199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:55.181199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:20:55.181199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:20:55.181199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:55.181199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:20:55.181591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:55.181591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:20:55.181927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:55.181927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:20:55.181927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:20:55.181927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:20:55.181927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:20:55.181927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:20:55.181927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:55.181927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:20:55.181927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:20:55.181927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:20:55.181927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:20:55.181927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:20:55.181927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:55.181927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:55.181927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:20:55.181927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:20:55.182580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:20:55.182580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:20:55.182580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:55.182580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:55.182580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:55.182580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:20:55.182580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:55.182580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:55.182580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:55.182580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:20:55.182580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:55.182580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:20:55.182580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:20:55.182580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:55.182580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:20:55.182580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:55.182580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:20:55.182580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:20:55.182580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:55.183253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:20:55.183253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:55.183253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:20:59.905910 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:20:59.907663 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:20:59.907663 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:20:59.910457 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:20:59.939832 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:20:59.940238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:59.942445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:20:59.942445 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:59.942445 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:20:59.942445 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:20:59.942445 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:59.942445 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:20:59.942445 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:59.942445 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:20:59.942445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:59.942445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:20:59.942445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:59.942445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:20:59.942445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:20:59.942445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:20:59.942445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:20:59.942920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:59.942920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:59.942920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:59.942920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:20:59.942920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:59.942920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:20:59.942920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:59.942920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:20:59.942920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:59.942920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:20:59.942920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:20:59.942920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:59.942920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:20:59.942920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:59.943240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:20:59.943240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:20:59.943240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:59.943240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:20:59.943240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:59.943240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:20:59.943240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:59.943240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:20:59.943240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:20:59.943240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:20:59.943240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:20:59.943240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:20:59.943240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:59.943240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:20:59.943551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:20:59.943551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:20:59.943551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:20:59.943551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:20:59.943551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:20:59.943551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:20:59.943551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:20:59.943551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:20:59.943551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:20:59.943551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:20:59.943551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:59.943551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:59.943551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:20:59.943551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:20:59.943551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:20:59.943551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:20:59.943551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:20:59.943551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:20:59.943551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:20:59.943551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:20:59.943551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:20:59.943988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:20:59.943988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:20:59.943988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:20:59.943988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:20:59.943988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:20:59.943988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:20:59.943988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:20:59.943988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:20:59.943988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:21:00.122364 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:21:00.122964 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:21:00.123175 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:21:00.132137 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:21:00.137102 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:21:00.137464 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:21:00.137709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:00.138895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:21:00.138895 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:00.138895 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:21:00.138895 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:21:00.138895 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:00.138895 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:21:00.138895 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:00.138895 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:21:00.141865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.017 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:00.141865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:21:00.141865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:00.141865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:21:00.141865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:21:00.141865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:21:00.141865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:21:00.141865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:00.141865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:00.141865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:00.141865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:21:00.141865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:00.142448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:00.142448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:00.142448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:21:00.142448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:00.142448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:21:00.142448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:21:00.142448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:00.142448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:21:00.142448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:00.142448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:21:00.142448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:21:00.142448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:00.142448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:21:00.142682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:00.142682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:21:00.144861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:00.144861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:21:00.144861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:21:00.144861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:21:00.144861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:21:00.144861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:21:00.144861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:00.144861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:21:00.144861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:21:00.144861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:21:00.144861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:21:00.144861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:21:00.144861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:00.144861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:00.144861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:21:00.144861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:21:00.145169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:21:00.145169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:21:00.145169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:00.145169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:00.145169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:00.145169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:21:00.145169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:00.145169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:00.145169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:00.145169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:21:00.145169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:00.145169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:21:00.145169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:21:00.145169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:00.145169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:21:00.145169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:00.145169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:21:00.145169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:21:00.145169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:00.145539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:21:00.145539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:00.145539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:21:00.145539 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:21:00.145539 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:21:00.145539 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:21:00.145947 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:21:00.146185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:00.146903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:21:00.146903 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:00.146903 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:21:00.146903 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:21:00.146903 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:00.146903 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:21:00.146903 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:00.146903 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:21:00.147715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:00.147715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:21:00.147715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:21:00.147715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:21:00.147715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:00.147715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:21:00.147715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:00.147715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:21:00.147715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:00.147715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:21:00.147715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:00.147715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:21:00.147715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:00.147715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:21:00.147715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:21:00.148445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:00.148445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:21:00.148445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:00.148445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:21:00.148445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:21:00.148445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:00.148445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:21:00.148445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:00.148445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:21:00.148445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:00.148445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:21:00.148445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:00.148445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:21:00.148445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:21:00.148717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:21:00.148717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:21:00.148717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:21:00.148717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:21:00.148717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:21:00.148717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:00.148717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:00.148717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:00.148717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:21:00.148717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:00.148717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:00.148717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:00.148717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:21:00.148962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:00.148962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:21:00.148962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:21:00.148962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:00.148962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:21:00.148962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:00.148962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:21:00.148962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:21:00.148962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:00.148962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:21:00.148962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:00.148962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:21:00.149185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:00.149185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:21:00.149185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:21:00.149185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:21:00.149185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:21:00.149185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:21:00.149185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:00.149185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:21:00.149185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:21:00.149185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:21:00.149185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:21:00.149185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:21:00.149185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:00.149185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:00.149185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:21:00.149185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:21:00.149485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:21:00.149485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:21:00.149485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:21:00.149485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:21:00.149485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:21:00.149485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:00.149485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:00.149485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:00.149485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:21:00.149485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:00.149485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:00.149485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:00.149485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:21:00.149485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:00.149485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:21:00.149485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:21:00.149485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:00.149485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:21:00.149819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:00.149819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:21:00.149819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:21:00.149819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:00.149819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:21:00.149819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:00.149819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:21:00.169984 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:21:00.170741 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:21:00.170907 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:21:00.171020 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:21:00.171590 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:21:00.171964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:00.172741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:21:00.172741 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:00.172741 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:21:00.172741 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:21:00.172741 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:00.172741 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:21:00.172741 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:00.172741 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:21:00.173530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:00.173530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:21:00.173530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:21:00.173530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:21:00.173530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:00.173530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:21:00.173530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:00.173530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:21:00.173530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:00.173530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:21:00.173530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:00.173530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:21:00.173530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:00.173530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:21:00.173530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:21:00.174160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:00.174160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:21:00.174160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:00.174160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:21:00.174160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:21:00.174160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:00.174160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:21:00.174160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:00.174160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:21:00.174552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:00.174552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:21:00.174552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:00.174552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:21:00.174552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:21:00.174552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:21:00.174552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:21:00.174552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:21:00.174552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:21:00.174552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:00.174552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:00.175027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:00.175027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:21:00.175027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:00.175027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:00.175027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:00.175027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:21:00.175027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:00.175027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:21:00.175027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:21:00.175027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:00.175027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:21:00.175027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:00.175027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:21:00.175027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:21:00.175613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:00.175613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:21:00.175613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:00.175613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:21:00.175613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:00.175613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:21:00.175613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:21:00.175613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:21:00.175613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:21:00.175613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:21:00.175613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:00.175613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:21:00.175613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:21:00.175613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:21:00.176202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:21:00.176202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:21:00.176202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:00.176202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:00.176202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:21:00.176202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:21:00.176202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:21:00.176202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:21:00.176202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:21:00.176202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:21:00.176202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:00.176202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:00.176202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:00.176202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:21:00.176202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:00.176202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:00.176202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:00.176202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:21:00.176202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:00.178728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:21:00.178728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:21:00.178728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:00.178728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:21:00.178728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:00.178728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:21:00.178728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:21:00.178728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:00.178728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:21:00.178728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:00.178728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:21:00.178728 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:21:00.178728 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:21:00.180100 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:21:00.180100 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:21:00.180100 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:21:00.180100 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:21:00.180100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:00.180664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:21:00.180664 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:00.180664 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:21:00.180664 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:21:00.180664 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:00.180664 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:21:00.180664 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:00.180664 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:21:00.181827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:00.181827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:21:00.181827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:00.181827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:21:00.181827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:21:00.181827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:21:00.181827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:21:00.181827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:00.181827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:00.181827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:00.181827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:21:00.181827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:00.182090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:00.182090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:00.182090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:21:00.182090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:00.182090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:21:00.182090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:21:00.182090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:00.182090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:21:00.182090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:00.182090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:21:00.182090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:21:00.182090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:00.182090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:21:00.182309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:00.182309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:21:00.182834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:00.182834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:21:00.182834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:21:00.182834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:21:00.182834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:21:00.182834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:21:00.182834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:00.182834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:21:00.182834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:21:00.182834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:21:00.182834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:21:00.182834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:21:00.182834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:00.182834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:00.182834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:21:00.182834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:21:00.183134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:21:00.183134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:21:00.183134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:00.183134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:00.183134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:00.183134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:21:00.183134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:00.183134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:00.183134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:00.183134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:21:00.183134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:00.183134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:21:00.183134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:21:00.183134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:00.183134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:21:00.183134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:00.183134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:21:00.183134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:21:00.183134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:00.183494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:21:00.183494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:00.183494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:21:04.907309 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:21:04.908268 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:21:04.908268 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:21:04.909453 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:21:04.940631 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:21:04.941060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:04.941917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:21:04.941917 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:04.941917 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:21:04.941917 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:21:04.941917 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:04.941917 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:21:04.941917 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:04.941917 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:21:04.942670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:04.942670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:21:04.942670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:04.942670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:21:04.942670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:21:04.942670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:21:04.942670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:21:04.942670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:04.942670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:04.942670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:04.942670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:21:04.942670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:04.943333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:04.943333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:04.943333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:21:04.943333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:04.943333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:21:04.943333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:21:04.943333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:04.943333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:21:04.943333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:04.943333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:21:04.943333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:21:04.943333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:04.943333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:21:04.944017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:04.944017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:21:04.944017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:04.944017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:21:04.944017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:21:04.944017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:21:04.944017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:21:04.944017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:21:04.944017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:04.944017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:21:04.944017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:21:04.944017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:21:04.944017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:21:04.944017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:21:04.944017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:04.944849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:04.944849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:21:04.944849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:21:04.944849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:21:04.944849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:21:04.944849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:04.944849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:04.944849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:04.944849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:21:04.944849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:04.944849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:04.944849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:04.944849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:21:04.944849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:04.944849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:21:04.944849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:21:04.944849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:04.944849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:21:04.944849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:04.945821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:21:04.945821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:21:04.945821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:04.945821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:21:04.945821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:04.945821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:21:05.123081 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:21:05.123663 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:21:05.123851 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:21:05.125035 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:21:05.125284 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:21:05.125468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:05.126187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:21:05.126187 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:05.126187 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:21:05.126187 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:21:05.126187 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:05.126187 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:21:05.126187 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:05.126187 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:21:05.127606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:05.127606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:21:05.127606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:05.127606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:21:05.127606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:21:05.127606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:21:05.127606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:21:05.127606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:05.127606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:05.127606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:05.127606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:21:05.127606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:05.128208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:05.128208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:05.128208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:21:05.128208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:05.128208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:21:05.128208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:21:05.128208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:05.128208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:21:05.128208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:05.128208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:21:05.128208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:21:05.128208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:05.128208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:21:05.128818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:05.128818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:21:05.128818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:05.128818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:21:05.128818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:21:05.128818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:21:05.128818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:21:05.128818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:21:05.128818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:05.128818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:21:05.128818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:21:05.128818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:21:05.128818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:21:05.128818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:21:05.128818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:05.129559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:05.129559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:21:05.129559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:21:05.129559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:21:05.129559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:21:05.129559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:05.129559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:05.129559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:05.129559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:21:05.129559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:05.129559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:05.129559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:05.129559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:21:05.129559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:05.129559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:21:05.129559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:21:05.129559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:05.129559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:21:05.129559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:05.130404 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:21:05.130404 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:21:05.130404 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:05.130404 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:21:05.130404 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:05.130404 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:21:05.145539 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:21:05.145779 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:21:05.145927 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:21:05.146153 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:21:05.146929 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:21:05.147268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:05.148198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:21:05.148198 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:05.148198 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:21:05.148198 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:21:05.148198 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:05.148198 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:21:05.148198 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:05.148198 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:21:05.149345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:05.149345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:21:05.149345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:21:05.149345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:21:05.149345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:05.149345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:21:05.149345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:05.149345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:21:05.149345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:05.149345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:21:05.149345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:05.149345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:21:05.149345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:05.149345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:21:05.149345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:21:05.150432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:05.150432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:21:05.150432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:05.150432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:21:05.150432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:21:05.150432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:05.150432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:21:05.150432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:05.150432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:21:05.150907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:05.150907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:21:05.150907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:05.150907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:21:05.150907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:21:05.150907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:21:05.150907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:21:05.150907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:21:05.150907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:21:05.150907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:21:05.150907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:05.150907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:05.151567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:05.151567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:21:05.151567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:05.151567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:05.151567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:05.151567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:21:05.151567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:05.151567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:21:05.151567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:21:05.151567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:05.151567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:21:05.151567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:05.151567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:21:05.151567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:21:05.152218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:05.152218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:21:05.152218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:05.152218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:21:05.152218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:05.152218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:21:05.152218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:21:05.152218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:21:05.152218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:21:05.152218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:21:05.152218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:05.152218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:21:05.152218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:21:05.152218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:21:05.152928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:21:05.152928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:21:05.152928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:05.152928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:05.152928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:21:05.152928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:21:05.152928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:21:05.152928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:21:05.152928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:21:05.152928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:21:05.152928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:21:05.152928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:05.152928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:05.152928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:05.152928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:21:05.152928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:05.152928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:05.152928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:05.152928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:21:05.153858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:05.153858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:21:05.153858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:21:05.153858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:05.153858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:21:05.153858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:05.153858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:21:05.153858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:21:05.153858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:05.153858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:21:05.153858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:05.153858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:21:05.170792 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:21:05.171079 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:21:05.171250 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:21:05.171464 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:21:05.171957 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:21:05.172306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:05.173096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:21:05.173096 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:05.173096 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:21:05.173096 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:21:05.173096 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:05.173096 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:21:05.173096 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:05.173096 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:21:05.174255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:05.174255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:21:05.174255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:21:05.174255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:21:05.174255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:05.174255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:21:05.174255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:05.174255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:21:05.174255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:05.174255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:21:05.174255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:05.174255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:21:05.174255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:05.174255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:21:05.174255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:21:05.175014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:05.175014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:21:05.175014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:05.175014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:21:05.175014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:21:05.175014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:05.175014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:21:05.175014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:05.175014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:21:05.175626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:05.175626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:21:05.175626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:05.175626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:21:05.175626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:21:05.175626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:21:05.175626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:21:05.175626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:21:05.175626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:21:05.175626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:05.175626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:05.176204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:05.176204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:21:05.176204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:05.176204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:05.176204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:05.176204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:21:05.176204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:05.176204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:21:05.176204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:21:05.176204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:05.176204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:21:05.176204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:05.176204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:21:05.176204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:21:05.176870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:05.176870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:21:05.176870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:05.176870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:21:05.177287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:05.177287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:21:05.177287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:21:05.177287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:21:05.177287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:21:05.177287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:21:05.177287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:05.177287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:21:05.177287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:21:05.177287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:21:05.177287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:21:05.177287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:21:05.177287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:05.177287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:05.177287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:21:05.177287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:21:05.177287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:21:05.178160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:21:05.178160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:21:05.178160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:21:05.178160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:05.178160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:05.178160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:05.178160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:21:05.178160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:05.178160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:05.178160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:05.178160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:21:05.178160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:05.178160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:21:05.178160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:21:05.178160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:05.178160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:21:05.178160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:05.178160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:21:05.178160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:21:05.179048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:05.179048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:21:05.179048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:05.179048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:21:05.179048 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:21:05.179048 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:21:05.179048 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:21:05.179451 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:21:05.179565 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:21:05.179894 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:21:05.180105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:05.180805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:21:05.180805 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:05.180805 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:21:05.180805 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:21:05.180805 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:05.180805 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:21:05.180805 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:05.180805 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:21:05.181686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:05.181686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:21:05.181686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:05.181686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:21:05.181686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:21:05.181686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:21:05.181686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:21:05.181686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:05.181686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:05.181686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:05.181686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:21:05.181686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:05.182053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:05.182053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:05.182053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:21:05.182053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:05.182053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:21:05.182053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:21:05.182053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:05.182053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:21:05.182053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:05.182053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:21:05.182053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:21:05.182053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:05.182053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:21:05.182442 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:05.182442 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:21:05.182657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:05.182657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:21:05.182657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:21:05.182657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:21:05.182657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:21:05.182657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:21:05.182657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:05.182657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:21:05.182657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:21:05.182657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:21:05.182657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:21:05.182657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:21:05.182657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:05.182657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:05.182657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:21:05.182657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:21:05.183110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:21:05.183110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:21:05.183110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:05.183110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:05.183110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:05.183110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:21:05.183110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:05.183110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:05.183110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:05.183110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:21:05.183110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:05.183110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:21:05.183110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:21:05.183110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:05.183110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:21:05.183110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:05.183110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:21:05.183110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:21:05.183110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:05.183678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:21:05.183678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:05.183678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:21:09.907923 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:21:09.908456 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:21:09.908551 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:21:09.909396 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:21:09.950496 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:21:09.950824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:09.951450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:21:09.951450 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:09.951450 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:21:09.951450 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:21:09.951450 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:09.951450 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:21:09.951450 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:09.951450 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:21:09.952743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:09.952743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:21:09.952743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:09.952743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:21:09.952743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:21:09.952743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:21:09.952743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:21:09.952743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:09.952743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:09.952743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:09.952743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:21:09.952743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:09.953578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:09.953578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:09.953578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:21:09.953578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:09.953578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:21:09.953578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:21:09.953578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:09.953578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:21:09.953578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:09.953578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:21:09.953578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:21:09.953578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:09.953578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:21:09.954322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:09.954322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:21:09.954322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:09.954322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:21:09.954322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:21:09.954322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:21:09.954322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:21:09.954322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:21:09.954322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:09.954322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:21:09.954322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:21:09.954322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:21:09.954322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:21:09.954322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:21:09.954322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:09.955387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:09.955387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:21:09.955387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:21:09.955387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:21:09.955387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:21:09.955387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:09.955387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:09.955387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:09.955387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:21:09.955387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:09.955387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:09.955387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:09.955387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:21:09.955387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:09.955387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:21:09.955387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:21:09.955387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:09.955387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:21:09.955387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:09.956534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:21:09.956534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:21:09.956534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:09.956534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:21:09.956534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:09.956534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:21:10.123701 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:21:10.124254 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:21:10.124485 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:21:10.125684 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:21:10.126081 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:21:10.126331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:10.127292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:21:10.127292 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:10.127292 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:21:10.127292 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:21:10.127292 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:10.127292 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:21:10.127292 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:10.127292 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:21:10.129066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:10.129066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:21:10.129066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:10.129066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:21:10.129066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:21:10.129066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:21:10.129066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:21:10.129066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:10.129066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:10.129066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:10.129066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:21:10.129066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:10.129712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:10.129712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:10.129712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:21:10.129712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:10.129712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:21:10.129712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:21:10.129712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:10.129712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:21:10.129712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:10.129712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:21:10.129712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:21:10.129712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:10.129712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:21:10.130435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:10.130435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:21:10.130435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:10.130435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:21:10.130435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:21:10.130435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:21:10.130435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:21:10.130435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:21:10.130435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:10.130435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:21:10.130435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:21:10.130435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:21:10.130435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:21:10.130435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:21:10.130435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:10.131281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:10.131281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:21:10.131281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:21:10.131281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:21:10.131281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:21:10.131281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:10.131281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:10.131281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:10.131281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:21:10.131281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:10.131281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:10.131281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:10.131281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:21:10.131281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:10.131281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:21:10.131281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:21:10.131281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:10.131281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:21:10.131281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:10.132323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:21:10.132323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:21:10.132323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:10.132323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:21:10.132323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:10.132323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:21:10.145937 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:21:10.146286 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:21:10.146466 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:21:10.146681 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:21:10.147442 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:21:10.147849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:10.148802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:21:10.148802 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:10.148802 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:21:10.148802 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:21:10.148802 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:10.148802 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:21:10.148802 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:10.148802 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:21:10.151046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:10.151046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:21:10.151046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:21:10.151046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:21:10.151046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:10.151046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:21:10.151046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:10.151046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:21:10.151046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:10.151046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:21:10.151046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:10.151046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:21:10.151046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:10.151046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:21:10.151046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:21:10.151500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:10.151500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:21:10.151500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:10.151500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:21:10.151500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:21:10.151500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:10.151500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:21:10.151500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:10.151500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:21:10.152387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:10.152387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:21:10.152387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:10.152387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:21:10.152387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:21:10.152387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:21:10.152387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:21:10.152387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:21:10.152387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:21:10.152387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:21:10.152387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:10.152387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:10.152836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:10.152836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:21:10.152836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:10.152836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:10.152836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:10.152836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:21:10.152836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:10.152836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:21:10.152836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:21:10.152836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:10.152836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:21:10.152836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:10.152836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:21:10.152836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:21:10.153248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:10.153248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:21:10.153248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:10.153248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:21:10.155241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:10.155241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:21:10.155241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:21:10.155241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:21:10.155241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:21:10.155241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:21:10.155241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:10.155241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:21:10.155241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:21:10.155241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:21:10.155241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:21:10.155241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:21:10.155241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:10.155241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:10.155241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:21:10.155241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:21:10.155831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:21:10.155831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:21:10.155831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:21:10.155831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:21:10.155831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:21:10.155831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:10.155831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:10.155831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:10.155831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:21:10.155831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:10.155831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:10.155831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:10.155831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:21:10.155831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:10.155831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:21:10.155831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:21:10.155831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:10.155831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:21:10.156431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:10.156431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:21:10.156431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:21:10.156431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:10.156431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:21:10.156431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:10.156431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:21:10.171309 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:21:10.171796 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:21:10.171959 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:21:10.172178 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:21:10.172841 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:21:10.173301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:10.174355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:21:10.174355 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:10.174355 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:21:10.174355 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:21:10.174355 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:10.174355 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:21:10.174355 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:10.174355 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:21:10.175733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:10.175733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:21:10.175733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:21:10.175733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:21:10.175733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:10.175733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:21:10.175733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:10.175733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:21:10.175733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:10.175733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:21:10.175733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:10.175733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:21:10.175733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:10.175733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:21:10.175733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:21:10.176485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:10.176485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:21:10.176485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:10.176485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:21:10.176485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:21:10.176485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:10.176485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:21:10.176485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:10.176485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:21:10.177274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:10.177274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:21:10.177274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:10.177274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:21:10.177274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:21:10.177274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:21:10.177274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:21:10.177274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:21:10.177274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:21:10.177274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:10.177274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:10.178035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:10.178035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:21:10.178035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:10.178035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:10.178035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:10.178035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:21:10.178035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:10.178035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:21:10.178035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:21:10.178035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:10.178035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:21:10.178035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:10.178035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:21:10.178035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:21:10.178820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:10.178820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:21:10.178820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:10.178820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:21:10.179173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:10.179173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:21:10.179173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:21:10.179173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:21:10.179173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:21:10.179173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:21:10.179173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:10.179173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:21:10.179173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:21:10.179173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:21:10.179173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:21:10.179173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:21:10.179173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:10.179173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:10.179173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:21:10.179173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:21:10.179173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:21:10.180217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:21:10.180217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:21:10.180217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:21:10.180217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:10.180217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:10.180217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:10.180217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:21:10.180217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:10.180217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:10.180217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:10.180217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:21:10.180217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:10.180217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:21:10.180217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:21:10.180217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:10.180217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:21:10.180217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:10.180217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:21:10.180217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:21:10.180821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:10.180821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:21:10.180821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:10.180821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:21:10.180821 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:21:10.180821 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:21:10.180821 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:21:10.180821 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:21:10.181121 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:21:10.181121 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:21:10.181121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:10.181494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:21:10.181494 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:10.181494 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:21:10.181494 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:21:10.181494 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:10.181494 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:21:10.181494 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:10.181494 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:21:10.182302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:10.182302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:21:10.182302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:10.182302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:21:10.182302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:21:10.182302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:21:10.182302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:21:10.182302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:10.182302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:10.182302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:10.182302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:21:10.182302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:10.182705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:10.182705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:10.182705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:21:10.182705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:10.182705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:21:10.182705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:21:10.182705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:10.182705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:21:10.182705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:10.182705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:21:10.182705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:21:10.182705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:10.182705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:21:10.183072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:10.183072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:21:10.183263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:10.183263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:21:10.183263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:21:10.183263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:21:10.183263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:21:10.183263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:21:10.183263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:10.183263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:21:10.183263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:21:10.183263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:21:10.183263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:21:10.183263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:21:10.183263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:10.183263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:10.183263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:21:10.183263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:21:10.183781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:21:10.183781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:21:10.183781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:10.183781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:10.183781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:10.183781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:21:10.183781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:10.183781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:10.183781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:10.183781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:21:10.183781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:10.183781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:21:10.183781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:21:10.183781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:10.183781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:21:10.183781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:10.183781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:21:10.183781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:21:10.183781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:10.184326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:21:10.184326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:10.184326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:21:14.908636 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:21:14.909728 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:21:14.909728 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:21:14.910476 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:21:14.945150 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:21:14.945647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:14.946488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:21:14.946488 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:14.946488 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:21:14.946488 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:21:14.946488 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:14.946488 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:21:14.946488 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:14.946488 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:21:14.947207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:14.947207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:21:14.947207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:14.947207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:21:14.947207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:21:14.947207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:21:14.947207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:21:14.947207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:14.947207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:14.947207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:14.947207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:21:14.947207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:14.948925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:14.948925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:14.948925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:21:14.948925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:14.948925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:21:14.948925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:21:14.948925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:14.948925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:21:14.948925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:14.948925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:21:14.948925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:21:14.948925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:14.948925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:21:14.949701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:14.949701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:21:14.949701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:14.949701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:21:14.949701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:21:14.949701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:21:14.949701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:21:14.949701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:21:14.949701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:14.949701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:21:14.949701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:21:14.949701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:21:14.949701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:21:14.949701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:21:14.949701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:14.950655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:14.950655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:21:14.950655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:21:14.950655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:21:14.950655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:21:14.950655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:14.950655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:14.950655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:14.950655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:21:14.950655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:14.950655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:14.950655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:14.950655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:21:14.950655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:14.950655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:21:14.950655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:21:14.950655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:14.950655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:21:14.950655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:14.952662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:21:14.952662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:21:14.952662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:14.952662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:21:14.952662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:14.952662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:21:15.124270 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:21:15.124833 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:21:15.125057 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:21:15.126035 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:21:15.126383 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:21:15.126661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:15.127656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:21:15.127656 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:15.127656 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:21:15.127656 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:21:15.127656 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:15.127656 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:21:15.127656 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:15.127656 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:21:15.129088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:15.129088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:21:15.129088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:15.129088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:21:15.129088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:21:15.129088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:21:15.129088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:21:15.129088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:15.129088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:15.129088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:15.129088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:21:15.129088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:15.129777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:15.129777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:15.129777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:21:15.129777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:15.129777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:21:15.129777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:21:15.129777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:15.129777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:21:15.129777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:15.129777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:21:15.129777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:21:15.129777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:15.129777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:21:15.130122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:15.130122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:21:15.130731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:15.130731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:21:15.130731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:21:15.130731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:21:15.130731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:21:15.130731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:21:15.130731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:15.130731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:21:15.130731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:21:15.130731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:21:15.130731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:21:15.130731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:21:15.130731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:15.130731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:15.130731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:21:15.130731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:21:15.131177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:21:15.131177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:21:15.131177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:15.131177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:15.131177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:15.131177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:21:15.131177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:15.131177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:15.131177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:15.131177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:21:15.131177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:15.131177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:21:15.131177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:21:15.131177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:15.131177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:21:15.131177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:15.131177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:21:15.131177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:21:15.131177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:15.132206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:21:15.132206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:15.132206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:21:15.146672 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:21:15.147129 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:21:15.147512 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:21:15.147669 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:21:15.148074 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:21:15.148358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:15.149164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:21:15.149164 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:15.149164 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:21:15.149164 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:21:15.149164 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:15.149164 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:21:15.149164 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:15.149164 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:21:15.150360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:15.150360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:21:15.150360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:21:15.150360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:21:15.150360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:15.150360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:21:15.150360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:15.150360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:21:15.150360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:15.150360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:21:15.150360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:15.150360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:21:15.150360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:15.150360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:21:15.150360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:21:15.151262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:15.151262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:21:15.151262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:15.151262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:21:15.151262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:21:15.151262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:15.151262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:21:15.151262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:15.151262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:21:15.152045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:15.152045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:21:15.152045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:15.152045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:21:15.152045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:21:15.152045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:21:15.152045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:21:15.152045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:21:15.152045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:21:15.152045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:21:15.152045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:15.152045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:15.153187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:15.153187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:21:15.153187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:15.153187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:15.153187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:15.153187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:21:15.153187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:15.153187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:21:15.153187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:21:15.153187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:15.153187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:21:15.153187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:15.153187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:21:15.153187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:21:15.154446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:15.154446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:21:15.154446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:15.154446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:21:15.154446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:15.154446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:21:15.154446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:21:15.154446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:21:15.154446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:21:15.154446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:21:15.154446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:15.154446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:21:15.154446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:21:15.154446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:21:15.155068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:21:15.155068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:21:15.155068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:15.155068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:15.155068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:21:15.155068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:21:15.155068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:21:15.155068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:21:15.155068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:21:15.155068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:21:15.155068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:21:15.155068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:15.155068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:15.155068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:15.155068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:21:15.155068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:15.155068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:15.155068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:15.155068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:21:15.155951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:15.155951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:21:15.155951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:21:15.155951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:15.155951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:21:15.155951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:15.155951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:21:15.155951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:21:15.155951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:15.155951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:21:15.155951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:15.155951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:21:15.172047 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:21:15.172524 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:21:15.172712 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:21:15.172957 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:21:15.173518 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:21:15.173975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:15.174823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:21:15.174823 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:15.174823 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:21:15.174823 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:21:15.174823 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:15.174823 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:21:15.174823 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:15.174823 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:21:15.176148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:15.176148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:21:15.176148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:21:15.176148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:21:15.176148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:15.176148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:21:15.176148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:15.176148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:21:15.176148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:15.176148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:21:15.176148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:15.176148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:21:15.176148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:15.176148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:21:15.176148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:21:15.177113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:15.177113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:21:15.177113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:15.177113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:21:15.177113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:21:15.177113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:15.177113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:21:15.177113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:15.177113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:21:15.178087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:15.178087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:21:15.178087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:15.178087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:21:15.178087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:21:15.178087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:21:15.178087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:21:15.178087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:21:15.178087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:21:15.178087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:15.178087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:15.178447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:15.178447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:21:15.178447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:15.178447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:15.178447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:15.178447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:21:15.178447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:15.178447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:21:15.178447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:21:15.178447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:15.178447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:21:15.178447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:15.178447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:21:15.178447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:21:15.178807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:15.178807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:21:15.178807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:15.178807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:21:15.179940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:15.179940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:21:15.179940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:21:15.179940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:21:15.179940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:21:15.179940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:21:15.179940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:15.179940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:21:15.179940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:21:15.179940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:21:15.179940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:21:15.179940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:21:15.179940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:15.179940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:15.179940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:21:15.179940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:21:15.179940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:21:15.180435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:21:15.180435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:21:15.180435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:21:15.180435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:15.180435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:15.180435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:15.180435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:21:15.180435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:15.180435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:15.180435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:15.180435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:21:15.180435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:15.180435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:21:15.180435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:21:15.180435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:15.180435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:21:15.180435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:15.180435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:21:15.180435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:21:15.180908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:15.180908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:21:15.180908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:15.180908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:21:15.180908 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:21:15.180908 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:21:15.180908 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:21:15.181201 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:21:15.181690 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:21:15.182040 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:21:15.182270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:15.183076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:21:15.183076 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:15.183076 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:21:15.183076 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:21:15.183076 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:15.183076 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:21:15.183076 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:15.183076 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:21:15.184431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:15.184431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:21:15.184431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:15.184431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:21:15.184431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:21:15.184431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:21:15.184431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:21:15.184431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:15.184431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:15.184431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:15.184431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:21:15.184431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:15.185065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:15.185065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:15.185065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:21:15.185065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:15.185065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:21:15.185065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:21:15.185065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:15.185065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:21:15.185065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:15.185065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:21:15.185065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:21:15.185065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:15.185065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:21:15.185386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:15.185386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:21:15.185386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:15.185386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:21:15.185386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:21:15.185386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:21:15.185386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:21:15.185386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:21:15.185386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:15.185386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:21:15.185386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:21:15.185386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:21:15.185386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:21:15.185386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:21:15.185386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:15.185826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:15.185826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:21:15.185826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:21:15.185826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:21:15.185826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:21:15.185826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:15.185826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:15.185826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:15.185826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:21:15.185826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:15.185826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:15.185826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:15.185826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:21:15.185826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:15.185826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:21:15.185826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:21:15.185826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:15.185826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:21:15.185826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:15.186313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:21:15.186313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:21:15.186313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:15.186313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:21:15.186313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:15.186313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:21:19.909346 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:21:19.909953 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:21:19.910145 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:21:19.911384 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:21:19.944070 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:21:19.945485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:19.945800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:21:19.945800 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:19.945800 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:21:19.945800 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:21:19.945800 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:19.945800 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:21:19.945800 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:19.945800 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:21:19.946520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:19.946520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:21:19.946520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:19.946520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:21:19.946520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:21:19.946520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:21:19.946520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:21:19.946520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:19.946520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:19.946520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:19.946520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:21:19.946520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:19.946935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:19.946935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:19.946935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:21:19.946935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:19.946935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:21:19.946935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:21:19.946935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:19.946935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:21:19.946935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:19.946935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:21:19.946935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:21:19.946935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:19.946935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:21:19.947608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:19.947608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:21:19.947608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:19.947608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:21:19.947608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:21:19.947608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:21:19.947608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:21:19.947608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:21:19.947608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:19.947608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:21:19.947608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:21:19.947608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:21:19.947608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:21:19.947608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:21:19.947608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:19.948724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:19.948724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:21:19.948724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:21:19.948724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:21:19.948724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:21:19.948724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:19.948724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:19.948724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:19.948724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:21:19.948724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:19.948724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:19.948724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:19.948724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:21:19.948724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:19.948724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:21:19.948724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:21:19.948724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:19.948724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:21:19.948724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:19.949601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:21:19.949601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:21:19.949601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:19.949601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:21:19.949601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:19.949601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:21:20.124853 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:21:20.125896 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:21:20.126074 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:21:20.127811 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:21:20.128291 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:21:20.128626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:20.129794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:21:20.129794 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:20.129794 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:21:20.129794 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:21:20.129794 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:20.129794 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:21:20.129794 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:20.129794 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:21:20.131520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:20.131520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:21:20.131520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:20.131520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:21:20.131520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:21:20.131520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:21:20.131520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:21:20.131520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:20.131520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:20.131520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:20.131520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:21:20.131520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:20.132199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:20.132199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:20.132199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:21:20.132199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:20.132199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:21:20.132199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:21:20.132199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:20.132199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:21:20.132199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:20.132199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:21:20.132199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:21:20.132199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:20.132199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:21:20.133008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:20.133008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:21:20.133230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:20.133230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:21:20.133230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:21:20.133230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:21:20.133230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:21:20.133230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:21:20.133230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:20.133230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:21:20.133230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:21:20.133230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:21:20.133230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:21:20.133230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:21:20.133230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:20.133230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:20.133230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:21:20.133230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:21:20.133976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:21:20.133976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:21:20.133976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:20.133976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:20.133976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:20.133976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:21:20.133976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:20.133976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:20.133976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:20.133976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:21:20.133976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:20.133976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:21:20.133976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:21:20.133976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:20.133976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:21:20.133976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:20.133976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:21:20.133976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:21:20.133976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:20.134819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:21:20.134819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:20.134819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:21:20.147300 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:21:20.147586 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:21:20.147722 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:21:20.147867 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:21:20.148510 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:21:20.148856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:20.149822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:21:20.149822 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:20.149822 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:21:20.149822 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:21:20.149822 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:20.149822 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:21:20.149822 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:20.149822 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:21:20.151338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:20.151338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:21:20.151338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:21:20.151338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:21:20.151338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:20.151338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:21:20.151338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:20.151338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:21:20.151338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:20.151338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:21:20.151338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:20.151338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:21:20.151338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:20.151338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:21:20.151338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:21:20.152093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:20.152093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:21:20.152093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:20.152093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:21:20.152093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:21:20.152093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:20.152093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:21:20.152093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:20.152093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:21:20.155713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:20.155713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:21:20.155713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:20.155713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:21:20.155713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:21:20.155713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:21:20.155713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:21:20.155713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:21:20.155713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:21:20.155713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:21:20.155713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:20.155713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:20.156108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:20.156108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:21:20.156108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:20.156108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:20.156108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:20.156108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:21:20.156108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:20.156108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:21:20.156108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:21:20.156108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:20.156108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:21:20.156108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:20.156108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:21:20.156108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:21:20.156532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:20.156532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:21:20.156532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:20.156532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:21:20.157003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:20.157003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:21:20.157003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:21:20.157003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:21:20.157003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:21:20.157003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:21:20.157003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:20.157003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:21:20.157003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:21:20.157003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:21:20.157003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:21:20.157003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:21:20.157003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:20.157003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:20.157003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:21:20.157003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:21:20.157478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:21:20.157478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:21:20.157478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:21:20.157478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:21:20.157478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:21:20.157478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:20.157478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:20.157478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:20.157478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:21:20.157478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:20.157478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:20.157478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:20.157478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:21:20.157478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:20.157478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:21:20.157478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:21:20.157478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:20.157478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:21:20.157937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:20.157937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:21:20.157937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:21:20.157937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:20.157937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:21:20.157937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:20.157937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:21:20.172694 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:21:20.173087 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:21:20.173274 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:21:20.173614 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:21:20.174266 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:21:20.174667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:20.175544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:21:20.175544 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:20.175544 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:21:20.175544 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:21:20.175544 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:20.175544 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:21:20.175544 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:20.175544 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:21:20.176891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:20.176891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:21:20.176891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:21:20.176891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:21:20.176891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:20.176891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:21:20.176891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:20.176891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:21:20.176891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:20.176891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:21:20.176891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:20.176891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:21:20.176891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:20.176891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:21:20.176891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:21:20.177613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:20.177613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:21:20.177613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:20.177613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:21:20.177613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:21:20.177613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:20.177613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:21:20.177613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:20.177613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:21:20.178578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:20.178578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:21:20.178578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:20.178578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:21:20.178578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:21:20.178578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:21:20.178578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:21:20.178578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:21:20.178578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:21:20.178578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:20.178578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:20.179263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:20.179263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:21:20.179263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:20.179263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:20.179263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:20.179263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:21:20.179263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:20.179263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:21:20.179263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:21:20.179263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:20.179263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:21:20.179263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:20.179263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:21:20.179263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:21:20.180144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:20.180144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:21:20.180144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:20.180144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:21:20.180144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:20.180144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:21:20.180144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:21:20.180144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:21:20.180144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:21:20.180144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:21:20.180144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:20.180144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:21:20.180144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:21:20.180144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:21:20.181256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:21:20.181256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:21:20.181256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:20.181256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:20.181256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:21:20.181256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:21:20.181256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:21:20.181256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:21:20.181256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:21:20.181256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:21:20.181256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:20.181256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:20.181256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:20.181256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:21:20.181256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:20.181256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:20.181256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:20.181256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:21:20.181256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:20.182303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:21:20.182303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:21:20.182303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:20.182303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:21:20.182303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:20.182303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:21:20.182303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:21:20.182303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:20.182303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:21:20.182303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:20.182303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:21:20.182303 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:21:20.182303 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:21:20.182933 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:21:20.182933 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:21:20.182933 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:21:20.182933 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:21:20.182933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:20.183485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:21:20.183485 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:20.183485 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:21:20.183485 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:21:20.183485 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:20.183485 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:21:20.183485 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:20.183485 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:21:20.184447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:20.184447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:21:20.184447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:20.184447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:21:20.184447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:21:20.184447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:21:20.184447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:21:20.184447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:20.184447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:20.184447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:20.184447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:21:20.184447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:20.185101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:20.185101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:20.185101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:21:20.185101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:20.185101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:21:20.185101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:21:20.185101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:20.185101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:21:20.185101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:20.185101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:21:20.185101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:21:20.185101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:20.185101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:21:20.185935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:20.185935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:21:20.185935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:20.185935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:21:20.185935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:21:20.185935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:21:20.185935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:21:20.185935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:21:20.185935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:20.185935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:21:20.185935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:21:20.185935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:21:20.185935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:21:20.185935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:21:20.185935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:20.186834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:20.186834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:21:20.186834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:21:20.186834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:21:20.186834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:21:20.186834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:20.186834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:20.186834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:20.186834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:21:20.186834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:20.186834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:20.186834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:20.186834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:21:20.186834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:20.186834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:21:20.186834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:21:20.186834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:20.186834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:21:20.186834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:20.187720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:21:20.187720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:21:20.187720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:20.187720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:21:20.187720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:20.187720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:21:24.910026 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:21:24.910597 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:21:24.910726 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:21:24.911936 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:21:24.913915 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG dbcounter [-] [105581] Writing DB stats neutron:SELECT=240 {{(pid=105581) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 14:21:24.946173 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:21:24.946666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:24.947536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:21:24.947536 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:24.947536 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:21:24.947536 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:21:24.947536 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:24.947536 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:21:24.947536 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:24.947536 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:21:24.948523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:24.948523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:21:24.948523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:24.948523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:21:24.948523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:21:24.948523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:21:24.948523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:21:24.948523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:24.948523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:24.948523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:24.948523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:21:24.948523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:24.949195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:24.949195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:24.949195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:21:24.949195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:24.949195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:21:24.949195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:21:24.949195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:24.949195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:21:24.949195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:24.949195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:21:24.949195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:21:24.949195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:24.949195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:21:24.949863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:24.949863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:21:24.949863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:24.949863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:21:24.949863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:21:24.949863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:21:24.949863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:21:24.949863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:21:24.949863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:24.949863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:21:24.949863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:21:24.949863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:21:24.949863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:21:24.949863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:21:24.949863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:24.950775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:24.950775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:21:24.950775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:21:24.950775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:21:24.950775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:21:24.950775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:24.950775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:24.950775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:24.950775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:21:24.950775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:24.950775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:24.950775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:24.950775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:21:24.950775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:24.950775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:21:24.950775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:21:24.950775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:24.950775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:21:24.950775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:24.951638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:21:24.951638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:21:24.951638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:24.951638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:21:24.951638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:24.951638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:21:25.126011 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:21:25.126566 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:21:25.126751 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:21:25.127802 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:21:25.128055 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:21:25.128248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:25.129020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:21:25.129020 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:25.129020 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:21:25.129020 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:21:25.129020 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:25.129020 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:21:25.129020 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:25.129020 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:21:25.130819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:25.130819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:21:25.130819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:25.130819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:21:25.130819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:21:25.130819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:21:25.130819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:21:25.130819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:25.130819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:25.130819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:25.130819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:21:25.130819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:25.131391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:25.131391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:25.131391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:21:25.131391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:25.131391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:21:25.131391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:21:25.131391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:25.131391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:21:25.131391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:25.131391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:21:25.131391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:21:25.131391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:25.131391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:21:25.132025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:25.132025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:21:25.132547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:25.132547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:21:25.132547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:21:25.132547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:21:25.132547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:21:25.132547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:21:25.132547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:25.132547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:21:25.132547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:21:25.132547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:21:25.132547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:21:25.132547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:21:25.132547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:25.132547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:25.132547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:21:25.132547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:21:25.133356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:21:25.133356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:21:25.133356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:25.133356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:25.133356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:25.133356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:21:25.133356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:25.133356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:25.133356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:25.133356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:21:25.133356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:25.133356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:21:25.133356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:21:25.133356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:25.133356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:21:25.133356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:25.133356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:21:25.133356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:21:25.133356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:25.134225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:21:25.134225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:25.134225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:21:25.147757 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:21:25.148133 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:21:25.148314 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:21:25.148512 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:21:25.148988 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:21:25.149191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:25.149815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:21:25.149815 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:25.149815 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:21:25.149815 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:21:25.149815 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:25.149815 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:21:25.149815 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:25.149815 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:21:25.150975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:25.150975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:21:25.150975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:21:25.150975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:21:25.150975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:25.150975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:21:25.150975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:25.150975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:21:25.150975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:25.150975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:21:25.150975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:25.150975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:21:25.150975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:25.150975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:21:25.150975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:21:25.151782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:25.151782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:21:25.151782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:25.151782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:21:25.151782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:21:25.151782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:25.151782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:21:25.151782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:25.151782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:21:25.152245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:25.152245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:21:25.152245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:25.152245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:21:25.152245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:21:25.152245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:21:25.152245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:21:25.152245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:21:25.152245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:21:25.152245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:21:25.152245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:25.152245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:25.153034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:25.153034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:21:25.153034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:25.153034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:25.153034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:25.153034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:21:25.153034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:25.153034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:21:25.153034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:21:25.153034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:25.153034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:21:25.153034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:25.153034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:21:25.153034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:21:25.153706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:25.153706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:21:25.153706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:25.153706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:21:25.153880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:25.153880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:21:25.153880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:21:25.153880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:21:25.153880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:21:25.153880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:21:25.153880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:25.153880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:21:25.153880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:21:25.153880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:21:25.153880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:21:25.153880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:21:25.153880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:25.153880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:25.153880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:21:25.153880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:21:25.154325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:21:25.154325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:21:25.154325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:21:25.154325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:21:25.154325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:21:25.154325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:25.154325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:25.154325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:25.154325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:21:25.154325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:25.154325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:25.154325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:25.154325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:21:25.154325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:25.154325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:21:25.154325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:21:25.154325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:25.154325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:21:25.154872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:25.154872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:21:25.154872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:21:25.154872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:25.154872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:21:25.154872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:25.154872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:21:25.173178 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:21:25.173516 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:21:25.173690 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:21:25.173882 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:21:25.174291 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:21:25.174516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:25.175036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:21:25.175036 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:25.175036 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:21:25.175036 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:21:25.175036 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:25.175036 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:21:25.175036 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:25.175036 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:21:25.176195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:25.176195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:21:25.176195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:21:25.176195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:21:25.176195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:25.176195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:21:25.176195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:25.176195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:21:25.176195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:25.176195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:21:25.176195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:25.176195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:21:25.176195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:25.176195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:21:25.176195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:21:25.176982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:25.176982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:21:25.176982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:25.176982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:21:25.176982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:21:25.176982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:25.176982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:21:25.176982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:25.176982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:21:25.177612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:25.177612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:21:25.177612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:25.177612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:21:25.177612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:21:25.177612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:21:25.177612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:21:25.177612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:21:25.177612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:21:25.177612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:25.177612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:25.178085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:25.178085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:21:25.178085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:25.178085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:25.178085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:25.178085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:21:25.178085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:25.178085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:21:25.178085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:21:25.178085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:25.178085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:21:25.178085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:25.178085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:21:25.178085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:21:25.178454 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:25.178454 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:21:25.178454 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:25.178454 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:21:25.179653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:25.179653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:21:25.179653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:21:25.179653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:21:25.179653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:21:25.179653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:21:25.179653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:25.179653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:21:25.179653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:21:25.179653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:21:25.179653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:21:25.179653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:21:25.179653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:25.179653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:25.179653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:21:25.179653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:21:25.179653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:21:25.180151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:21:25.180151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:21:25.180151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:21:25.180151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:25.180151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:25.180151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:25.180151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:21:25.180151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:25.180151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:25.180151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:25.180151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:21:25.180151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:25.180151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:21:25.180151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:21:25.180151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:25.180151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:21:25.180151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:25.180151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:21:25.180151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:21:25.180663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:25.180663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:21:25.180663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:25.180663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:21:25.181099 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:21:25.181439 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:21:25.181632 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:21:25.181855 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:21:25.182191 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:21:25.182421 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:21:25.182580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:25.183049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:21:25.183049 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:25.183049 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:21:25.183049 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:21:25.183049 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:25.183049 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:21:25.183049 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:25.183049 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:21:25.183848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:25.183848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:21:25.183848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:25.183848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:21:25.183848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:21:25.183848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:21:25.183848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:21:25.183848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:25.183848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:25.183848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:25.183848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:21:25.183848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:25.184191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:25.184191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:25.184191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:21:25.184191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:25.184191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:21:25.184191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:21:25.184191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:25.184191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:21:25.184191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:25.184191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:21:25.184191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:21:25.184191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:25.184191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:21:25.184568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:25.184568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:21:25.184915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:25.184915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:21:25.184915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:21:25.184915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:21:25.184915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:21:25.184915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:21:25.184915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:25.184915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:21:25.184915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:21:25.184915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:21:25.184915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:21:25.184915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:21:25.184915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:25.184915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:25.184915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:21:25.184915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:21:25.185351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:21:25.185351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:21:25.185351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:25.185351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:25.185351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:25.185351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:21:25.185351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:25.185351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:25.185351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:25.185351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:21:25.185351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:25.185351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:21:25.185351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:21:25.185351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:25.185351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:21:25.185351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:25.185351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:21:25.185351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:21:25.185351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:25.185846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:21:25.185846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:25.185846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:21:29.910869 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:21:29.911708 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:21:29.911789 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:21:29.913128 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:21:29.949518 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:21:29.949518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:29.950149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:21:29.950149 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:29.950149 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:21:29.950149 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:21:29.950149 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:29.950149 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:21:29.950149 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:29.950149 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:21:29.952331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:29.952331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:21:29.952331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:29.952331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:21:29.952331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:21:29.952331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:21:29.952331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:21:29.952331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:29.952331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:29.952331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:29.952331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:21:29.952331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:29.952678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:29.952678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:29.952678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:21:29.952678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:29.952678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:21:29.952678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:21:29.952678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:29.952678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:21:29.952678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:29.952678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:21:29.952678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:21:29.952678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:29.952678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:21:29.952956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:29.952956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:21:29.952956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:29.952956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:21:29.952956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:21:29.952956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:21:29.952956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:21:29.952956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:21:29.952956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:29.952956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:21:29.952956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:21:29.952956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:21:29.952956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:21:29.952956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:21:29.952956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:29.953290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:29.953290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:21:29.953290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:21:29.953290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:21:29.953290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:21:29.953290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:29.953290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:29.953290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:29.953290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:21:29.953290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:29.953290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:29.953290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:29.953290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:21:29.953290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:29.953290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:21:29.953290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:21:29.953290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:29.953290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:21:29.953290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:29.953905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:21:29.953905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:21:29.953905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:29.953905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:21:29.953905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:29.953905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:21:30.126647 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:21:30.128512 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:21:30.128512 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:21:30.128929 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:21:30.129604 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:21:30.129604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:30.133688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:21:30.133688 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:30.133688 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:21:30.133688 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:21:30.133688 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:30.133688 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:21:30.133688 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:30.133688 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:21:30.133688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:30.133688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:21:30.133688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:30.133688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:21:30.133688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:21:30.133688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:21:30.133688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:21:30.135220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:30.135220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:30.135220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:30.135220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:21:30.135220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:30.135220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:30.135220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:30.135220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:21:30.135220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:30.135220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:21:30.135220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:21:30.135220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:30.135220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:21:30.135220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:30.136556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:21:30.136556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:21:30.136556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:30.136556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:21:30.136556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:30.136556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:21:30.138576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:30.138576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:21:30.138576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:21:30.138576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:21:30.138576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:21:30.138576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:21:30.138576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:30.138576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:21:30.138576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:21:30.138576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:21:30.138576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:21:30.138576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:21:30.138576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:30.138576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:30.138576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:21:30.138576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:21:30.138938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:21:30.138938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:21:30.138938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:30.138938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:30.138938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:30.138938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:21:30.138938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:30.138938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:30.138938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:30.138938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:21:30.138938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:30.138938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:21:30.138938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:21:30.138938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:30.138938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:21:30.138938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:30.138938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:21:30.138938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:21:30.138938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:30.139300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:21:30.139300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:30.139300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:21:30.148168 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:21:30.148689 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:21:30.148689 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:21:30.148689 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:21:30.149206 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:21:30.149517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:30.150469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:21:30.150469 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:30.150469 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:21:30.150469 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:21:30.150469 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:30.150469 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:21:30.150469 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:30.150469 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:21:30.151636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:30.151636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:21:30.151636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:21:30.151636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:21:30.151636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:30.151636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:21:30.151636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:30.151636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:21:30.151636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:30.151636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:21:30.151636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:30.151636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:21:30.151636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:30.151636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:21:30.151636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:21:30.152306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:30.152306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:21:30.152306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:30.152306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:21:30.152306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:21:30.152306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:30.152306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:21:30.152306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:30.152306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:21:30.152306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:30.152306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:21:30.152306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:30.152306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:21:30.152306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:21:30.152801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:21:30.152801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:21:30.152801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:21:30.152801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:21:30.152801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:21:30.152801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:30.152801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:30.152801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:30.152801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:21:30.152801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:30.152801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:30.152801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:30.152801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:21:30.153195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:30.153195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:21:30.153195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:21:30.153195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:30.153195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:21:30.153195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:30.153195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:21:30.153195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:21:30.153195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:30.153195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:21:30.153195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:30.153195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:21:30.153536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:30.153536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:21:30.153536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:21:30.153536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:21:30.153536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:21:30.153536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:21:30.153536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:30.153536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:21:30.153536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:21:30.153536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:21:30.153536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:21:30.153536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:21:30.153536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:30.153536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:30.153536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:21:30.153536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:21:30.153984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:21:30.153984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:21:30.153984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:21:30.153984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:21:30.153984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:21:30.153984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:30.153984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:30.153984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:30.153984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:21:30.153984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:30.153984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:30.153984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:30.153984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:21:30.153984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:30.153984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:21:30.153984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:21:30.153984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:30.153984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:21:30.154497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:30.154497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:21:30.154497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:21:30.154497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:30.154497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:21:30.154497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:30.154497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:21:30.173617 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:21:30.173756 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:21:30.173836 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:21:30.173924 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:21:30.174600 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:21:30.175012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:30.175757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:21:30.175757 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:30.175757 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:21:30.175757 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:21:30.175757 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:30.175757 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:21:30.175757 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:30.175757 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:21:30.176279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:30.176279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:21:30.176279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:21:30.176279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:21:30.176279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:30.176279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:21:30.176279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:30.176279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:21:30.176279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:30.176279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:21:30.176279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:30.176279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:21:30.176279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:30.176279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:21:30.176279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:21:30.176971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:30.176971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:21:30.176971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:30.176971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:21:30.176971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:21:30.176971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:30.176971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:21:30.176971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:30.176971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:21:30.176971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:30.176971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:21:30.176971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:30.176971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:21:30.176971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:21:30.177638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:21:30.177638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:21:30.177638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:21:30.177638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:21:30.177638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:30.177638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:30.177638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:30.177638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:21:30.177638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:30.177638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:30.177638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:30.177638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:21:30.177638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:30.178226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:21:30.178226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:21:30.178226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:30.178226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:21:30.178226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:30.178226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:21:30.178226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:21:30.178226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:30.178226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:21:30.178226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:30.178226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:21:30.178226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:30.178792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:21:30.178792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:21:30.178792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:21:30.178792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:21:30.178792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:21:30.178792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:30.178792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:21:30.178792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:21:30.178792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:21:30.178792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:21:30.178792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:21:30.178792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:30.178792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:30.178792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:21:30.178792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:21:30.178792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:21:30.178792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:21:30.178792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:21:30.178792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:21:30.179681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:30.179681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:30.179681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:30.179681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:21:30.179681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:30.179681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:30.179681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:30.179681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:21:30.179681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:30.179681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:21:30.179681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:21:30.179681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:30.179681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:21:30.179681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:30.179681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:21:30.179681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:21:30.179681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:30.179681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:21:30.179681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:30.179681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:21:30.181542 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:21:30.181755 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:21:30.181850 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:21:30.181914 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:21:30.182230 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:21:30.182464 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:21:30.182626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:30.183055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:21:30.183055 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:30.183055 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:21:30.183055 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:21:30.183055 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:30.183055 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:21:30.183055 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:30.183055 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:21:30.183667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:30.183667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:21:30.183667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:30.183667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:21:30.183667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:21:30.183667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:21:30.183667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:21:30.183667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:30.183667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:30.183667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:30.183667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:21:30.183667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:30.184021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:30.184021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:30.184021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:21:30.184021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:30.184021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:21:30.184021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:21:30.184021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:30.184021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:21:30.184021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:30.184021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:21:30.184021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:21:30.184021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:30.184021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:21:30.184394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:30.184394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:21:30.184394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:30.184394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:21:30.184394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:21:30.184394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:21:30.184394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:21:30.184394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:21:30.184394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:30.184394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:21:30.184394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:21:30.184394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:21:30.184394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:21:30.184394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:21:30.184394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:30.184834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:30.184834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:21:30.184834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:21:30.184834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:21:30.184834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:21:30.184834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:30.184834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:30.184834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:30.184834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:21:30.184834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:30.184834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:30.184834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:30.184834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:21:30.184834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:30.184834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:21:30.184834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:21:30.184834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:30.184834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:21:30.184834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:30.185357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:21:30.185357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:21:30.185357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:30.185357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:21:30.185357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:30.185357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:21:34.911715 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:21:34.912889 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:21:34.912889 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:21:34.913590 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:21:34.948019 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:21:34.948493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:34.949384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:21:34.949384 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:34.949384 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:21:34.949384 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:21:34.949384 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:34.949384 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:21:34.949384 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:34.949384 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:21:34.950171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:34.950171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:21:34.950171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:34.950171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:21:34.950171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:21:34.950171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:21:34.950171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:21:34.950171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:34.950171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:34.950171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:34.950171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:21:34.950171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:34.950607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:34.950607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:34.950607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:21:34.950607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:34.950607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:21:34.950607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:21:34.950607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:34.950607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:21:34.950607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:34.950607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:21:34.950607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:21:34.950607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:34.950607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:21:34.950979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:34.950979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:21:34.950979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:34.950979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:21:34.950979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:21:34.950979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:21:34.950979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:21:34.950979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:21:34.950979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:34.950979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:21:34.950979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:21:34.950979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:21:34.950979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:21:34.950979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:21:34.950979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:34.951445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:34.951445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:21:34.951445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:21:34.951445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:21:34.951445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:21:34.951445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:34.951445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:34.951445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:34.951445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:21:34.951445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:34.951445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:34.951445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:34.951445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:21:34.951445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:34.951445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:21:34.951445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:21:34.951445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:34.951445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:21:34.951445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:34.951978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:21:34.951978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:21:34.951978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:34.951978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:21:34.951978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:34.951978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:21:35.127682 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:21:35.128221 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:21:35.128428 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:21:35.129610 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:21:35.129938 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:21:35.130208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:35.131302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:21:35.131302 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:35.131302 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:21:35.131302 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:21:35.131302 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:35.131302 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:21:35.131302 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:35.131302 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:21:35.132868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:35.132868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:21:35.132868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:35.132868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:21:35.132868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:21:35.132868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:21:35.132868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:21:35.132868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:35.132868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:35.132868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:35.132868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:21:35.132868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:35.133675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:35.133675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:35.133675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:21:35.133675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:35.133675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:21:35.133675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:21:35.133675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:35.133675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:21:35.133675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:35.133675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:21:35.133675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:21:35.133675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:35.133675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:21:35.134373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:35.134373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:21:35.134549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:35.134549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:21:35.134549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:21:35.134549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:21:35.134549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:21:35.134549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:21:35.134549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:35.134549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:21:35.134549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:21:35.134549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:21:35.134549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:21:35.134549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:21:35.134549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:35.134549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:35.134549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:21:35.134549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:21:35.134972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:21:35.134972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:21:35.134972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:35.134972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:35.134972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:35.134972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:21:35.134972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:35.134972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:35.134972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:35.134972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:21:35.134972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:35.134972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:21:35.134972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:21:35.134972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:35.134972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:21:35.134972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:35.134972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:21:35.134972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:21:35.134972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:35.135483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:21:35.135483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:35.135483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:21:35.148797 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:21:35.149261 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:21:35.149479 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:21:35.149699 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:21:35.150146 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:21:35.150684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:35.150940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:21:35.150940 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:35.150940 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:21:35.150940 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:21:35.150940 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:35.150940 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:21:35.150940 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:35.150940 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:21:35.152186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:35.152186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:21:35.152186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:21:35.152186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:21:35.152186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:35.152186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:21:35.152186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:35.152186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:21:35.152186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:35.152186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:21:35.152186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:35.152186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:21:35.152186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:35.152186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:21:35.152186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:21:35.153172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:35.153172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:21:35.153172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:35.153172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:21:35.153172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:21:35.153172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:35.153172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:21:35.153172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:35.153172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:21:35.153694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:35.153694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:21:35.153694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:35.153694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:21:35.153694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:21:35.153694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:21:35.153694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:21:35.153694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:21:35.153694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:21:35.153694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:21:35.153694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:35.153694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:35.154339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:35.154339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:21:35.154339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:35.154339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:35.154339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:35.154339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:21:35.154339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:35.154339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:21:35.154339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:21:35.154339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:35.154339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:21:35.154339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:35.154339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:21:35.154339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:21:35.155004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:35.155004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:21:35.155004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:35.155004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:21:35.155400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:35.155400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:21:35.155400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:21:35.155400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:21:35.155400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:21:35.155400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:21:35.155400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:35.155400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:21:35.155400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:21:35.155400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:21:35.155400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:21:35.155400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:21:35.155400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:35.155400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:35.155400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:21:35.155400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:21:35.156148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:21:35.156148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:21:35.156148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:21:35.156148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:21:35.156148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:21:35.156148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:35.156148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:35.156148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:35.156148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:21:35.156148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:35.156148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:35.156148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:35.156148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:21:35.156148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:35.156148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:21:35.156148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:21:35.156148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:35.156148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:21:35.157182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:35.157182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:21:35.157182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:21:35.157182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:35.157182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:21:35.157182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:35.157182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:21:35.174080 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:21:35.174417 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:21:35.174627 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:21:35.174850 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:21:35.175338 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:21:35.175709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:35.176441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:21:35.176441 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:35.176441 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:21:35.176441 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:21:35.176441 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:35.176441 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:21:35.176441 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:35.176441 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:21:35.177652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:35.177652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:21:35.177652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:21:35.177652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:21:35.177652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:35.177652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:21:35.177652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:35.177652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:21:35.177652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:35.177652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:21:35.177652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:35.177652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:21:35.177652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:35.177652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:21:35.177652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:21:35.178327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:35.178327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:21:35.178327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:35.178327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:21:35.178327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:21:35.178327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:35.178327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:21:35.178327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:35.178327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:21:35.178789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:35.178789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:21:35.178789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:35.178789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:21:35.178789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:21:35.178789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:21:35.178789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:21:35.178789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:21:35.178789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:21:35.178789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:35.178789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:35.179326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:35.179326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:21:35.179326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:35.179326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:35.179326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:35.179326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:21:35.179326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:35.179326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:21:35.179326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:21:35.179326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:35.179326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:21:35.179326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:35.179326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:21:35.179326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:21:35.179957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:35.179957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:21:35.179957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:35.179957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:21:35.179957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:35.179957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:21:35.179957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:21:35.179957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:21:35.179957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:21:35.179957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:21:35.179957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:35.179957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:21:35.179957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:21:35.179957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:21:35.180586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:21:35.180586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:21:35.180586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:35.180586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:35.180586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:21:35.180586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:21:35.180586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:21:35.180586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:21:35.180586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:21:35.180586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:21:35.180586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:35.180586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:35.180586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:35.180586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:21:35.180586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:35.180586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:35.180586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:35.180586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:21:35.180586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:35.181383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:21:35.181383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:21:35.181383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:35.181383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:21:35.181383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:35.181383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:21:35.181383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:21:35.181383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:35.181383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:21:35.181383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:35.181383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:21:35.182057 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:21:35.182324 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:21:35.182438 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:21:35.182597 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:21:35.183004 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:21:35.183368 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:21:35.183584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:35.184238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:21:35.184238 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:35.184238 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:21:35.184238 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:21:35.184238 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:35.184238 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:21:35.184238 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:35.184238 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:21:35.185076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:35.185076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:21:35.185076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:35.185076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:21:35.185076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:21:35.185076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:21:35.185076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:21:35.185076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:35.185076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:35.185076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:35.185076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:21:35.185076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:35.185508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:35.185508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:35.185508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:21:35.185508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:35.185508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:21:35.185508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:21:35.185508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:35.185508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:21:35.185508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:35.185508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:21:35.185508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:21:35.185508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:35.185508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:21:35.185868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:35.185868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:21:35.186071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:35.186071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:21:35.186071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:21:35.186071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:21:35.186071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:21:35.186071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:21:35.186071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:35.186071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:21:35.186071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:21:35.186071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:21:35.186071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:21:35.186071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:21:35.186071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:35.186071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:35.186071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:21:35.186071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:21:35.186512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:21:35.186512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:21:35.186512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:35.186512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:35.186512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:35.186512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:21:35.186512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:35.186512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:35.186512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:35.186512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:21:35.186512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:35.186512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:21:35.186512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:21:35.186512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:35.186512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:21:35.186512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:35.186512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:21:35.186512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:21:35.186512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:35.187135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:21:35.187135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:35.187135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:21:39.912583 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:21:39.913329 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:21:39.913329 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:21:39.914485 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:21:39.944450 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:21:39.944981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:39.946069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:21:39.946069 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:39.946069 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:21:39.946069 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:21:39.946069 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:39.946069 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:21:39.946069 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:39.946069 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:21:39.947664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:39.947664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:21:39.947664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:39.947664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:21:39.947664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:21:39.947664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:21:39.947664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:21:39.947664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:39.947664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:39.947664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:39.947664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:21:39.947664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:39.948218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:39.948218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:39.948218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:21:39.948218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:39.948218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:21:39.948218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:21:39.948218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:39.948218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:21:39.948218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:39.948218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:21:39.948218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:21:39.948218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:39.948218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:21:39.950172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:39.950172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:21:39.950172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:39.950172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:21:39.950172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:21:39.950172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:21:39.950172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:21:39.950172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:21:39.950172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:39.950172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:21:39.950172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:21:39.950172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:21:39.950172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:21:39.950172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:21:39.950172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:39.950946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:39.950946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:21:39.950946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:21:39.950946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:21:39.950946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:21:39.950946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:39.950946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:39.950946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:39.950946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:21:39.950946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:39.950946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:39.950946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:39.950946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:21:39.950946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:39.950946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:21:39.950946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:21:39.950946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:39.950946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:21:39.950946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:39.951897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:21:39.951897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:21:39.951897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:39.951897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:21:39.951897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:39.951897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:21:40.128345 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:21:40.128755 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:21:40.128927 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:21:40.130029 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:21:40.130342 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:21:40.130669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:40.131540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:21:40.131540 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:40.131540 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:21:40.131540 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:21:40.131540 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:40.131540 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:21:40.131540 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:40.131540 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:21:40.133441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:40.133441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:21:40.133441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:40.133441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:21:40.133441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:21:40.133441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:21:40.133441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:21:40.133441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:40.133441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:40.133441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:40.133441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:21:40.133441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:40.134063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:40.134063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:40.134063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:21:40.134063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:40.134063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:21:40.134063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:21:40.134063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:40.134063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:21:40.134063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:40.134063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:21:40.134063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:21:40.134063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:40.134063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:21:40.134713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:40.134713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:21:40.135250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:40.135250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:21:40.135250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:21:40.135250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:21:40.135250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:21:40.135250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:21:40.135250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:40.135250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:21:40.135250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:21:40.135250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:21:40.135250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:21:40.135250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:21:40.135250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:40.135250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:40.135250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:21:40.135250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:21:40.135974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:21:40.135974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:21:40.135974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:40.135974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:40.135974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:40.135974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:21:40.135974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:40.135974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:40.135974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:40.135974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:21:40.135974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:40.135974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:21:40.135974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:21:40.135974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:40.135974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:21:40.135974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:40.135974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:21:40.135974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:21:40.135974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:40.136934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:21:40.136934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:40.136934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:21:40.149324 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:21:40.149581 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:21:40.149691 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:21:40.149807 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:21:40.150307 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:21:40.150660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:40.151376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:21:40.151376 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:40.151376 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:21:40.151376 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:21:40.151376 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:40.151376 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:21:40.151376 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:40.151376 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:21:40.152518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:40.152518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:21:40.152518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:21:40.152518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:21:40.152518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:40.152518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:21:40.152518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:40.152518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:21:40.152518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:40.152518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:21:40.152518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:40.152518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:21:40.152518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:40.152518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:21:40.152518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:21:40.153372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:40.153372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:21:40.153372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:40.153372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:21:40.153372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:21:40.153372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:40.153372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:21:40.153372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:40.153372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:21:40.153875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:40.153875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:21:40.153875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:40.153875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:21:40.153875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:21:40.153875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:21:40.153875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:21:40.153875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:21:40.153875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:21:40.153875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:21:40.153875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:40.153875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:40.154426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:40.154426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:21:40.154426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:40.154426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:40.154426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:40.154426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:21:40.154426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:40.154426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:21:40.154426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:21:40.154426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:40.154426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:21:40.154426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:40.154426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:21:40.154426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:21:40.155030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:40.155030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:21:40.155030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:40.155030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:21:40.155598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:40.155598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:21:40.155598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:21:40.155598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:21:40.155598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:21:40.155598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:21:40.155598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:40.155598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:21:40.155598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:21:40.155598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:21:40.155598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:21:40.155598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:21:40.155598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:40.155598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:40.155598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:21:40.155598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:21:40.156269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:21:40.156269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:21:40.156269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:21:40.156269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:21:40.156269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:21:40.156269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:40.156269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:40.156269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:40.156269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:21:40.156269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:40.156269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:40.156269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:40.156269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:21:40.156269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:40.156269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:21:40.156269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:21:40.156269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:40.156269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:21:40.157297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:40.157297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:21:40.157297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:21:40.157297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:40.157297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:21:40.157297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:40.157297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:21:40.174564 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:21:40.174869 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:21:40.175053 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:21:40.175254 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:21:40.175754 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:21:40.176097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:40.176943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:21:40.176943 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:40.176943 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:21:40.176943 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:21:40.176943 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:40.176943 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:21:40.176943 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:40.176943 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:21:40.178085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:40.178085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:21:40.178085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:21:40.178085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:21:40.178085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:40.178085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:21:40.178085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:40.178085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:21:40.178085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:40.178085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:21:40.178085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:40.178085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:21:40.178085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:40.178085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:21:40.178085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:21:40.178813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:40.178813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:21:40.178813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:40.178813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:21:40.178813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:21:40.178813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:40.178813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:21:40.178813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:40.178813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:21:40.179477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:40.179477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:21:40.179477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:40.179477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:21:40.179477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:21:40.179477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:21:40.179477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:21:40.179477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:21:40.179477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:21:40.179477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:40.179477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:40.179936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:40.179936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:21:40.179936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:40.179936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:40.179936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:40.179936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:21:40.179936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:40.179936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:21:40.179936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:21:40.179936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:40.179936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:21:40.179936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:40.179936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:21:40.179936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:21:40.180525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:40.180525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:21:40.180525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:40.180525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:21:40.181200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:40.181200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:21:40.181200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:21:40.181200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:21:40.181200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:21:40.181200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:21:40.181200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:40.181200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:21:40.181200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:21:40.181200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:21:40.181200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:21:40.181200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:21:40.181200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:40.181200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:40.181200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:21:40.181200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:21:40.181200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:21:40.181947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:21:40.181947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:21:40.181947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:21:40.181947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:40.181947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:40.181947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:40.181947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:21:40.181947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:40.181947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:40.181947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:40.181947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:21:40.181947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:40.181947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:21:40.181947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:21:40.181947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:40.181947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:21:40.181947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:40.181947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:21:40.181947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:21:40.182802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:40.182802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:21:40.182802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:40.182802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:21:40.182802 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:21:40.182802 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:21:40.183078 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:21:40.183137 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:21:40.183688 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:21:40.184030 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:21:40.184247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:40.185136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:21:40.185136 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:40.185136 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:21:40.185136 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:21:40.185136 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:40.185136 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:21:40.185136 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:40.185136 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:21:40.186067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:40.186067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:21:40.186067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:40.186067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:21:40.186067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:21:40.186067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:21:40.186067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:21:40.186067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:40.186067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:40.186067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:40.186067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:21:40.186067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:40.186429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:40.186429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:40.186429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:21:40.186429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:40.186429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:21:40.186429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:21:40.186429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:40.186429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:21:40.186429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:40.186429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:21:40.186429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:21:40.186429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:40.186429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:21:40.186820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:40.186820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:21:40.186991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:40.186991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:21:40.186991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:21:40.186991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:21:40.186991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:21:40.186991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:21:40.186991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:40.186991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:21:40.186991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:21:40.186991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:21:40.186991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:21:40.186991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:21:40.186991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:40.186991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:40.186991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:21:40.186991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:21:40.187436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:21:40.187436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:21:40.187436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:40.187436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:40.187436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:40.187436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:21:40.187436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:40.187436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:40.187436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:40.187436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:21:40.187436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:40.187436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:21:40.187436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:21:40.187436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:40.187436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:21:40.187436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:40.187436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:21:40.187436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:21:40.187436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:40.187962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:21:40.187962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:40.187962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:21:44.913101 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:21:44.914041 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:21:44.914041 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:21:44.915574 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:21:44.948154 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:21:44.948601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:44.949509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:21:44.949509 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:44.949509 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:21:44.949509 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:21:44.949509 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:44.949509 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:21:44.949509 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:44.949509 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:21:44.950523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:44.950523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:21:44.950523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:44.950523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:21:44.950523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:21:44.950523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:21:44.950523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:21:44.950523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:44.950523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:44.950523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:44.950523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:21:44.950523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:44.951180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:44.951180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:44.951180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:21:44.951180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:44.951180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:21:44.951180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:21:44.951180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:44.951180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:21:44.951180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:44.951180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:21:44.951180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:21:44.951180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:44.951180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:21:44.951860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:44.951860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:21:44.951860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:44.951860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:21:44.951860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:21:44.951860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:21:44.951860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:21:44.951860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:21:44.951860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:44.951860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:21:44.951860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:21:44.951860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:21:44.951860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:21:44.951860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:21:44.951860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:44.952610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:44.952610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:21:44.952610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:21:44.952610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:21:44.952610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:21:44.952610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:44.952610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:44.952610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:44.952610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:21:44.952610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:44.952610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:44.952610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:44.952610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:21:44.952610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:44.952610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:21:44.952610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:21:44.952610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:44.952610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:21:44.952610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:44.953542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:21:44.953542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:21:44.953542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:44.953542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:21:44.953542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:44.953542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:21:45.128848 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:21:45.129264 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:21:45.129487 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:21:45.130336 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:21:45.130605 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:21:45.130762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:45.131481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:21:45.131481 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:45.131481 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:21:45.131481 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:21:45.131481 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:45.131481 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:21:45.131481 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:45.131481 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:21:45.132947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:45.132947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:21:45.132947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:45.132947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:21:45.132947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:21:45.132947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:21:45.132947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:21:45.132947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:45.132947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:45.132947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:45.132947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:21:45.132947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:45.133375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:45.133375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:45.133375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:21:45.133375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:45.133375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:21:45.133375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:21:45.133375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:45.133375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:21:45.133375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:45.133375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:21:45.133375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:21:45.133375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:45.133375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:21:45.133843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:45.133843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:21:45.134568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:45.134568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:21:45.134568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:21:45.134568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:21:45.134568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:21:45.134568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:21:45.134568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:45.134568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:21:45.134568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:21:45.134568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:21:45.134568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:21:45.134568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:21:45.134568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:45.134568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:45.134568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:21:45.134568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:21:45.135161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:21:45.135161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:21:45.135161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:45.135161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:45.135161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:45.135161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:21:45.135161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:45.135161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:45.135161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:45.135161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:21:45.135161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:45.135161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:21:45.135161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:21:45.135161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:45.135161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:21:45.135161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:45.135161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:21:45.135161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:21:45.135161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:45.135924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:21:45.135924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:45.135924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:21:45.149748 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:21:45.150168 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:21:45.150375 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:21:45.150626 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:21:45.151257 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:21:45.151557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:45.152084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:21:45.152084 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:45.152084 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:21:45.152084 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:21:45.152084 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:45.152084 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:21:45.152084 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:45.152084 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:21:45.153359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:45.153359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:21:45.153359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:21:45.153359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:21:45.153359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:45.153359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:21:45.153359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:45.153359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:21:45.153359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:45.153359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:21:45.153359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:45.153359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:21:45.153359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:45.153359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:21:45.153359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:21:45.153966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:45.153966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:21:45.153966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:45.153966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:21:45.153966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:21:45.153966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:45.153966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:21:45.153966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:45.153966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:21:45.154804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:45.154804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:21:45.154804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:45.154804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:21:45.154804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:21:45.154804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:21:45.154804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:21:45.154804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:21:45.154804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:21:45.154804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:21:45.154804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:45.154804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:45.155193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:45.155193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:21:45.155193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:45.155193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:45.155193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:45.155193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:21:45.155193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:45.155193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:21:45.155193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:21:45.155193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:45.155193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:21:45.155193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:45.155193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:21:45.155193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:21:45.155756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:45.155756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:21:45.155756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:45.155756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:21:45.156613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:45.156613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:21:45.156613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:21:45.156613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:21:45.156613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:21:45.156613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:21:45.156613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:45.156613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:21:45.156613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:21:45.156613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:21:45.156613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:21:45.156613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:21:45.156613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:45.156613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:45.156613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:21:45.156613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:21:45.157089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:21:45.157089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:21:45.157089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:21:45.157089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:21:45.157089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:21:45.157089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:45.157089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:45.157089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:45.157089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:21:45.157089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:45.157089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:45.157089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:45.157089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:21:45.157089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:45.157089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:21:45.157089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:21:45.157089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:45.157089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:21:45.157632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:45.157632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:21:45.157632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:21:45.157632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:45.157632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:21:45.157632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:45.157632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:21:45.175114 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:21:45.178030 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:21:45.178302 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:21:45.178621 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:21:45.179105 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:21:45.179540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:45.180262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:21:45.180262 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:45.180262 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:21:45.180262 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:21:45.180262 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:45.180262 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:21:45.180262 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:45.180262 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:21:45.181542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:45.181542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:21:45.181542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:21:45.181542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:21:45.181542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:45.181542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:21:45.181542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:45.181542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:21:45.181542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:45.181542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:21:45.181542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:45.181542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:21:45.181542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:45.181542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:21:45.181542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:21:45.182468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:45.182468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:21:45.182468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:45.182468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:21:45.182468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:21:45.182468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:45.182468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:21:45.182468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:45.182468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:21:45.183036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:45.183036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:21:45.183036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:45.183036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:21:45.183036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:21:45.183036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:21:45.183036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:21:45.183036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:21:45.183036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:21:45.183036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:45.183036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:45.183764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:45.183764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:21:45.183764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:45.183764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:45.183764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:45.183764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:21:45.183764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:45.183764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:21:45.183764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:21:45.183764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:45.183764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:21:45.183764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:45.183764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:21:45.183764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:21:45.184337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:45.184337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:21:45.184337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:45.184337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:21:45.184556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:45.184556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:21:45.184556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:21:45.184556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:21:45.184556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:21:45.184556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:21:45.184556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:45.184556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:21:45.184556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:21:45.184556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:21:45.184556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:21:45.184556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:21:45.184556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:45.184556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:45.184556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:21:45.184556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:21:45.184556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:21:45.185325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:21:45.185325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:21:45.185325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:21:45.185325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:45.185325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:45.185325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:45.185325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:21:45.185325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:45.185325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:45.185325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:45.185325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:21:45.185325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:45.185325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:21:45.185325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:21:45.185325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:45.185325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:21:45.185325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:45.185325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:21:45.185325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:21:45.186278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:45.186278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:21:45.186278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:45.186278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:21:45.186278 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:21:45.186278 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:21:45.186278 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:21:45.186278 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:21:45.186925 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:21:45.186925 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:21:45.186925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:45.186925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:21:45.186925 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:45.186925 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:21:45.186925 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:21:45.186925 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:45.186925 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:21:45.186925 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:45.186925 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:21:45.187734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:45.187734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:21:45.187734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:45.187734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:21:45.187734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:21:45.187734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:21:45.187734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:21:45.187734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:45.187734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:45.187734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:45.187734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:21:45.187734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:45.188151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:45.188151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:45.188151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:21:45.188151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:45.188151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:21:45.188151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:21:45.188151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:45.188151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:21:45.188151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:45.188151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:21:45.188151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:21:45.188151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:45.188151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:21:45.188535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:45.188535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:21:45.188721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:45.188721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:21:45.188721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:21:45.188721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:21:45.188721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:21:45.188721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:21:45.188721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:45.188721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:21:45.188721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:21:45.188721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:21:45.188721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:21:45.188721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:21:45.188721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:45.188721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:45.188721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:21:45.188721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:21:45.189171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:21:45.189171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:21:45.189171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:45.189171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:45.189171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:45.189171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:21:45.189171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:45.189171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:45.189171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:45.189171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:21:45.189171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:45.189171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:21:45.189171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:21:45.189171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:45.189171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:21:45.189171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:45.189171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:21:45.189171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:21:45.189171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:45.189751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:21:45.189751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:45.189751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:21:49.913846 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:21:49.914777 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:21:49.914777 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:21:49.915809 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:21:49.948574 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:21:49.948888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:49.949613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:21:49.949613 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:49.949613 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:21:49.949613 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:21:49.949613 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:49.949613 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:21:49.949613 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:49.949613 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:21:49.950289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:49.950289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:21:49.950289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:49.950289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:21:49.950289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:21:49.950289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:21:49.950289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:21:49.950289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:49.950289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:49.950289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:49.950289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:21:49.950289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:49.950849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:49.950849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:49.950849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:21:49.950849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:49.950849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:21:49.950849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:21:49.950849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:49.950849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:21:49.950849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:49.950849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:21:49.950849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:21:49.950849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:49.950849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:21:49.951347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:49.951347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:21:49.951347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:49.951347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:21:49.951347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:21:49.951347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:21:49.951347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:21:49.951347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:21:49.951347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:49.951347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:21:49.951347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:21:49.951347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:21:49.951347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:21:49.951347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:21:49.951347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:49.951988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:49.951988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:21:49.951988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:21:49.951988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:21:49.951988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:21:49.951988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:49.951988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:49.951988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:49.951988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:21:49.951988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:49.951988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:49.951988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:49.951988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:21:49.951988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:49.951988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:21:49.951988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:21:49.951988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:49.951988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:21:49.951988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:49.952744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:21:49.952744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:21:49.952744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:49.952744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:21:49.952744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:49.952744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:21:50.129637 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:21:50.130267 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:21:50.130512 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:21:50.131623 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:21:50.131988 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:21:50.132239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:50.133132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:21:50.133132 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:50.133132 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:21:50.133132 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:21:50.133132 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:50.133132 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:21:50.133132 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:50.133132 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:21:50.135625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:50.135625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:21:50.135625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:50.135625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:21:50.135625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:21:50.135625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:21:50.135625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:21:50.135625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:50.135625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:50.135625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:50.135625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:21:50.135625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:50.136115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:50.136115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:50.136115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:21:50.136115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:50.136115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:21:50.136115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:21:50.136115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:50.136115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:21:50.136115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:50.136115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:21:50.136115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:21:50.136115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:50.136115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:21:50.136560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:50.136560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:21:50.136647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:50.136647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:21:50.136647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:21:50.136647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:21:50.136647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:21:50.136647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:21:50.136647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:50.136647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:21:50.136647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:21:50.136647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:21:50.136647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:21:50.136647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:21:50.136647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:50.136647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:50.136647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:21:50.136647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:21:50.137218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:21:50.137218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:21:50.137218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:50.137218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:50.137218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:50.137218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:21:50.137218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:50.137218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:50.137218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:50.137218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:21:50.137218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:50.137218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:21:50.137218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:21:50.137218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:50.137218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:21:50.137218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:50.137218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:21:50.137218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:21:50.137218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:50.137870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:21:50.137870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:50.137870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:21:50.150248 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:21:50.150592 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:21:50.150699 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:21:50.150804 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:21:50.151172 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:21:50.151383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:50.151974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:21:50.151974 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:50.151974 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:21:50.151974 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:21:50.151974 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:50.151974 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:21:50.151974 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:50.151974 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:21:50.153716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:50.153716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:21:50.153716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:21:50.153716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:21:50.153716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:50.153716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:21:50.153716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:50.153716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:21:50.153716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:50.153716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:21:50.153716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:50.153716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:21:50.153716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:50.153716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:21:50.153716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:21:50.154207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:50.154207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:21:50.154207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:50.154207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:21:50.154207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:21:50.154207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:50.154207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:21:50.154207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:50.154207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:21:50.155508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:50.155508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:21:50.155508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:50.155508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:21:50.155508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:21:50.155508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:21:50.155508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:21:50.155508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:21:50.155508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:21:50.155508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:21:50.155508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:50.155508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:50.156026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:50.156026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:21:50.156026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:50.156026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:50.156026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:50.156026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:21:50.156026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:50.156026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:21:50.156026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:21:50.156026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:50.156026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:21:50.156026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:50.156026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:21:50.156026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:21:50.156735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:50.156735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:21:50.156735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:50.156735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:21:50.156735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:50.156735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:21:50.156735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:21:50.156735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:21:50.156735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:21:50.156735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:21:50.156735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:50.156735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:21:50.156735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:21:50.156735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:21:50.157374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:21:50.157374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:21:50.157374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:50.157374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:50.157374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:21:50.157374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:21:50.157374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:21:50.157374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:21:50.157374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:21:50.157374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:21:50.157374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:21:50.157374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:50.157374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:50.157374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:50.157374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:21:50.157374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:50.157374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:50.157374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:50.157374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:21:50.158858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:50.158858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:21:50.158858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:21:50.158858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:50.158858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:21:50.158858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:50.158858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:21:50.158858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:21:50.158858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:50.158858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:21:50.158858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:50.158858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:21:50.175619 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:21:50.175913 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:21:50.176038 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:21:50.176143 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:21:50.176679 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:21:50.177035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:50.177839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:21:50.177839 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:50.177839 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:21:50.177839 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:21:50.177839 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:50.177839 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:21:50.177839 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:50.177839 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:21:50.178865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:50.178865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:21:50.178865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:21:50.178865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:21:50.178865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:50.178865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:21:50.178865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:50.178865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:21:50.178865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:50.178865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:21:50.178865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:50.178865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:21:50.178865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:50.178865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:21:50.178865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:21:50.181087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:50.181087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:21:50.181087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:50.181087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:21:50.181087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:21:50.181087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:50.181087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:21:50.181087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:50.181087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:21:50.181087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:50.181087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:21:50.181087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:50.181087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:21:50.181087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:21:50.181777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:21:50.181777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:21:50.181777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:21:50.181777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:21:50.181777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:50.181777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:50.181777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:50.181777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:21:50.181777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:50.181777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:50.181777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:50.181777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:21:50.181777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:50.182462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:21:50.182462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:21:50.182462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:50.182462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:21:50.182462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:50.182462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:21:50.182462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:21:50.182462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:50.182462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:21:50.182462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:50.182462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:21:50.182462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:50.183011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:21:50.183011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:21:50.183011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:21:50.183011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:21:50.183011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:21:50.183011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:50.183011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:21:50.183011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:21:50.183011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:21:50.183011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:21:50.183011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:21:50.183011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:50.183011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:50.183011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:21:50.183011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:21:50.183011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:21:50.183011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:21:50.183011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:21:50.183011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:21:50.184136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:50.184136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:50.184136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:50.184136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:21:50.184136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:50.184136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:50.184136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:50.184136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:21:50.184136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:50.184136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:21:50.184136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:21:50.184136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:50.184136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:21:50.184136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:50.184136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:21:50.184136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:21:50.184136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:50.184136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:21:50.184136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:50.184136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:21:50.185195 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:21:50.185434 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:21:50.185553 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:21:50.185708 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:21:50.186268 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:21:50.186699 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:21:50.186920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:50.187701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:21:50.187701 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:50.187701 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:21:50.187701 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:21:50.187701 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:50.187701 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:21:50.187701 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:50.187701 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:21:50.188522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:50.188522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:21:50.188522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:50.188522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:21:50.188522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:21:50.188522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:21:50.188522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:21:50.188522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:50.188522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:50.188522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:50.188522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:21:50.188522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:50.188847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:50.188847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:50.188847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:21:50.188847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:50.188847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:21:50.188847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:21:50.188847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:50.188847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:21:50.188847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:50.188847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:21:50.188847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:21:50.188847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:50.188847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:21:50.189179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:50.189179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:21:50.189439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:50.189439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:21:50.189439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:21:50.189439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:21:50.189439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:21:50.189439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:21:50.189439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:50.189439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:21:50.189439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:21:50.189439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:21:50.189439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:21:50.189439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:21:50.189439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:50.189439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:50.189439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:21:50.189439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:21:50.189854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:21:50.189854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:21:50.189854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:50.189854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:50.189854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:50.189854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:21:50.189854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:50.189854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:50.189854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:50.189854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:21:50.189854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:50.189854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:21:50.189854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:21:50.189854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:50.189854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:21:50.189854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:50.189854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:21:50.189854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:21:50.189854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:50.190344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:21:50.190344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:50.190344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:21:54.914563 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:21:54.915470 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:21:54.915470 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:21:54.916556 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:21:54.948568 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:21:54.948991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:54.950374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:21:54.950374 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:54.950374 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:21:54.950374 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:21:54.950374 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:54.950374 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:21:54.950374 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:54.950374 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:21:54.950911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:54.950911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:21:54.950911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:54.950911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:21:54.950911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:21:54.950911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:21:54.950911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:21:54.950911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:54.950911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:54.950911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:54.950911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:21:54.950911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:54.951610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:54.951610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:54.951610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:21:54.951610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:54.951610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:21:54.951610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:21:54.951610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:54.951610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:21:54.951610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:54.951610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:21:54.951610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:21:54.951610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:54.951610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:21:54.952304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:54.952304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:21:54.952304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:54.952304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:21:54.952304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:21:54.952304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:21:54.952304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:21:54.952304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:21:54.952304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:54.952304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:21:54.952304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:21:54.952304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:21:54.952304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:21:54.952304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:21:54.952304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:54.953119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:54.953119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:21:54.953119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:21:54.953119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:21:54.953119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:21:54.953119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:54.953119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:54.953119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:54.953119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:21:54.953119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:54.953119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:54.953119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:54.953119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:21:54.953119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:54.953119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:21:54.953119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:21:54.953119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:54.953119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:21:54.953119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:54.953802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:21:54.953802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:21:54.953802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:54.953802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:21:54.953802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:54.953802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:21:55.130258 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:21:55.130647 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:21:55.130859 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:21:55.132009 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:21:55.132232 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:21:55.132490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:55.133390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:21:55.133390 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:55.133390 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:21:55.133390 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:21:55.133390 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:55.133390 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:21:55.133390 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:55.133390 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:21:55.135087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:55.135087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:21:55.135087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:55.135087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:21:55.135087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:21:55.135087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:21:55.135087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:21:55.135087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:55.135087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:55.135087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:55.135087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:21:55.135087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:55.136378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:55.136378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:55.136378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:21:55.136378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:55.136378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:21:55.136378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:21:55.136378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:55.136378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:21:55.136378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:55.136378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:21:55.136378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:21:55.136378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:55.136378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:21:55.137025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:55.137025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:21:55.137025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:55.137025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:21:55.137025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:21:55.137025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:21:55.137025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:21:55.137025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:21:55.137025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:55.137025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:21:55.137025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:21:55.137025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:21:55.137025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:21:55.137025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:21:55.137025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:55.137731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:55.137731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:21:55.137731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:21:55.137731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:21:55.137731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:21:55.137731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:55.137731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:55.137731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:55.137731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:21:55.137731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:55.137731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:55.137731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:55.137731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:21:55.137731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:55.137731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:21:55.137731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:21:55.137731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:55.137731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:21:55.137731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:55.139795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:21:55.139795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:21:55.139795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:55.139795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:21:55.139795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:55.139795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:21:55.150981 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:21:55.151440 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:21:55.151672 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:21:55.151862 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:21:55.152357 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:21:55.152619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:55.153207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:21:55.153207 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:55.153207 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:21:55.153207 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:21:55.153207 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:55.153207 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:21:55.153207 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:55.153207 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:21:55.154523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:55.154523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:21:55.154523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:21:55.154523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:21:55.154523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:55.154523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:21:55.154523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:55.154523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:21:55.154523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:55.154523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:21:55.154523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:55.154523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:21:55.154523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:55.154523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:21:55.154523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:21:55.155562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:55.155562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:21:55.155562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:55.155562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:21:55.155562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:21:55.155562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:55.155562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:21:55.155562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:55.155562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:21:55.157208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:55.157208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:21:55.157208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:55.157208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:21:55.157208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:21:55.157208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:21:55.157208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:21:55.157208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:21:55.157208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:21:55.157208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:21:55.157208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:55.157208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:55.158130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:55.158130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:21:55.158130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:55.158130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:55.158130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:55.158130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:21:55.158130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:55.158130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:21:55.158130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:21:55.158130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:55.158130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:21:55.158130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:55.158130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:21:55.158130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:21:55.158736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:55.158736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:21:55.158736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:55.158736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:21:55.158736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:55.158736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:21:55.158736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:21:55.158736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:21:55.158736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:21:55.158736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:21:55.158736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:55.158736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:21:55.158736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:21:55.158736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:21:55.159768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:21:55.159768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:21:55.159768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:55.159768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:55.159768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:21:55.159768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:21:55.159768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:21:55.159768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:21:55.159768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:21:55.159768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:21:55.159768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:21:55.159768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:55.159768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:55.159768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:55.159768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:21:55.159768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:55.159768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:55.159768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:55.159768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:21:55.160297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:55.160297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:21:55.160297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:21:55.160297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:55.160297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:21:55.160297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:55.160297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:21:55.160297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:21:55.160297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:55.160297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:21:55.160297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:55.160297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:21:55.176146 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:21:55.176541 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:21:55.176825 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:21:55.177060 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:21:55.177675 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:21:55.177941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:55.178527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:21:55.178527 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:55.178527 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:21:55.178527 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:21:55.178527 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:55.178527 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:21:55.178527 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:55.178527 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:21:55.179280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:55.179280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:21:55.179280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:21:55.179280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:21:55.179280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:55.179280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:21:55.179280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:55.179280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:21:55.179280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:55.179280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:21:55.179280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:55.179280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:21:55.179280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:55.179280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:21:55.179280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:21:55.180845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:55.180845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:21:55.180845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:55.180845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:21:55.180845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:21:55.180845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:55.180845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:21:55.180845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:55.180845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:21:55.180845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:55.180845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:21:55.180845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:55.180845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:21:55.180845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:21:55.181362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:21:55.181362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:21:55.181362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:21:55.181362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:21:55.181362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:55.181362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:55.181362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:55.181362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:21:55.181362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:55.181362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:55.181362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:55.181362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:21:55.181362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:55.181720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:21:55.181720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:21:55.181720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:55.181720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:21:55.181720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:55.181720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:21:55.181720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:21:55.181720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:55.181720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:21:55.181720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:55.181720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:21:55.181720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:55.182098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:21:55.182098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:21:55.182098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:21:55.182098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:21:55.182098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:21:55.182098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:55.182098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:21:55.182098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:21:55.182098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:21:55.182098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:21:55.182098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:21:55.182098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:55.182098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:55.182098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:21:55.182098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:21:55.182098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:21:55.182098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:21:55.182098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:21:55.182098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:21:55.182777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:55.182777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:55.182777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:55.182777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:21:55.182777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:55.182777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:55.182777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:55.182777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:21:55.182777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:55.182777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:21:55.182777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:21:55.182777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:55.182777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:21:55.182777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:55.182777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:21:55.182777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:21:55.182777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:55.182777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:21:55.182777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:55.182777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:21:55.185523 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:21:55.185772 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:21:55.185861 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:21:55.186015 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:21:55.186320 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:21:55.186564 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:21:55.186710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:55.187203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:21:55.187203 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:55.187203 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:21:55.187203 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:21:55.187203 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:55.187203 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:21:55.187203 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:55.187203 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:21:55.188103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:55.188103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:21:55.188103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:55.188103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:21:55.188103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:21:55.188103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:21:55.188103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:21:55.188103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:55.188103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:55.188103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:55.188103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:21:55.188103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:55.188926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:55.188926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:55.188926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:21:55.188926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:55.188926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:21:55.188926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:21:55.188926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:55.188926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:21:55.188926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:55.188926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:21:55.188926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:21:55.188926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:55.188926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:21:55.189836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:55.189836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:21:55.189836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:55.189836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:21:55.189836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:21:55.189836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:21:55.189836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:21:55.189836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:21:55.189836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:55.189836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:21:55.189836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:21:55.189836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:21:55.189836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:21:55.189836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:21:55.189836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:55.190702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:55.190702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:21:55.190702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:21:55.190702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:21:55.190702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:21:55.190702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:55.190702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:55.190702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:55.190702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:21:55.190702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:55.190702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:55.190702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:55.190702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:21:55.190702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:55.190702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:21:55.190702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:21:55.190702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:55.190702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:21:55.190702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:55.191597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:21:55.191597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:21:55.191597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:55.191597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:21:55.191597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:55.191597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:21:59.915379 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:21:59.916235 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:21:59.916235 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:21:59.917067 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:21:59.952769 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:21:59.953153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:59.953914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:21:59.953914 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:59.953914 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:21:59.953914 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:21:59.953914 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:59.953914 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:21:59.953914 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:59.953914 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:21:59.954630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:59.954630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:21:59.954630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:59.954630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:21:59.954630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:21:59.954630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:21:59.954630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:21:59.954630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:59.954630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:59.954630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:59.954630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:21:59.954630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:59.955326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:21:59.955326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:59.955326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:21:59.955326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:59.955326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:21:59.955326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:21:59.955326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:59.955326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:21:59.955326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:59.955326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:21:59.955326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:21:59.955326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:59.955326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:21:59.956009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:59.956009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:21:59.956009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:59.956009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:21:59.956009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:21:59.956009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:21:59.956009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:21:59.956009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:21:59.956009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:59.956009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:21:59.956009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:21:59.956009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:21:59.956009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:21:59.956009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:21:59.956009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:21:59.956775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:21:59.956775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:21:59.956775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:21:59.956775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:21:59.956775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:21:59.956775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:59.956775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:59.956775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:21:59.956775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:21:59.956775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:21:59.956775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:21:59.956775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:21:59.956775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:21:59.956775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:21:59.956775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:21:59.956775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:21:59.956775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:21:59.956775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:21:59.956775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:21:59.957725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:21:59.957725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:21:59.957725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:21:59.957725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:21:59.957725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:21:59.957725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:22:00.130754 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:22:00.131154 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:22:00.131377 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:22:00.132396 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:22:00.132698 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:22:00.132829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:00.133619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:22:00.133619 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:00.133619 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:22:00.133619 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:22:00.133619 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:00.133619 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:22:00.133619 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:00.133619 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:22:00.135047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:00.135047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:22:00.135047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:00.135047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:22:00.135047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:22:00.135047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:22:00.135047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:22:00.135047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:00.135047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:00.135047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:00.135047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:22:00.135047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:00.135754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:00.135754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:00.135754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:22:00.135754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:00.135754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:22:00.135754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:22:00.135754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:00.135754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:22:00.135754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:00.135754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:22:00.135754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:22:00.135754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:00.135754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:22:00.136299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:00.136299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:22:00.136574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:00.136574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:22:00.136574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:22:00.136574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:22:00.136574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:22:00.136574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:22:00.136574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:00.136574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:22:00.136574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:22:00.136574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:22:00.136574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:22:00.136574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:22:00.136574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:00.136574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:00.136574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:22:00.136574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:22:00.137344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:22:00.137344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:22:00.137344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:00.137344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:00.137344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:00.137344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:22:00.137344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:00.137344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:00.137344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:00.137344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:22:00.137344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:00.137344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:22:00.137344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:22:00.137344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:00.137344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:22:00.137344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:00.137344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:22:00.137344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:22:00.137344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:00.138427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:22:00.138427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:00.138427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:22:00.151511 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:22:00.151771 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:22:00.151925 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:22:00.152128 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:22:00.152637 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:22:00.152907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:00.153615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:22:00.153615 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:00.153615 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:22:00.153615 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:22:00.153615 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:00.153615 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:22:00.153615 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:00.153615 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:22:00.154747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:00.154747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:22:00.154747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:22:00.154747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:22:00.154747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:00.154747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:22:00.154747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:00.154747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:22:00.154747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:00.154747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:22:00.154747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:00.154747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:22:00.154747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:00.154747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:22:00.154747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:22:00.155281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:00.155281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:22:00.155281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:00.155281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:22:00.155281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:22:00.155281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:00.155281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:22:00.155281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:00.155281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:22:00.156065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:00.156065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:22:00.156065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:00.156065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:22:00.156065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:22:00.156065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:22:00.156065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:22:00.156065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:22:00.156065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:22:00.156065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:22:00.156065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:00.156065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:00.156473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:00.156473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:22:00.156473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:00.156473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:00.156473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:00.156473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:22:00.156473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:00.156473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:22:00.156473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:22:00.156473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:00.156473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:22:00.156473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:00.156473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:22:00.156473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:22:00.156849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:00.156849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:22:00.156849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:00.156849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:22:00.157113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:00.157113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:22:00.157113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:22:00.157113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:22:00.157113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:22:00.157113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:22:00.157113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:00.157113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:22:00.157113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:22:00.157113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:22:00.157113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:22:00.157113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:22:00.157113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:00.157113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:00.157113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:22:00.157113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:22:00.157581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:22:00.157581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:22:00.157581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:22:00.157581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:22:00.157581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:22:00.157581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:00.157581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:00.157581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:00.157581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:22:00.157581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:00.157581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:00.157581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:00.157581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:22:00.157581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:00.157581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:22:00.157581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:22:00.157581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:00.157581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:22:00.158085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:00.158085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:22:00.158085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:22:00.158085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:00.158085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:22:00.158085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:00.158085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:22:00.176555 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:22:00.176836 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:22:00.176999 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:22:00.177190 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:22:00.178088 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:22:00.178584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:00.179576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:22:00.179576 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:00.179576 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:22:00.179576 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:22:00.179576 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:00.179576 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:22:00.179576 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:00.179576 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:22:00.180715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:00.180715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:22:00.180715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:22:00.180715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:22:00.180715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:00.180715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:22:00.180715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:00.180715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:22:00.180715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:00.180715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:22:00.180715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:00.180715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:22:00.180715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:00.180715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:22:00.180715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:22:00.181582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:00.181582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:22:00.181582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:00.181582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:22:00.181582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:22:00.181582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:00.181582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:22:00.181582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:00.181582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:22:00.182067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:00.182067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:22:00.182067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:00.182067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:22:00.182067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:22:00.182067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:22:00.182067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:22:00.182067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:22:00.182067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:22:00.182067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:00.182067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:00.182742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:00.182742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:22:00.182742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:00.182742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:00.182742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:00.182742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:22:00.182742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:00.182742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:22:00.182742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:22:00.182742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:00.182742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:22:00.182742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:00.182742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:22:00.182742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:22:00.183493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:00.183493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:22:00.183493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:00.183493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:22:00.183704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:00.183704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:22:00.183704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:22:00.183704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:22:00.183704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:22:00.183704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:22:00.183704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:00.183704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:22:00.183704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:22:00.183704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:22:00.183704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:22:00.183704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:22:00.183704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:00.183704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:00.183704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:22:00.183704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:22:00.183704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:22:00.184841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:22:00.184841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:22:00.184841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:22:00.184841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:00.184841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:00.184841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:00.184841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:22:00.184841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:00.184841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:00.184841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:00.184841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:22:00.184841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:00.184841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:22:00.184841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:22:00.184841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:00.184841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:22:00.184841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:00.184841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:22:00.184841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:22:00.185949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:00.185949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:22:00.185949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:00.185949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:22:00.185949 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:22:00.186281 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:22:00.186281 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:22:00.186471 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:22:00.186908 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:22:00.187260 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:22:00.187571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:00.188190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:22:00.188190 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:00.188190 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:22:00.188190 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:22:00.188190 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:00.188190 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:22:00.188190 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:00.188190 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:22:00.188985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:00.188985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:22:00.188985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:00.188985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:22:00.188985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:22:00.188985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:22:00.188985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:22:00.188985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:00.188985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:00.188985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:00.188985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:22:00.188985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:00.189507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:00.189507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:00.189507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:22:00.189507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:00.189507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:22:00.189507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:22:00.189507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:00.189507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:22:00.189507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:00.189507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:22:00.189507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:22:00.189507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:00.189507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:22:00.189994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:00.189994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:22:00.189994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:00.189994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:22:00.189994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:22:00.189994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:22:00.189994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:22:00.189994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:22:00.189994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:00.189994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:22:00.189994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:22:00.189994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:22:00.189994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:22:00.189994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:22:00.189994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:00.190505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:00.190505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:22:00.190505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:22:00.190505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:22:00.190505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:22:00.190505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:00.190505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:00.190505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:00.190505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:22:00.190505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:00.190505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:00.190505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:00.190505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:22:00.190505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:00.190505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:22:00.190505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:22:00.190505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:00.190505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:22:00.190505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:00.191042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:22:00.191042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:22:00.191042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:00.191042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:22:00.191042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:00.191042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:22:04.915742 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:22:04.918834 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:22:04.918834 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:22:04.918834 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:22:04.954395 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:22:04.955237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:04.956829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:22:04.956829 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:04.956829 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:22:04.956829 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:22:04.956829 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:04.956829 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:22:04.956829 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:04.956829 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:22:04.959632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:04.959632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:22:04.959632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:04.959632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:22:04.959632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:22:04.959632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:22:04.959632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:22:04.959632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:04.959632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:04.959632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:04.959632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:22:04.959632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:04.960457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:04.960457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:04.960457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:22:04.960457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:04.960457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:22:04.960457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:22:04.960457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:04.960457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:22:04.960457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:04.960457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:22:04.960457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:22:04.960457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:04.960457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:22:04.961100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:04.961100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:22:04.961100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:04.961100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:22:04.961100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:22:04.961100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:22:04.961100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:22:04.961100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:22:04.961100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:04.961100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:22:04.961100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:22:04.961100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:22:04.961100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:22:04.961100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:22:04.961100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:04.961889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:04.961889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:22:04.961889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:22:04.961889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:22:04.961889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:22:04.961889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:04.961889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:04.961889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:04.961889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:22:04.961889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:04.961889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:04.961889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:04.961889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:22:04.961889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:04.961889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:22:04.961889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:22:04.961889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:04.961889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:22:04.961889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:04.962822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:22:04.962822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:22:04.962822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:04.962822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:22:04.962822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:04.962822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:22:05.131367 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:22:05.131593 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:22:05.131779 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:22:05.132549 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:22:05.132742 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:22:05.132882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:05.133726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:22:05.133726 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:05.133726 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:22:05.133726 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:22:05.133726 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:05.133726 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:22:05.133726 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:05.133726 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:22:05.134829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:05.134829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:22:05.134829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:05.134829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:22:05.134829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:22:05.134829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:22:05.134829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:22:05.134829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:05.134829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:05.134829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:05.134829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:22:05.134829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:05.135236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:05.135236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:05.135236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:22:05.135236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:05.135236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:22:05.135236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:22:05.135236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:05.135236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:22:05.135236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:05.135236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:22:05.135236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:22:05.135236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:05.135236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:22:05.135735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:05.135735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:22:05.136376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:05.136376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:22:05.136376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:22:05.136376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:22:05.136376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:22:05.136376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:22:05.136376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:05.136376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:22:05.136376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:22:05.136376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:22:05.136376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:22:05.136376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:22:05.136376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:05.136376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:05.136376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:22:05.136376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:22:05.137055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:22:05.137055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:22:05.137055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:05.137055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:05.137055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:05.137055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:22:05.137055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:05.137055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:05.137055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:05.137055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:22:05.137055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:05.137055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:22:05.137055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:22:05.137055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:05.137055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:22:05.137055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:05.137055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:22:05.137055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:22:05.137055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:05.137834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:22:05.137834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:05.137834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:22:05.151897 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:22:05.152238 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:22:05.152422 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:22:05.152643 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:22:05.153115 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:22:05.153464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:05.154231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:22:05.154231 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:05.154231 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:22:05.154231 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:22:05.154231 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:05.154231 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:22:05.154231 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:05.154231 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:22:05.156270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:05.156270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:22:05.156270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:22:05.156270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:22:05.156270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:05.156270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:22:05.156270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:05.156270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:22:05.156270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:05.156270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:22:05.156270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:05.156270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:22:05.156270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:05.156270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:22:05.156270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:22:05.156990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:05.156990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:22:05.156990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:05.156990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:22:05.156990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:22:05.156990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:05.156990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:22:05.156990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:05.156990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:22:05.157662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:05.157662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:22:05.157662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:05.157662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:22:05.157662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:22:05.157662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:22:05.157662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:22:05.157662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:22:05.157662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:22:05.157662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:22:05.157662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:05.157662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:05.158260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:05.158260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:22:05.158260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:05.158260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:05.158260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:05.158260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:22:05.158260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:05.158260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:22:05.158260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:22:05.158260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:05.158260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:22:05.158260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:05.158260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:22:05.158260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:22:05.158839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:05.158839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:22:05.158839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:05.158839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:22:05.159180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:05.159180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:22:05.159180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:22:05.159180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:22:05.159180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:22:05.159180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:22:05.159180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:05.159180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:22:05.159180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:22:05.159180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:22:05.159180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:22:05.159180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:22:05.159180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:05.159180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:05.159180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:22:05.159180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:22:05.159816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:22:05.159816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:22:05.159816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:22:05.159816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:22:05.159816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:22:05.159816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:05.159816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:05.159816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:05.159816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:22:05.159816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:05.159816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:05.159816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:05.159816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:22:05.159816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:05.159816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:22:05.159816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:22:05.159816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:05.159816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:22:05.160574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:05.160574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:22:05.160574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:22:05.160574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:05.160574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:22:05.160574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:05.160574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:22:05.176974 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:22:05.177393 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:22:05.177393 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:22:05.177648 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:22:05.178813 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:22:05.179327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:05.180431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:22:05.180431 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:05.180431 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:22:05.180431 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:22:05.180431 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:05.180431 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:22:05.180431 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:05.180431 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:22:05.181703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:05.181703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:22:05.181703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:22:05.181703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:22:05.181703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:05.181703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:22:05.181703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:05.181703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:22:05.181703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:05.181703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:22:05.181703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:05.181703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:22:05.181703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:05.181703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:22:05.181703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:22:05.182714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:05.182714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:22:05.182714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:05.182714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:22:05.182714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:22:05.182714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:05.182714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:22:05.182714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:05.182714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:22:05.183276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:05.183276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:22:05.183276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:05.183276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:22:05.183276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:22:05.183276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:22:05.183276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:22:05.183276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:22:05.183276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:22:05.183276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:05.183276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:05.184063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:05.184063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:22:05.184063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:05.184063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:05.184063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:05.184063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:22:05.184063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:05.184063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:22:05.184063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:22:05.184063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:05.184063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:22:05.184063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:05.184063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:22:05.184063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:22:05.184942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:05.184942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:22:05.184942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:05.184942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:22:05.184942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:05.184942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:22:05.184942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:22:05.184942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:22:05.184942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:22:05.184942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:22:05.184942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:05.184942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:22:05.184942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:22:05.184942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:22:05.186029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:22:05.186029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:22:05.186029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:05.186029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:05.186029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:22:05.186029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:22:05.186029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:22:05.186029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:22:05.186029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:22:05.186029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:22:05.186029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:05.186029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:05.186029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:05.186029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:22:05.186029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:05.186029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:05.186029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:05.186029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:22:05.186029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:05.186973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:22:05.186973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:22:05.186973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:05.186973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:22:05.186973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:05.186973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:22:05.186973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:22:05.186973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:05.186973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:22:05.186973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:05.186973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:22:05.186973 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:22:05.186973 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:22:05.187562 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:22:05.187562 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:22:05.187562 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:22:05.187717 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:22:05.187976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:05.188567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:22:05.188567 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:05.188567 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:22:05.188567 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:22:05.188567 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:05.188567 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:22:05.188567 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:05.188567 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:22:05.189387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:05.189387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:22:05.189387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:05.189387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:22:05.189387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:22:05.189387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:22:05.189387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:22:05.189387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:05.189387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:05.189387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:05.189387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:22:05.189387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:05.189974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:05.189974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:05.189974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:22:05.189974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:05.189974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:22:05.189974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:22:05.189974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:05.189974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:22:05.189974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:05.189974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:22:05.189974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:22:05.189974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:05.189974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:22:05.190542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:05.190542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:22:05.190542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:05.190542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:22:05.190542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:22:05.190542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:22:05.190542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:22:05.190542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:22:05.190542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:05.190542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:22:05.190542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:22:05.190542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:22:05.190542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:22:05.190542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:22:05.190542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:05.191142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:05.191142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:22:05.191142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:22:05.191142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:22:05.191142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:22:05.191142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:05.191142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:05.191142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:05.191142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:22:05.191142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:05.191142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:05.191142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:05.191142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:22:05.191142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:05.191142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:22:05.191142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:22:05.191142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:05.191142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:22:05.191142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:05.191901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:22:05.191901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:22:05.191901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:05.191901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:22:05.191901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:05.191901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:22:09.917014 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:22:09.918048 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:22:09.918048 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:22:09.919242 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:22:09.955730 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:22:09.956066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:09.956981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:22:09.956981 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:09.956981 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:22:09.956981 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:22:09.956981 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:09.956981 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:22:09.956981 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:09.956981 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:22:09.958293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:09.958293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:22:09.958293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:09.958293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:22:09.958293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:22:09.958293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:22:09.958293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:22:09.958293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:09.958293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:09.958293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:09.958293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:22:09.958293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:09.959033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:09.959033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:09.959033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:22:09.959033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:09.959033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:22:09.959033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:22:09.959033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:09.959033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:22:09.959033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:09.959033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:22:09.959033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:22:09.959033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:09.959033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:22:09.959927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:09.959927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:22:09.959927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:09.959927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:22:09.959927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:22:09.959927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:22:09.959927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:22:09.959927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:22:09.959927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:09.959927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:22:09.959927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:22:09.959927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:22:09.959927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:22:09.959927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:22:09.959927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:09.961154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:09.961154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:22:09.961154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:22:09.961154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:22:09.961154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:22:09.961154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:09.961154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:09.961154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:09.961154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:22:09.961154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:09.961154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:09.961154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:09.961154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:22:09.961154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:09.961154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:22:09.961154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:22:09.961154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:09.961154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:22:09.961154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:09.962320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:22:09.962320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:22:09.962320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:09.962320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:22:09.962320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:09.962320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:22:10.132027 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:22:10.132677 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:22:10.132922 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:22:10.134176 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:22:10.134574 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:22:10.134877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:10.135789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:22:10.135789 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:10.135789 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:22:10.135789 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:22:10.135789 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:10.135789 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:22:10.135789 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:10.135789 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:22:10.137602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:10.137602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:22:10.137602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:10.137602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:22:10.137602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:22:10.137602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:22:10.137602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:22:10.137602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:10.137602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:10.137602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:10.137602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:22:10.137602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:10.138378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:10.138378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:10.138378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:22:10.138378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:10.138378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:22:10.138378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:22:10.138378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:10.138378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:22:10.138378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:10.138378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:22:10.138378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:22:10.138378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:10.138378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:22:10.139007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:10.139007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:22:10.139007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:10.139007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:22:10.139007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:22:10.139007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:22:10.139007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:22:10.139007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:22:10.139007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:10.139007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:22:10.139007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:22:10.139007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:22:10.139007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:22:10.139007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:22:10.139007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:10.139997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:10.139997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:22:10.139997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:22:10.139997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:22:10.139997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:22:10.139997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:10.139997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:10.139997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:10.139997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:22:10.139997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:10.139997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:10.139997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:10.139997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:22:10.139997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:10.139997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:22:10.139997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:22:10.139997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:10.139997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:22:10.139997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:10.141445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:22:10.141445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:22:10.141445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:10.141445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:22:10.141445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:10.141445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:22:10.152551 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:22:10.152932 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:22:10.153061 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:22:10.153178 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:22:10.153669 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:22:10.153948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:10.154737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:22:10.154737 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:10.154737 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:22:10.154737 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:22:10.154737 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:10.154737 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:22:10.154737 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:10.154737 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:22:10.156089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:10.156089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:22:10.156089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:22:10.156089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:22:10.156089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:10.156089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:22:10.156089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:10.156089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:22:10.156089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:10.156089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:22:10.156089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:10.156089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:22:10.156089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:10.156089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:22:10.156089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:22:10.156904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:10.156904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:22:10.156904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:10.156904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:22:10.156904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:22:10.156904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:10.156904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:22:10.156904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:10.156904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:22:10.157707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:10.157707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:22:10.157707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:10.157707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:22:10.157707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:22:10.157707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:22:10.157707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:22:10.157707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:22:10.157707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:22:10.157707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:22:10.157707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:10.157707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:10.158363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:10.158363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:22:10.158363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:10.158363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:10.158363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:10.158363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:22:10.158363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:10.158363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:22:10.158363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:22:10.158363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:10.158363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:22:10.158363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:10.158363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:22:10.158363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:22:10.159079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:10.159079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:22:10.159079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:10.159079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:22:10.159496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:10.159496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:22:10.159496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:22:10.159496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:22:10.159496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:22:10.159496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:22:10.159496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:10.159496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:22:10.159496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:22:10.159496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:22:10.159496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:22:10.159496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:22:10.159496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:10.159496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:10.159496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:22:10.159496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:22:10.160303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:22:10.160303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:22:10.160303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:22:10.160303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:22:10.160303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:22:10.160303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:10.160303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:10.160303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:10.160303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:22:10.160303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:10.160303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:10.160303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:10.160303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:22:10.160303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:10.160303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:22:10.160303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:22:10.160303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:10.160303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:22:10.161144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:10.161144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:22:10.161144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:22:10.161144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:10.161144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:22:10.161144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:10.161144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:22:10.177576 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:22:10.177867 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:22:10.178054 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:22:10.178314 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:22:10.178916 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:22:10.179532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:10.180237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:22:10.180237 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:10.180237 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:22:10.180237 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:22:10.180237 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:10.180237 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:22:10.180237 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:10.180237 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:22:10.182340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:10.182340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:22:10.182340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:22:10.182340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:22:10.182340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:10.182340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:22:10.182340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:10.182340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:22:10.182340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:10.182340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:22:10.182340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:10.182340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:22:10.182340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:10.182340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:22:10.182340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:22:10.183047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:10.183047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:22:10.183047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:10.183047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:22:10.183047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:22:10.183047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:10.183047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:22:10.183047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:10.183047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:22:10.183047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:10.183047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:22:10.183047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:10.183047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:22:10.183047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:22:10.183715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:22:10.183715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:22:10.183715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:22:10.183715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:22:10.183715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:10.183715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:10.183715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:10.183715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:22:10.183715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:10.183715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:10.183715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:10.183715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:22:10.183715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:10.184273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:22:10.184273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:22:10.184273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:10.184273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:22:10.184273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:10.184273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:22:10.184273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:22:10.184273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:10.184273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:22:10.184273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:10.184273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:22:10.185113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:10.185113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:22:10.185113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:22:10.185113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:22:10.185113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:22:10.185113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:22:10.185113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:10.185113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:22:10.185113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:22:10.185113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:22:10.185113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:22:10.185113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:22:10.185113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:10.185113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:10.185113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:22:10.185113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:22:10.185113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:22:10.186352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:22:10.186352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:22:10.186352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:22:10.186352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:10.186352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:10.186352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:10.186352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:22:10.186352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:10.186352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:10.186352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:10.186352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:22:10.186352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:10.186352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:22:10.186352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:22:10.186352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:10.186352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:22:10.186352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:10.186352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:22:10.186352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:22:10.188022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:10.188022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:22:10.188022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:10.188022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:22:10.188022 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:22:10.188022 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:22:10.188022 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:22:10.188022 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:22:10.188893 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:22:10.188893 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:22:10.188893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:10.188893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:22:10.188893 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:10.188893 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:22:10.188893 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:22:10.188893 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:10.188893 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:22:10.188893 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:10.188893 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:22:10.189869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:10.189869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:22:10.189869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:10.189869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:22:10.189869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:22:10.189869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:22:10.189869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:22:10.189869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:10.189869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:10.189869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:10.189869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:22:10.189869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:10.190451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:10.190451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:10.190451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:22:10.190451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:10.190451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:22:10.190451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:22:10.190451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:10.190451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:22:10.190451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:10.190451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:22:10.190451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:22:10.190451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:10.190451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:22:10.190887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:10.190887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:22:10.191014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:10.191014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:22:10.191014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:22:10.191014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:22:10.191014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:22:10.191014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:22:10.191014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:10.191014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:22:10.191014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:22:10.191014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:22:10.191014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:22:10.191014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:22:10.191014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:10.191014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:10.191014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:22:10.191014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:22:10.191369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:22:10.191369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:22:10.191369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:10.191369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:10.191369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:10.191369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:22:10.191369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:10.191369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:10.191369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:10.191369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:22:10.191369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:10.191369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:22:10.191369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:22:10.191369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:10.191369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:22:10.191369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:10.191369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:22:10.191369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:22:10.191369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:10.191895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:22:10.191895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:10.191895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:22:14.917778 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:22:14.918513 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:22:14.918594 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:22:14.919926 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:22:14.955045 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:22:14.955235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:14.955639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:22:14.955639 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:14.955639 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:22:14.955639 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:22:14.955639 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:14.955639 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:22:14.955639 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:14.955639 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:22:14.956271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:14.956271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:22:14.956271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:14.956271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:22:14.956271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:22:14.956271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:22:14.956271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:22:14.956271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:14.956271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:14.956271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:14.956271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:22:14.956271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:14.956776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:14.956776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:14.956776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:22:14.956776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:14.956776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:22:14.956776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:22:14.956776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:14.956776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:22:14.956776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:14.956776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:22:14.956776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:22:14.956776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:14.956776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:22:14.957549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:14.957549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:22:14.957549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:14.957549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:22:14.957549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:22:14.957549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:22:14.957549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:22:14.957549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:22:14.957549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:14.957549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:22:14.957549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:22:14.957549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:22:14.957549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:22:14.957549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:22:14.957549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:14.957984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:14.957984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:22:14.957984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:22:14.957984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:22:14.957984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:22:14.957984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:14.957984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:14.957984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:14.957984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:22:14.957984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:14.957984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:14.957984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:14.957984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:22:14.957984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:14.957984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:22:14.957984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:22:14.957984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:14.957984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:22:14.957984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:14.958521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:22:14.958521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:22:14.958521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:14.958521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:22:14.958521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:14.958521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:22:15.132710 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:22:15.133167 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:22:15.133348 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:22:15.134262 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:22:15.134535 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:22:15.134764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:15.135642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:22:15.135642 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:15.135642 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:22:15.135642 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:22:15.135642 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:15.135642 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:22:15.135642 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:15.135642 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:22:15.137091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:15.137091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:22:15.137091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:15.137091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:22:15.137091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:22:15.137091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:22:15.137091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:22:15.137091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:15.137091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:15.137091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:15.137091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:22:15.137091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:15.137701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:15.137701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:15.137701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:22:15.137701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:15.137701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:22:15.137701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:22:15.137701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:15.137701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:22:15.137701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:15.137701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:22:15.137701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:22:15.137701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:15.137701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:22:15.138194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:15.138194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:22:15.138825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:15.138825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:22:15.138825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:22:15.138825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:22:15.138825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:22:15.138825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:22:15.138825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:15.138825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:22:15.138825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:22:15.138825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:22:15.138825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:22:15.138825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:22:15.138825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:15.138825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:15.138825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:22:15.138825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:22:15.139556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:22:15.139556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:22:15.139556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:15.139556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:15.139556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:15.139556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:22:15.139556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:15.139556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:15.139556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:15.139556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:22:15.139556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:15.139556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:22:15.139556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:22:15.139556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:15.139556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:22:15.139556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:15.139556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:22:15.139556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:22:15.139556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:15.140331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:22:15.140331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:15.140331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:22:15.153379 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:22:15.153857 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:22:15.154040 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:22:15.154272 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:22:15.154649 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:22:15.154959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:15.155740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:22:15.155740 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:15.155740 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:22:15.155740 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:22:15.155740 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:15.155740 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:22:15.155740 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:15.155740 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:22:15.157034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:15.157034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:22:15.157034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:22:15.157034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:22:15.157034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:15.157034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:22:15.157034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:15.157034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:22:15.157034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:15.157034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:22:15.157034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:15.157034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:22:15.157034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:15.157034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:22:15.157034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:22:15.158113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:15.158113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:22:15.158113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:15.158113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:22:15.158113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:22:15.158113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:15.158113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:22:15.158113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:15.158113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:22:15.158377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:15.158377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:22:15.158377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:15.158377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:22:15.158377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:22:15.158377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:22:15.158377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:22:15.158377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:22:15.158377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:22:15.158377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:22:15.158377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:15.158377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:15.158746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:15.158746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:22:15.158746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:15.158746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:15.158746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:15.158746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:22:15.158746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:15.158746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:22:15.158746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:22:15.158746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:15.158746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:22:15.158746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:15.158746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:22:15.158746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:22:15.159085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:15.159085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:22:15.159085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:15.159085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:22:15.160160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:15.160160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:22:15.160160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:22:15.160160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:22:15.160160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:22:15.160160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:22:15.160160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:15.160160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:22:15.160160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:22:15.160160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:22:15.160160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:22:15.160160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:22:15.160160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:15.160160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:15.160160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:22:15.160160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:22:15.161264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:22:15.161264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:22:15.161264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:22:15.161264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:22:15.161264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:22:15.161264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:15.161264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:15.161264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:15.161264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:22:15.161264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:15.161264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:15.161264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:15.161264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:22:15.161264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:15.161264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:22:15.161264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:22:15.161264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:15.161264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:22:15.161767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:15.161767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:22:15.161767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:22:15.161767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:15.161767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:22:15.161767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:15.161767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:22:15.178063 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:22:15.178537 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:22:15.178666 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:22:15.178890 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:22:15.179793 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:22:15.180310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:15.181578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:22:15.181578 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:15.181578 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:22:15.181578 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:22:15.181578 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:15.181578 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:22:15.181578 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:15.181578 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:22:15.182911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:15.182911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:22:15.182911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:22:15.182911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:22:15.182911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:15.182911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:22:15.182911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:15.182911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:22:15.182911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:15.182911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:22:15.182911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:15.182911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:22:15.182911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:15.182911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:22:15.182911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:22:15.183525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:15.183525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:22:15.183525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:15.183525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:22:15.183525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:22:15.183525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:15.183525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:22:15.183525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:15.183525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:22:15.184826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:15.184826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:22:15.184826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:15.184826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:22:15.184826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:22:15.184826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:22:15.184826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:22:15.184826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:22:15.184826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:22:15.184826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:15.184826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:15.185197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:15.185197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:22:15.185197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:15.185197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:15.185197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:15.185197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:22:15.185197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:15.185197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:22:15.185197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:22:15.185197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:15.185197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:22:15.185197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:15.185197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:22:15.185197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:22:15.185576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:15.185576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:22:15.185576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:15.185576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:22:15.186560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:15.186560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:22:15.186560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:22:15.186560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:22:15.186560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:22:15.186560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:22:15.186560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:15.186560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:22:15.186560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:22:15.186560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:22:15.186560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:22:15.186560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:22:15.186560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:15.186560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:15.186560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:22:15.186560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:22:15.186560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:22:15.187030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:22:15.187030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:22:15.187030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:22:15.187030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:15.187030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:15.187030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:15.187030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:22:15.187030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:15.187030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:15.187030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:15.187030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:22:15.187030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:15.187030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:22:15.187030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:22:15.187030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:15.187030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:22:15.187030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:15.187030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:22:15.187030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:22:15.187683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:15.187683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:22:15.187683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:15.187683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:22:15.187683 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:22:15.187843 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:22:15.187843 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:22:15.188014 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:22:15.188465 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:22:15.188813 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:22:15.189035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:15.189721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:22:15.189721 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:15.189721 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:22:15.189721 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:22:15.189721 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:15.189721 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:22:15.189721 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:15.189721 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:22:15.190552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:15.190552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:22:15.190552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:15.190552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:22:15.190552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:22:15.190552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:22:15.190552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:22:15.190552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:15.190552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:15.190552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:15.190552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:22:15.190552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:15.190912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:15.190912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:15.190912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:22:15.190912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:15.190912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:22:15.190912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:22:15.190912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:15.190912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:22:15.190912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:15.190912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:22:15.190912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:22:15.190912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:15.190912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:22:15.191298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:15.191298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:22:15.191561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:15.191561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:22:15.191561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:22:15.191561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:22:15.191561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:22:15.191561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:22:15.191561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:15.191561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:22:15.191561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:22:15.191561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:22:15.191561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:22:15.191561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:22:15.191561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:15.191561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:15.191561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:22:15.191561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:22:15.192010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:22:15.192010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:22:15.192010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:15.192010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:15.192010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:15.192010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:22:15.192010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:15.192010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:15.192010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:15.192010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:22:15.192010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:15.192010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:22:15.192010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:22:15.192010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:15.192010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:22:15.192010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:15.192010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:22:15.192010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:22:15.192010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:15.192533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:22:15.192533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:15.192533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:22:19.918508 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:22:19.919300 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:22:19.919657 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:22:19.921312 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:22:19.955274 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:22:19.955710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:19.956876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:22:19.956876 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:19.956876 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:22:19.956876 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:22:19.956876 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:19.956876 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:22:19.956876 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:19.956876 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:22:19.957590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:19.957590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:22:19.957590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:19.957590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:22:19.957590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:22:19.957590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:22:19.957590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:22:19.957590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:19.957590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:19.957590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:19.957590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:22:19.957590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:19.958285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:19.958285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:19.958285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:22:19.958285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:19.958285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:22:19.958285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:22:19.958285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:19.958285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:22:19.958285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:19.958285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:22:19.958285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:22:19.958285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:19.958285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:22:19.958967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:19.958967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:22:19.958967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:19.958967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:22:19.958967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:22:19.958967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:22:19.958967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:22:19.958967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:22:19.958967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:19.958967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:22:19.958967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:22:19.958967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:22:19.958967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:22:19.958967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:22:19.958967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:19.959746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:19.959746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:22:19.959746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:22:19.959746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:22:19.959746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:22:19.959746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:19.959746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:19.959746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:19.959746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:22:19.959746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:19.959746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:19.959746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:19.959746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:22:19.959746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:19.959746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:22:19.959746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:22:19.959746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:19.959746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:22:19.959746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:19.960780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:22:19.960780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:22:19.960780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:19.960780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:22:19.960780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:19.960780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:22:20.133327 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:22:20.133915 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:22:20.134135 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:22:20.135365 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:22:20.135755 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:22:20.136011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:20.136985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:22:20.136985 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:20.136985 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:22:20.136985 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:22:20.136985 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:20.136985 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:22:20.136985 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:20.136985 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:22:20.138590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:20.138590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:22:20.138590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:20.138590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:22:20.138590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:22:20.138590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:22:20.138590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:22:20.138590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:20.138590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:20.138590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:20.138590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:22:20.138590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:20.139173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:20.139173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:20.139173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:22:20.139173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:20.139173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:22:20.139173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:22:20.139173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:20.139173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:22:20.139173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:20.139173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:22:20.139173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:22:20.139173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:20.139173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:22:20.139787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:20.139787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:22:20.140340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:20.140340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:22:20.140340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:22:20.140340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:22:20.140340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:22:20.140340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:22:20.140340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:20.140340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:22:20.140340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:22:20.140340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:22:20.140340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:22:20.140340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:22:20.140340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:20.140340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:20.140340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:22:20.140340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:22:20.141062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:22:20.141062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:22:20.141062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:20.141062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:20.141062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:20.141062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:22:20.141062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:20.141062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:20.141062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:20.141062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:22:20.141062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:20.141062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:22:20.141062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:22:20.141062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:20.141062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:22:20.141062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:20.141062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:22:20.141062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:22:20.141062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:20.141618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:22:20.141618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:20.141618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:22:20.154102 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:22:20.154700 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:22:20.154961 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:22:20.155073 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:22:20.155654 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:22:20.156005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:20.156936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:22:20.156936 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:20.156936 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:22:20.156936 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:22:20.156936 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:20.156936 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:22:20.156936 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:20.156936 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:22:20.158438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:20.158438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:22:20.158438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:22:20.158438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:22:20.158438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:20.158438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:22:20.158438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:20.158438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:22:20.158438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:20.158438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:22:20.158438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:20.158438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:22:20.158438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:20.158438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:22:20.158438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:22:20.159166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:20.159166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:22:20.159166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:20.159166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:22:20.159166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:22:20.159166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:20.159166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:22:20.159166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:20.159166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:22:20.160193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:20.160193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:22:20.160193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:20.160193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:22:20.160193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:22:20.160193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:22:20.160193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:22:20.160193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:22:20.160193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:22:20.160193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:22:20.160193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:20.160193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:20.160811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:20.160811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:22:20.160811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:20.160811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:20.160811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:20.160811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:22:20.160811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:20.160811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:22:20.160811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:22:20.160811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:20.160811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:22:20.160811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:20.160811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:22:20.160811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:22:20.161397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:20.161397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:22:20.161397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:20.161397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:22:20.162062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:20.162062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:22:20.162062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:22:20.162062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:22:20.162062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:22:20.162062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:22:20.162062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:20.162062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:22:20.162062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:22:20.162062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:22:20.162062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:22:20.162062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:22:20.162062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:20.162062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:20.162062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:22:20.162062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:22:20.162831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:22:20.162831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:22:20.162831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:22:20.162831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:22:20.162831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:22:20.162831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:20.162831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:20.162831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:20.162831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:22:20.162831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:20.162831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:20.162831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:20.162831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:22:20.162831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:20.162831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:22:20.162831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:22:20.162831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:20.162831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:22:20.163638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:20.163638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:22:20.163638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:22:20.163638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:20.163638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:22:20.163638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:20.163638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:22:20.178657 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:22:20.179017 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:22:20.179207 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:22:20.179414 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:22:20.180175 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:22:20.180609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:20.181444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:22:20.181444 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:20.181444 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:22:20.181444 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:22:20.181444 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:20.181444 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:22:20.181444 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:20.181444 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:22:20.182779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:20.182779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:22:20.182779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:22:20.182779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:22:20.182779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:20.182779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:22:20.182779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:20.182779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:22:20.182779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:20.182779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:22:20.182779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:20.182779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:22:20.182779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:20.182779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:22:20.182779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:22:20.183291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:20.183291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:22:20.183291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:20.183291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:22:20.183291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:22:20.183291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:20.183291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:22:20.183291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:20.183291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:22:20.184222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:20.184222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:22:20.184222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:20.184222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:22:20.184222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:22:20.184222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:22:20.184222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:22:20.184222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:22:20.184222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:22:20.184222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:20.184222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:20.184590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:20.184590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:22:20.184590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:20.184590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:20.184590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:20.184590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:22:20.184590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:20.184590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:22:20.184590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:22:20.184590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:20.184590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:22:20.184590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:20.184590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:22:20.184590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:22:20.184970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:20.184970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:22:20.184970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:20.184970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:22:20.186052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:20.186052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:22:20.186052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:22:20.186052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:22:20.186052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:22:20.186052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:22:20.186052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:20.186052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:22:20.186052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:22:20.186052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:22:20.186052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:22:20.186052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:22:20.186052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:20.186052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:20.186052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:22:20.186052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:22:20.186052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:22:20.186622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:22:20.186622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:22:20.186622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:22:20.186622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:20.186622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:20.186622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:20.186622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:22:20.186622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:20.186622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:20.186622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:20.186622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:22:20.186622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:20.186622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:22:20.186622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:22:20.186622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:20.186622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:22:20.186622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:20.186622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:22:20.186622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:22:20.187144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:20.187144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:22:20.187144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:20.187144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:22:20.187853 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:22:20.188162 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:22:20.188330 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:22:20.188592 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:22:20.189059 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:22:20.189465 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:22:20.189708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:20.190478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:22:20.190478 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:20.190478 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:22:20.190478 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:22:20.190478 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:20.190478 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:22:20.190478 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:20.190478 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:22:20.191300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:20.191300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:22:20.191300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:20.191300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:22:20.191300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:22:20.191300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:22:20.191300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:22:20.191300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:20.191300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:20.191300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:20.191300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:22:20.191300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:20.191682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:20.191682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:20.191682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:22:20.191682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:20.191682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:22:20.191682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:22:20.191682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:20.191682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:22:20.191682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:20.191682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:22:20.191682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:22:20.191682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:20.191682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:22:20.192018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:20.192018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:22:20.192283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:20.192283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:22:20.192283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:22:20.192283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:22:20.192283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:22:20.192283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:22:20.192283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:20.192283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:22:20.192283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:22:20.192283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:22:20.192283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:22:20.192283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:22:20.192283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:20.192283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:20.192283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:22:20.192283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:22:20.192728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:22:20.192728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:22:20.192728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:20.192728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:20.192728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:20.192728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:22:20.192728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:20.192728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:20.192728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:20.192728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:22:20.192728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:20.192728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:22:20.192728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:22:20.192728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:20.192728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:22:20.192728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:20.192728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:22:20.192728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:22:20.192728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:20.193262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:22:20.193262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:20.193262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:22:24.919268 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:22:24.920120 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:22:24.920120 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:22:24.921543 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:22:24.923213 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG dbcounter [-] [105581] Writing DB stats neutron:SELECT=240 {{(pid=105581) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 14:22:24.960204 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:22:24.960804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:24.961853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:22:24.961853 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:24.961853 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:22:24.961853 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:22:24.961853 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:24.961853 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:22:24.961853 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:24.961853 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:22:24.962735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:24.962735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:22:24.962735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:24.962735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:22:24.962735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:22:24.962735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:22:24.962735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:22:24.962735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:24.962735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:24.962735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:24.962735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:22:24.962735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:24.963546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:24.963546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:24.963546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:22:24.963546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:24.963546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:22:24.963546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:22:24.963546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:24.963546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:22:24.963546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:24.963546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:22:24.963546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:22:24.963546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:24.963546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:22:24.964290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:24.964290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:22:24.964290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:24.964290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:22:24.964290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:22:24.964290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:22:24.964290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:22:24.964290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:22:24.964290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:24.964290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:22:24.964290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:22:24.964290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:22:24.964290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:22:24.964290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:22:24.964290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:24.965269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:24.965269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:22:24.965269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:22:24.965269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:22:24.965269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:22:24.965269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:24.965269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:24.965269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:24.965269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:22:24.965269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:24.965269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:24.965269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:24.965269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:22:24.965269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:24.965269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:22:24.965269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:22:24.965269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:24.965269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:22:24.965269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:24.966651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:22:24.966651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:22:24.966651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:24.966651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:22:24.966651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:24.966651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:22:25.133864 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:22:25.134352 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:22:25.134583 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:22:25.136087 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:22:25.136576 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:22:25.136855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:25.137953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:22:25.137953 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:25.137953 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:22:25.137953 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:22:25.137953 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:25.137953 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:22:25.137953 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:25.137953 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:22:25.139546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:25.139546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:22:25.139546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:25.139546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:22:25.139546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:22:25.139546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:22:25.139546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:22:25.139546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:25.139546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:25.139546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:25.139546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:22:25.139546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:25.140157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:25.140157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:25.140157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:22:25.140157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:25.140157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:22:25.140157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:22:25.140157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:25.140157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:22:25.140157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:25.140157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:22:25.140157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:22:25.140157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:25.140157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:22:25.140826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:25.140826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:22:25.141178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:25.141178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:22:25.141178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:22:25.141178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:22:25.141178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:22:25.141178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:22:25.141178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:25.141178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:22:25.141178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:22:25.141178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:22:25.141178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:22:25.141178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:22:25.141178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:25.141178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:25.141178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:22:25.141178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:22:25.141957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:22:25.141957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:22:25.141957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:25.141957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:25.141957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:25.141957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:22:25.141957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:25.141957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:25.141957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:25.141957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:22:25.141957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:25.141957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:22:25.141957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:22:25.141957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:25.141957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:22:25.141957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:25.141957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:22:25.141957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:22:25.141957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:25.142866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:22:25.142866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:25.142866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:22:25.154754 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:22:25.155139 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:22:25.155318 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:22:25.156363 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:22:25.157081 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:22:25.157914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:25.158285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:22:25.158285 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:25.158285 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:22:25.158285 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:22:25.158285 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:25.158285 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:22:25.158285 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:25.158285 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:22:25.159076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:25.159076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:22:25.159076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:22:25.159076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:22:25.159076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:25.159076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:22:25.159076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:25.159076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:22:25.159076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:25.159076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:22:25.159076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:25.159076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:22:25.159076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:25.159076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:22:25.159076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:22:25.160079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:25.160079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:22:25.160079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:25.160079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:22:25.160079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:22:25.160079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:25.160079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:22:25.160079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:25.160079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:22:25.160079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:25.160079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:22:25.160079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:25.160079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:22:25.160079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:22:25.160895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:22:25.160895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:22:25.160895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:22:25.160895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:22:25.160895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:22:25.160895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:25.160895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:25.160895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:25.160895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:22:25.160895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:25.160895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:25.160895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:25.160895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:22:25.161398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:25.161398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:22:25.161398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:22:25.161398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:25.161398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:22:25.161398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:25.161398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:22:25.161398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:22:25.161398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:25.161398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:22:25.161398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:25.161398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:22:25.161746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:25.161746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:22:25.161746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:22:25.161746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:22:25.161746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:22:25.161746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:22:25.161746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:25.161746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:22:25.161746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:22:25.161746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:22:25.161746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:22:25.161746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:22:25.161746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:25.161746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:25.161746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:22:25.161746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:22:25.162157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:22:25.162157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:22:25.162157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:22:25.162157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:22:25.162157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:22:25.162157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:25.162157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:25.162157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:25.162157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:22:25.162157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:25.162157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:25.162157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:25.162157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:22:25.162157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:25.162157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:22:25.162157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:22:25.162157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:25.162157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:22:25.162735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:25.162735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:22:25.162735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:22:25.162735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:25.162735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:22:25.162735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:25.162735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:22:25.179352 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:22:25.179674 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:22:25.179810 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:22:25.179939 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:22:25.181129 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:22:25.182447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:25.182972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:22:25.182972 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:25.182972 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:22:25.182972 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:22:25.182972 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:25.182972 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:22:25.182972 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:25.182972 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:22:25.184071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:25.184071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:22:25.184071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:22:25.184071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:22:25.184071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:25.184071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:22:25.184071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:25.184071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:22:25.184071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:25.184071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:22:25.184071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:25.184071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:22:25.184071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:25.184071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:22:25.184071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:22:25.185066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:25.185066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:22:25.185066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:25.185066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:22:25.185066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:22:25.185066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:25.185066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:22:25.185066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:25.185066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:22:25.185543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:25.185543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:22:25.185543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:25.185543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:22:25.185543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:22:25.185543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:22:25.185543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:22:25.185543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:22:25.185543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:22:25.185543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:25.185543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:25.185945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:25.185945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:22:25.185945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:25.185945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:25.185945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:25.185945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:22:25.185945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:25.185945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:22:25.185945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:22:25.185945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:25.185945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:22:25.185945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:25.185945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:22:25.185945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:22:25.186464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:25.186464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:22:25.186464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:25.186464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:22:25.186626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:25.186626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:22:25.186626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:22:25.186626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:22:25.186626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:22:25.186626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:22:25.186626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:25.186626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:22:25.186626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:22:25.186626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:22:25.186626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:22:25.186626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:22:25.186626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:25.186626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:25.186626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:22:25.186626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:22:25.186626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:22:25.187202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:22:25.187202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:22:25.187202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:22:25.187202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:25.187202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:25.187202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:25.187202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:22:25.187202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:25.187202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:25.187202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:25.187202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:22:25.187202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:25.187202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:22:25.187202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:22:25.187202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:25.187202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:22:25.187202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:25.187202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:22:25.187202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:22:25.187825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:25.187825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:22:25.187825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:25.187825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:22:25.188264 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:22:25.188672 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:22:25.188823 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:22:25.189039 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:22:25.189390 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:22:25.189646 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:22:25.189819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:25.190298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:22:25.190298 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:25.190298 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:22:25.190298 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:22:25.190298 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:25.190298 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:22:25.190298 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:25.190298 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:22:25.191139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:25.191139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:22:25.191139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:25.191139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:22:25.191139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:22:25.191139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:22:25.191139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:22:25.191139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:25.191139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:25.191139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:25.191139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:22:25.191139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:25.191400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:25.191400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:25.191400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:22:25.191400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:25.191400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:22:25.191400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:22:25.191400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:25.191400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:22:25.191400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:25.191400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:22:25.191400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:22:25.191400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:25.191400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:22:25.191682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:25.191682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:22:25.192221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:25.192221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:22:25.192221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:22:25.192221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:22:25.192221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:22:25.192221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:22:25.192221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:25.192221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:22:25.192221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:22:25.192221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:22:25.192221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:22:25.192221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:22:25.192221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:25.192221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:25.192221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:22:25.192221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:22:25.192601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:22:25.192601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:22:25.192601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:25.192601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:25.192601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:25.192601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:22:25.192601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:25.192601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:25.192601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:25.192601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:22:25.192601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:25.192601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:22:25.192601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:22:25.192601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:25.192601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:22:25.192601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:25.192601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:22:25.192601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:22:25.192601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:25.192971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:22:25.192971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:25.192971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:22:29.919924 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:22:29.920872 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:22:29.920872 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:22:29.922355 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:22:29.956569 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:22:29.956951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:29.957842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:22:29.957842 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:29.957842 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:22:29.957842 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:22:29.957842 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:29.957842 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:22:29.957842 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:29.957842 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:22:29.958636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:29.958636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:22:29.958636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:29.958636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:22:29.958636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:22:29.958636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:22:29.958636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:22:29.958636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:29.958636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:29.958636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:29.958636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:22:29.958636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:29.959123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:29.959123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:29.959123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:22:29.959123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:29.959123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:22:29.959123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:22:29.959123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:29.959123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:22:29.959123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:29.959123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:22:29.959123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:22:29.959123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:29.959123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:22:29.959767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:29.959767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:22:29.959767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:29.959767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:22:29.959767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:22:29.959767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:22:29.959767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:22:29.959767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:22:29.959767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:29.959767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:22:29.959767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:22:29.959767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:22:29.959767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:22:29.959767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:22:29.959767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:29.960495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:29.960495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:22:29.960495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:22:29.960495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:22:29.960495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:22:29.960495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:29.960495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:29.960495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:29.960495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:22:29.960495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:29.960495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:29.960495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:29.960495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:22:29.960495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:29.960495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:22:29.960495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:22:29.960495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:29.960495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:22:29.960495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:29.961232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:22:29.961232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:22:29.961232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:29.961232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:22:29.961232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:29.961232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:22:30.134442 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:22:30.135056 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:22:30.135306 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:22:30.136571 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:22:30.136918 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:22:30.137200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:30.138193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:22:30.138193 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:30.138193 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:22:30.138193 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:22:30.138193 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:30.138193 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:22:30.138193 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:30.138193 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:22:30.140340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:30.140340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:22:30.140340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:30.140340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:22:30.140340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:22:30.140340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:22:30.140340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:22:30.140340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:30.140340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:30.140340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:30.140340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:22:30.140340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:30.140754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:30.140754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:30.140754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:22:30.140754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:30.140754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:22:30.140754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:22:30.140754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:30.140754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:22:30.140754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:30.140754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:22:30.140754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:22:30.140754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:30.140754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:22:30.141106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:30.141106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:22:30.142775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:30.142775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:22:30.142775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:22:30.142775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:22:30.142775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:22:30.142775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:22:30.142775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:30.142775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:22:30.142775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:22:30.142775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:22:30.142775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:22:30.142775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:22:30.142775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:30.142775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:30.142775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:22:30.142775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:22:30.143215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:22:30.143215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:22:30.143215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:30.143215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:30.143215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:30.143215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:22:30.143215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:30.143215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:30.143215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:30.143215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:22:30.143215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:30.143215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:22:30.143215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:22:30.143215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:30.143215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:22:30.143215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:30.143215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:22:30.143215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:22:30.143215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:30.143753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:22:30.143753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:30.143753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:22:30.155191 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:22:30.155452 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:22:30.155576 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:22:30.155696 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:22:30.156221 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:22:30.156573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:30.157327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:22:30.157327 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:30.157327 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:22:30.157327 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:22:30.157327 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:30.157327 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:22:30.157327 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:30.157327 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:22:30.158102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:30.158102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:22:30.158102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:22:30.158102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:22:30.158102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:30.158102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:22:30.158102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:30.158102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:22:30.158102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:30.158102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:22:30.158102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:30.158102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:22:30.158102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:30.158102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:22:30.158102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:22:30.158658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:30.158658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:22:30.158658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:30.158658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:22:30.158658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:22:30.158658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:30.158658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:22:30.158658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:30.158658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:22:30.158961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:30.158961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:22:30.158961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:30.158961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:22:30.158961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:22:30.158961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:22:30.158961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:22:30.158961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:22:30.158961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:22:30.158961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:22:30.158961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:30.158961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:30.159324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:30.159324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:22:30.159324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:30.159324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:30.159324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:30.159324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:22:30.159324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:30.159324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:22:30.159324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:22:30.159324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:30.159324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:22:30.159324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:30.159324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:22:30.159324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:22:30.159681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:30.159681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:22:30.159681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:30.159681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:22:30.160068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:30.160068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:22:30.160068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:22:30.160068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:22:30.160068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:22:30.160068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:22:30.160068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:30.160068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:22:30.160068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:22:30.160068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:22:30.160068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:22:30.160068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:22:30.160068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:30.160068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:30.160068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:22:30.160068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:22:30.160536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:22:30.160536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:22:30.160536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:22:30.160536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:22:30.160536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:22:30.160536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:30.160536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:30.160536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:30.160536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:22:30.160536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:30.160536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:30.160536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:30.160536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:22:30.160536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:30.160536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:22:30.160536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:22:30.160536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:30.160536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:22:30.161034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:30.161034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:22:30.161034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:22:30.161034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:30.161034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:22:30.161034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:30.161034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:22:30.179936 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:22:30.180368 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:22:30.180587 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:22:30.180864 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:22:30.181369 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:22:30.181768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:30.182537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:22:30.182537 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:30.182537 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:22:30.182537 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:22:30.182537 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:30.182537 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:22:30.182537 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:30.182537 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:22:30.183311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:30.183311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:22:30.183311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:22:30.183311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:22:30.183311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:30.183311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:22:30.183311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:30.183311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:22:30.183311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:30.183311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:22:30.183311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:30.183311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:22:30.183311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:30.183311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:22:30.183311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:22:30.183739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:30.183739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:22:30.183739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:30.183739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:22:30.183739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:22:30.183739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:30.183739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:22:30.183739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:30.183739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:22:30.184242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:30.184242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:22:30.184242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:30.184242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:22:30.184242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:22:30.184242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:22:30.184242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:22:30.184242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:22:30.184242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:22:30.184242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:30.184242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:30.184593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:30.184593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:22:30.184593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:30.184593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:30.184593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:30.184593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:22:30.184593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:30.184593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:22:30.184593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:22:30.184593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:30.184593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:22:30.184593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:30.184593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:22:30.184593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:22:30.184950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:30.184950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:22:30.184950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:30.184950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:22:30.185279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:30.185279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:22:30.185279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:22:30.185279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:22:30.185279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:22:30.185279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:22:30.185279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:30.185279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:22:30.185279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:22:30.185279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:22:30.185279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:22:30.185279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:22:30.185279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:30.185279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:30.185279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:22:30.185279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:22:30.185279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:22:30.185763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:22:30.185763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:22:30.185763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:22:30.185763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:30.185763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:30.185763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:30.185763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:22:30.185763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:30.185763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:30.185763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:30.185763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:22:30.185763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:30.185763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:22:30.185763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:22:30.185763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:30.185763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:22:30.185763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:30.185763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:22:30.185763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:22:30.186253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:30.186253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:22:30.186253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:30.186253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:22:30.188620 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:22:30.188860 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:22:30.188971 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:22:30.189118 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:22:30.189492 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:22:30.189706 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:22:30.189848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:30.190329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:22:30.190329 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:30.190329 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:22:30.190329 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:22:30.190329 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:30.190329 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:22:30.190329 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:30.190329 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:22:30.191184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:30.191184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:22:30.191184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:30.191184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:22:30.191184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:22:30.191184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:22:30.191184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:22:30.191184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:30.191184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:30.191184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:30.191184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:22:30.191184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:30.191561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:30.191561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:30.191561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:22:30.191561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:30.191561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:22:30.191561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:22:30.191561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:30.191561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:22:30.191561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:30.191561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:22:30.191561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:22:30.191561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:30.191561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:22:30.191925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:30.191925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:22:30.192126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:30.192126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:22:30.192126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:22:30.192126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:22:30.192126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:22:30.192126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:22:30.192126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:30.192126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:22:30.192126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:22:30.192126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:22:30.192126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:22:30.192126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:22:30.192126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:30.192126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:30.192126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:22:30.192126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:22:30.192575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:22:30.192575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:22:30.192575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:30.192575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:30.192575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:30.192575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:22:30.192575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:30.192575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:30.192575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:30.192575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:22:30.192575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:30.192575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:22:30.192575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:22:30.192575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:30.192575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:22:30.192575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:30.192575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:22:30.192575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:22:30.192575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:30.193089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:22:30.193089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:30.193089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:22:34.920671 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:22:34.921589 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:22:34.921589 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:22:34.922748 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:22:34.953277 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:22:34.953748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:34.954556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:22:34.954556 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:34.954556 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:22:34.954556 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:22:34.954556 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:34.954556 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:22:34.954556 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:34.954556 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:22:34.955582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:34.955582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:22:34.955582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:34.955582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:22:34.955582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:22:34.955582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:22:34.955582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:22:34.955582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:34.955582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:34.955582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:34.955582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:22:34.955582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:34.956255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:34.956255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:34.956255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:22:34.956255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:34.956255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:22:34.956255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:22:34.956255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:34.956255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:22:34.956255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:34.956255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:22:34.956255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:22:34.956255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:34.956255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:22:34.957081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:34.957081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:22:34.957081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:34.957081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:22:34.957081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:22:34.957081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:22:34.957081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:22:34.957081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:22:34.957081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:34.957081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:22:34.957081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:22:34.957081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:22:34.957081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:22:34.957081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:22:34.957081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:34.958085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:34.958085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:22:34.958085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:22:34.958085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:22:34.958085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:22:34.958085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:34.958085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:34.958085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:34.958085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:22:34.958085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:34.958085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:34.958085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:34.958085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:22:34.958085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:34.958085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:22:34.958085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:22:34.958085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:34.958085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:22:34.958085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:34.959012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:22:34.959012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:22:34.959012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:34.959012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:22:34.959012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:34.959012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:22:35.135123 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:22:35.135722 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:22:35.135850 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:22:35.136710 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:22:35.137024 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:22:35.137252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:35.138118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:22:35.138118 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:35.138118 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:22:35.138118 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:22:35.138118 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:35.138118 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:22:35.138118 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:35.138118 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:22:35.139327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:35.139327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:22:35.139327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:35.139327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:22:35.139327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:22:35.139327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:22:35.139327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:22:35.139327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:35.139327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:35.139327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:35.139327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:22:35.139327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:35.139968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:35.139968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:35.139968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:22:35.139968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:35.139968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:22:35.139968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:22:35.139968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:35.139968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:22:35.139968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:35.139968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:22:35.139968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:22:35.139968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:35.139968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:22:35.140563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:35.140563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:22:35.140832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:35.140832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:22:35.140832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:22:35.140832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:22:35.140832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:22:35.140832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:22:35.140832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:35.140832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:22:35.140832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:22:35.140832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:22:35.140832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:22:35.140832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:22:35.140832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:35.140832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:35.140832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:22:35.140832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:22:35.141821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:22:35.141821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:22:35.141821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:35.141821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:35.141821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:35.141821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:22:35.141821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:35.141821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:35.141821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:35.141821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:22:35.141821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:35.141821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:22:35.141821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:22:35.141821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:35.141821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:22:35.141821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:35.141821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:22:35.141821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:22:35.141821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:35.142688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:22:35.142688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:35.142688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:22:35.155661 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:22:35.156154 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:22:35.156520 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:22:35.156746 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:22:35.157333 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:22:35.157607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:35.158503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:22:35.158503 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:35.158503 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:22:35.158503 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:22:35.158503 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:35.158503 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:22:35.158503 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:35.158503 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:22:35.159852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:35.159852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:22:35.159852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:22:35.159852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:22:35.159852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:35.159852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:22:35.159852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:35.159852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:22:35.159852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:35.159852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:22:35.159852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:35.159852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:22:35.159852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:35.159852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:22:35.159852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:22:35.160313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:35.160313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:22:35.160313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:35.160313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:22:35.160313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:22:35.160313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:35.160313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:22:35.160313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:35.160313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:22:35.161504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:35.161504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:22:35.161504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:35.161504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:22:35.161504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:22:35.161504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:22:35.161504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:22:35.161504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:22:35.161504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:22:35.161504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:22:35.161504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:35.161504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:35.161906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:35.161906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:22:35.161906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:35.161906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:35.161906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:35.161906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:22:35.161906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:35.161906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:22:35.161906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:22:35.161906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:35.161906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:22:35.161906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:35.161906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:22:35.161906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:22:35.162256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:35.162256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:22:35.162256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:35.162256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:22:35.163314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:35.163314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:22:35.163314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:22:35.163314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:22:35.163314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:22:35.163314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:22:35.163314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:35.163314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:22:35.163314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:22:35.163314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:22:35.163314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:22:35.163314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:22:35.163314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:35.163314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:35.163314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:22:35.163314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:22:35.163805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:22:35.163805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:22:35.163805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:22:35.163805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:22:35.163805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:22:35.163805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:35.163805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:35.163805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:35.163805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:22:35.163805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:35.163805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:35.163805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:35.163805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:22:35.163805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:35.163805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:22:35.163805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:22:35.163805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:35.163805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:22:35.164392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:35.164392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:22:35.164392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:22:35.164392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:35.164392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:22:35.164392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:35.164392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:22:35.180498 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:22:35.180972 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:22:35.181170 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:22:35.181394 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:22:35.182051 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:22:35.182495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:35.183534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:22:35.183534 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:35.183534 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:22:35.183534 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:22:35.183534 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:35.183534 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:22:35.183534 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:35.183534 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:22:35.185038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:35.185038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:22:35.185038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:22:35.185038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:22:35.185038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:35.185038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:22:35.185038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:35.185038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:22:35.185038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:35.185038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:22:35.185038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:35.185038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:22:35.185038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:35.185038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:22:35.185038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:22:35.185905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:35.185905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:22:35.185905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:35.185905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:22:35.185905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:22:35.185905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:35.185905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:22:35.185905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:35.185905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:22:35.186887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:35.186887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:22:35.186887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:35.186887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:22:35.186887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:22:35.186887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:22:35.186887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:22:35.186887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:22:35.186887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:22:35.186887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:35.186887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:35.187526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:35.187526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:22:35.187526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:35.187526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:35.187526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:35.187526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:22:35.187526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:35.187526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:22:35.187526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:22:35.187526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:35.187526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:22:35.187526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:35.187526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:22:35.187526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:22:35.188141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:35.188141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:22:35.188141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:35.188141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:22:35.189148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:35.189148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:22:35.189148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:22:35.189148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:22:35.189148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:22:35.189148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:22:35.189148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:35.189148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:22:35.189148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:22:35.189148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:22:35.189148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:22:35.189148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:22:35.189148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:35.189148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:35.189148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:22:35.189148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:22:35.189148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:22:35.189688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:22:35.189688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:22:35.189688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:22:35.189688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:35.189688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:35.189688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:35.189688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:22:35.189688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:35.189688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:35.189688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:35.189688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:22:35.189688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:35.189688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:22:35.189688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:22:35.189688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:35.189688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:22:35.189688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:35.189688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:22:35.189688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:22:35.190217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:35.190217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:22:35.190217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:35.190217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:22:35.190217 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:22:35.190217 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:22:35.190217 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:22:35.190473 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:22:35.190872 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:22:35.191215 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:22:35.191474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:35.192209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:22:35.192209 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:35.192209 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:22:35.192209 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:22:35.192209 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:35.192209 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:22:35.192209 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:35.192209 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:22:35.193319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:35.193319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:22:35.193319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:35.193319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:22:35.193319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:22:35.193319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:22:35.193319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:22:35.193319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:35.193319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:35.193319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:35.193319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:22:35.193319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:35.193718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:35.193718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:35.193718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:22:35.193718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:35.193718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:22:35.193718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:22:35.193718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:35.193718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:22:35.193718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:35.193718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:22:35.193718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:22:35.193718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:35.193718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:22:35.194113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:35.194113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:22:35.194294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:35.194294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:22:35.194294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:22:35.194294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:22:35.194294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:22:35.194294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:22:35.194294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:35.194294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:22:35.194294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:22:35.194294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:22:35.194294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:22:35.194294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:22:35.194294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:35.194294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:35.194294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:22:35.194294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:22:35.194788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:22:35.194788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:22:35.194788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:35.194788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:35.194788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:35.194788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:22:35.194788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:35.194788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:35.194788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:35.194788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:22:35.194788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:35.194788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:22:35.194788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:22:35.194788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:35.194788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:22:35.194788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:35.194788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:22:35.194788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:22:35.194788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:35.195324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:22:35.195324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:35.195324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:22:39.921293 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:22:39.922193 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:22:39.922193 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:22:39.923159 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:22:39.955771 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:22:39.956038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:39.956858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:22:39.956858 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:39.956858 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:22:39.956858 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:22:39.956858 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:39.956858 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:22:39.956858 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:39.956858 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:22:39.957576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:39.957576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:22:39.957576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:39.957576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:22:39.957576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:22:39.957576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:22:39.957576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:22:39.957576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:39.957576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:39.957576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:39.957576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:22:39.957576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:39.958195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:39.958195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:39.958195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:22:39.958195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:39.958195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:22:39.958195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:22:39.958195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:39.958195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:22:39.958195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:39.958195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:22:39.958195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:22:39.958195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:39.958195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:22:39.958836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:39.958836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:22:39.958836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:39.958836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:22:39.958836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:22:39.958836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:22:39.958836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:22:39.958836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:22:39.958836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:39.958836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:22:39.958836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:22:39.958836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:22:39.958836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:22:39.958836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:22:39.958836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:39.959586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:39.959586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:22:39.959586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:22:39.959586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:22:39.959586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:22:39.959586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:39.959586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:39.959586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:39.959586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:22:39.959586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:39.959586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:39.959586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:39.959586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:22:39.959586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:39.959586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:22:39.959586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:22:39.959586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:39.959586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:22:39.959586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:39.960703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:22:39.960703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:22:39.960703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:39.960703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:22:39.960703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:39.960703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:22:40.135951 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:22:40.136427 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:22:40.136662 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:22:40.137756 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:22:40.138129 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:22:40.138377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:40.139267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:22:40.139267 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:40.139267 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:22:40.139267 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:22:40.139267 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:40.139267 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:22:40.139267 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:40.139267 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:22:40.140709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:40.140709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:22:40.140709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:40.140709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:22:40.140709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:22:40.140709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:22:40.140709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:22:40.140709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:40.140709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:40.140709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:40.140709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:22:40.140709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:40.141298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:40.141298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:40.141298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:22:40.141298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:40.141298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:22:40.141298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:22:40.141298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:40.141298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:22:40.141298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:40.141298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:22:40.141298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:22:40.141298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:40.141298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:22:40.142080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:40.142080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:22:40.142273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:40.142273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:22:40.142273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:22:40.142273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:22:40.142273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:22:40.142273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:22:40.142273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:40.142273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:22:40.142273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:22:40.142273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:22:40.142273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:22:40.142273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:22:40.142273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:40.142273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:40.142273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:22:40.142273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:22:40.143039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:22:40.143039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:22:40.143039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:40.143039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:40.143039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:40.143039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:22:40.143039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:40.143039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:40.143039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:40.143039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:22:40.143039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:40.143039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:22:40.143039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:22:40.143039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:40.143039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:22:40.143039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:40.143039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:22:40.143039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:22:40.143039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:40.143961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:22:40.143961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:40.143961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:22:40.156159 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:22:40.156653 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:22:40.156832 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:22:40.157031 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:22:40.157417 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:22:40.157656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:40.158233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:22:40.158233 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:40.158233 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:22:40.158233 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:22:40.158233 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:40.158233 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:22:40.158233 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:40.158233 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:22:40.159367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:40.159367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:22:40.159367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:22:40.159367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:22:40.159367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:40.159367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:22:40.159367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:40.159367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:22:40.159367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:40.159367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:22:40.159367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:40.159367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:22:40.159367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:40.159367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:22:40.159367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:22:40.160103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:40.160103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:22:40.160103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:40.160103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:22:40.160103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:22:40.160103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:40.160103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:22:40.160103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:40.160103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:22:40.160732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:40.160732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:22:40.160732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:40.160732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:22:40.160732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:22:40.160732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:22:40.160732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:22:40.160732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:22:40.160732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:22:40.160732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:22:40.160732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:40.160732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:40.161279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:40.161279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:22:40.161279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:40.161279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:40.161279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:40.161279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:22:40.161279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:40.161279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:22:40.161279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:22:40.161279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:40.161279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:22:40.161279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:40.161279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:22:40.161279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:22:40.161940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:40.161940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:22:40.161940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:40.161940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:22:40.162309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:40.162309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:22:40.162309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:22:40.162309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:22:40.162309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:22:40.162309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:22:40.162309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:40.162309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:22:40.162309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:22:40.162309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:22:40.162309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:22:40.162309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:22:40.162309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:40.162309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:40.162309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:22:40.162309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:22:40.163232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:22:40.163232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:22:40.163232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:22:40.163232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:22:40.163232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:22:40.163232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:40.163232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:40.163232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:40.163232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:22:40.163232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:40.163232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:40.163232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:40.163232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:22:40.163232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:40.163232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:22:40.163232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:22:40.163232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:40.163232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:22:40.164051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:40.164051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:22:40.164051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:22:40.164051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:40.164051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:22:40.164051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:40.164051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:22:40.181000 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:22:40.181297 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:22:40.181402 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:22:40.181590 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:22:40.182098 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:22:40.182328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:40.182865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:22:40.182865 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:40.182865 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:22:40.182865 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:22:40.182865 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:40.182865 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:22:40.182865 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:40.182865 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:22:40.183640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:40.183640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:22:40.183640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:22:40.183640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:22:40.183640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:40.183640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:22:40.183640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:40.183640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:22:40.183640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:40.183640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:22:40.183640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:40.183640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:22:40.183640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:40.183640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:22:40.183640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:22:40.184294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:40.184294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:22:40.184294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:40.184294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:22:40.184294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:22:40.184294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:40.184294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:22:40.184294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:40.184294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:22:40.184812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:40.184812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:22:40.184812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:40.184812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:22:40.184812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:22:40.184812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:22:40.184812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:22:40.184812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:22:40.184812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:22:40.184812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:40.184812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:40.185359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:40.185359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:22:40.185359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:40.185359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:40.185359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:40.185359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:22:40.185359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:40.185359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:22:40.185359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:22:40.185359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:40.185359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:22:40.185359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:40.185359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:22:40.185359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:22:40.186012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:40.186012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:22:40.186012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:40.186012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:22:40.186012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:40.186012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:22:40.186012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:22:40.186012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:22:40.186012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:22:40.186012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:22:40.186012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:40.186012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:22:40.186012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:22:40.186012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:22:40.186638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:22:40.186638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:22:40.186638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:40.186638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:40.186638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:22:40.186638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:22:40.186638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:22:40.186638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:22:40.186638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:22:40.186638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:22:40.186638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:40.186638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:40.186638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:40.186638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:22:40.186638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:40.186638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:40.186638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:40.186638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:22:40.186638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:40.187501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:22:40.187501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:22:40.187501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:40.187501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:22:40.187501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:40.187501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:22:40.187501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:22:40.187501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:40.187501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:22:40.187501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:40.187501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:22:40.190001 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:22:40.190272 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:22:40.190369 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:22:40.190544 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:22:40.190851 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:22:40.191068 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:22:40.191216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:40.191682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:22:40.191682 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:40.191682 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:22:40.191682 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:22:40.191682 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:40.191682 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:22:40.191682 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:40.191682 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:22:40.192484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:40.192484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:22:40.192484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:40.192484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:22:40.192484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:22:40.192484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:22:40.192484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:22:40.192484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:40.192484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:40.192484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:40.192484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:22:40.192484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:40.192856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:40.192856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:40.192856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:22:40.192856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:40.192856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:22:40.192856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:22:40.192856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:40.192856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:22:40.192856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:40.192856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:22:40.192856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:22:40.192856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:40.192856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:22:40.193230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:40.193230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:22:40.193440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:40.193440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:22:40.193440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:22:40.193440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:22:40.193440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:22:40.193440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:22:40.193440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:40.193440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:22:40.193440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:22:40.193440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:22:40.193440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:22:40.193440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:22:40.193440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:40.193440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:40.193440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:22:40.193440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:22:40.193883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:22:40.193883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:22:40.193883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:40.193883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:40.193883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:40.193883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:22:40.193883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:40.193883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:40.193883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:40.193883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:22:40.193883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:40.193883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:22:40.193883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:22:40.193883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:40.193883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:22:40.193883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:40.193883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:22:40.193883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:22:40.193883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:40.194518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:22:40.194518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:40.194518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:22:44.921893 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:22:44.922917 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:22:44.922917 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:22:44.923918 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:22:44.955563 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:22:44.955966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:44.956849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:22:44.956849 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:44.956849 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:22:44.956849 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:22:44.956849 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:44.956849 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:22:44.956849 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:44.956849 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:22:44.957867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:44.957867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:22:44.957867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:44.957867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:22:44.957867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:22:44.957867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:22:44.957867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:22:44.957867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:44.957867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:44.957867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:44.957867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:22:44.957867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:44.958347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:44.958347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:44.958347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:22:44.958347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:44.958347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:22:44.958347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:22:44.958347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:44.958347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:22:44.958347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:44.958347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:22:44.958347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:22:44.958347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:44.958347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:22:44.958777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:44.958777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:22:44.958850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:44.958850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:22:44.958850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:22:44.958850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:22:44.958850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:22:44.958850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:22:44.958850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:44.958850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:22:44.958850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:22:44.958850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:22:44.958850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:22:44.958850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:22:44.958850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:44.958850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:44.958850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:22:44.958850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:22:44.959359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:22:44.959359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:22:44.959359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:44.959359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:44.959359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:44.959359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:22:44.959359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:44.959359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:44.959359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:44.959359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:22:44.959359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:44.959359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:22:44.959359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:22:44.959359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:44.959359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:22:44.959359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:44.959359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:22:44.959359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:22:44.959359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:44.959981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:22:44.959981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:44.959981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:22:45.136637 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:22:45.137155 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:22:45.137344 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:22:45.138368 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:22:45.138741 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:22:45.139178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:45.140127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:22:45.140127 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:45.140127 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:22:45.140127 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:22:45.140127 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:45.140127 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:22:45.140127 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:45.140127 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:22:45.141604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:45.141604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:22:45.141604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:45.141604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:22:45.141604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:22:45.141604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:22:45.141604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:22:45.141604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:45.141604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:45.141604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:45.141604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:22:45.141604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:45.142221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:45.142221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:45.142221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:22:45.142221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:45.142221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:22:45.142221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:22:45.142221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:45.142221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:22:45.142221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:45.142221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:22:45.142221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:22:45.142221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:45.142221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:22:45.142902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:45.142902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:22:45.143206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:45.143206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:22:45.143206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:22:45.143206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:22:45.143206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:22:45.143206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:22:45.143206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:45.143206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:22:45.143206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:22:45.143206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:22:45.143206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:22:45.143206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:22:45.143206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:45.143206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:45.143206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:22:45.143206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:22:45.143986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:22:45.143986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:22:45.143986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:45.143986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:45.143986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:45.143986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:22:45.143986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:45.143986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:45.143986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:45.143986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:22:45.143986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:45.143986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:22:45.143986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:22:45.143986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:45.143986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:22:45.143986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:45.143986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:22:45.143986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:22:45.143986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:45.144858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:22:45.144858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:45.144858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:22:45.156606 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:22:45.157485 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:22:45.157485 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:22:45.157485 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:22:45.157809 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:22:45.158017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:45.159036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:22:45.159036 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:45.159036 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:22:45.159036 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:22:45.159036 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:45.159036 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:22:45.159036 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:45.159036 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:22:45.159914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:45.159914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:22:45.159914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:22:45.159914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:22:45.159914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:45.159914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:22:45.159914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:45.159914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:22:45.159914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:45.159914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:22:45.159914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:45.159914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:22:45.159914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:45.159914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:22:45.159914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:22:45.161380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:45.161380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:22:45.161380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:45.161380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:22:45.161380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:22:45.161380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:45.161380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:22:45.161380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:45.161380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:22:45.161380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:45.161380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:22:45.161380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:45.161380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:22:45.161380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:22:45.162360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:22:45.162360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:22:45.162360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:22:45.162360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:22:45.162360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:22:45.162360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:45.162360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:45.162360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:45.162360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:22:45.162360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:45.162360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:45.162360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:45.162360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:22:45.163026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:45.163026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:22:45.163026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:22:45.163026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:45.163026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:22:45.163026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:45.163026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:22:45.163026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:22:45.163026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:45.163026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:22:45.163026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:45.163026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:22:45.163805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:45.163805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:22:45.163805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:22:45.163805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:22:45.163805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:22:45.163805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:22:45.163805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:45.163805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:22:45.163805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:22:45.163805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:22:45.163805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:22:45.163805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:22:45.163805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:45.163805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:45.163805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:22:45.163805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:22:45.164317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:22:45.164317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:22:45.164317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:22:45.164317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:22:45.164317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:22:45.164317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:45.164317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:45.164317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:45.164317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:22:45.164317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:45.164317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:45.164317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:45.164317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:22:45.164317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:45.164317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:22:45.164317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:22:45.164317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:45.164317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:22:45.164847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:45.164847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:22:45.164847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:22:45.164847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:45.164847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:22:45.164847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:45.164847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:22:45.181468 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:22:45.181633 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:22:45.181773 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:22:45.181854 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:22:45.182235 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:22:45.182508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:45.183107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:22:45.183107 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:45.183107 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:22:45.183107 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:22:45.183107 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:45.183107 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:22:45.183107 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:45.183107 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:22:45.184238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:45.184238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:22:45.184238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:22:45.184238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:22:45.184238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:45.184238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:22:45.184238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:45.184238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:22:45.184238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:45.184238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:22:45.184238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:45.184238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:22:45.184238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:45.184238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:22:45.184238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:22:45.184915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:45.184915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:22:45.184915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:45.184915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:22:45.184915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:22:45.184915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:45.184915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:22:45.184915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:45.184915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:22:45.185597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:45.185597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:22:45.185597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:45.185597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:22:45.185597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:22:45.185597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:22:45.185597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:22:45.185597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:22:45.185597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:22:45.185597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:45.185597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:45.186184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:45.186184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:22:45.186184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:45.186184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:45.186184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:45.186184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:22:45.186184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:45.186184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:22:45.186184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:22:45.186184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:45.186184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:22:45.186184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:45.186184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:22:45.186184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:22:45.186803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:45.186803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:22:45.186803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:45.186803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:22:45.187299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:45.187299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:22:45.187299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:22:45.187299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:22:45.187299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:22:45.187299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:22:45.187299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:45.187299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:22:45.187299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:22:45.187299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:22:45.187299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:22:45.187299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:22:45.187299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:45.187299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:45.187299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:22:45.187299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:22:45.187299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:22:45.188254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:22:45.188254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:22:45.188254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:22:45.188254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:45.188254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:45.188254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:45.188254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:22:45.188254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:45.188254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:45.188254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:45.188254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:22:45.188254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:45.188254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:22:45.188254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:22:45.188254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:45.188254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:22:45.188254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:45.188254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:22:45.188254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:22:45.189072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:45.189072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:22:45.189072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:45.189072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:22:45.190421 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:22:45.190723 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:22:45.190897 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:22:45.191113 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:22:45.191595 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:22:45.191949 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:22:45.192177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:45.192862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:22:45.192862 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:45.192862 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:22:45.192862 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:22:45.192862 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:45.192862 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:22:45.192862 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:45.192862 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:22:45.193654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:45.193654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:22:45.193654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:45.193654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:22:45.193654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:22:45.193654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:22:45.193654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:22:45.193654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:45.193654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:45.193654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:45.193654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:22:45.193654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:45.194044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:45.194044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:45.194044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:22:45.194044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:45.194044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:22:45.194044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:22:45.194044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:45.194044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:22:45.194044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:45.194044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:22:45.194044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:22:45.194044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:45.194044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:22:45.194385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:45.194385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:22:45.194562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:45.194562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:22:45.194562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:22:45.194562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:22:45.194562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:22:45.194562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:22:45.194562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:45.194562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:22:45.194562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:22:45.194562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:22:45.194562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:22:45.194562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:22:45.194562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:45.194562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:45.194562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:22:45.194562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:22:45.195004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:22:45.195004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:22:45.195004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:45.195004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:45.195004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:45.195004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:22:45.195004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:45.195004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:45.195004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:45.195004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:22:45.195004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:45.195004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:22:45.195004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:22:45.195004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:45.195004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:22:45.195004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:45.195004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:22:45.195004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:22:45.195004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:45.195550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:22:45.195550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:45.195550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:22:49.922598 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:22:49.923382 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:22:49.923382 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:22:49.924711 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:22:49.960450 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:22:49.960868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:49.961760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:22:49.961760 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:49.961760 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:22:49.961760 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:22:49.961760 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:49.961760 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:22:49.961760 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:49.961760 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:22:49.962766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:49.962766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:22:49.962766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:49.962766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:22:49.962766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:22:49.962766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:22:49.962766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:22:49.962766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:49.962766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:49.962766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:49.962766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:22:49.962766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:49.963373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:49.963373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:49.963373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:22:49.963373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:49.963373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:22:49.963373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:22:49.963373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:49.963373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:22:49.963373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:49.963373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:22:49.963373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:22:49.963373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:49.963373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:22:49.963985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:49.963985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:22:49.963985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:49.963985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:22:49.963985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:22:49.963985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:22:49.963985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:22:49.963985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:22:49.963985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:49.963985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:22:49.963985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:22:49.963985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:22:49.963985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:22:49.963985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:22:49.963985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:49.964662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:49.964662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:22:49.964662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:22:49.964662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:22:49.964662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:22:49.964662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:49.964662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:49.964662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:49.964662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:22:49.964662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:49.964662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:49.964662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:49.964662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:22:49.964662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:49.964662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:22:49.964662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:22:49.964662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:49.964662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:22:49.964662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:49.965559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:22:49.965559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:22:49.965559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:49.965559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:22:49.965559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:49.965559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:22:50.137439 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:22:50.138116 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:22:50.138361 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:22:50.139386 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:22:50.139606 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:22:50.139773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:50.140424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:22:50.140424 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:50.140424 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:22:50.140424 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:22:50.140424 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:50.140424 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:22:50.140424 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:50.140424 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:22:50.142042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:50.142042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:22:50.142042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:50.142042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:22:50.142042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:22:50.142042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:22:50.142042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:22:50.142042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:50.142042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:50.142042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:50.142042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:22:50.142042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:50.142710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:50.142710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:50.142710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:22:50.142710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:50.142710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:22:50.142710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:22:50.142710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:50.142710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:22:50.142710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:50.142710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:22:50.142710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:22:50.142710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:50.142710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:22:50.143339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:50.143339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:22:50.143339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:50.143339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:22:50.143339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:22:50.143339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:22:50.143339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:22:50.143339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:22:50.143339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:50.143339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:22:50.143339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:22:50.143339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:22:50.143339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:22:50.143339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:22:50.143339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:50.144091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:50.144091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:22:50.144091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:22:50.144091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:22:50.144091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:22:50.144091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:50.144091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:50.144091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:50.144091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:22:50.144091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:50.144091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:50.144091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:50.144091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:22:50.144091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:50.144091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:22:50.144091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:22:50.144091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:50.144091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:22:50.144091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:50.145040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:22:50.145040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:22:50.145040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:50.145040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:22:50.145040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:50.145040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:22:50.157046 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:22:50.157472 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:22:50.157662 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:22:50.157919 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:22:50.158323 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:22:50.158568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:50.159278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:22:50.159278 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:50.159278 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:22:50.159278 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:22:50.159278 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:50.159278 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:22:50.159278 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:50.159278 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:22:50.160425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:50.160425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:22:50.160425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:22:50.160425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:22:50.160425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:50.160425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:22:50.160425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:50.160425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:22:50.160425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:50.160425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:22:50.160425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:50.160425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:22:50.160425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:50.160425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:22:50.160425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:22:50.161143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:50.161143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:22:50.161143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:50.161143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:22:50.161143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:22:50.161143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:50.161143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:22:50.161143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:50.161143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:22:50.161783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:50.161783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:22:50.161783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:50.161783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:22:50.161783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:22:50.161783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:22:50.161783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:22:50.161783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:22:50.161783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:22:50.161783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:22:50.161783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:50.161783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:50.162361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:50.162361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:22:50.162361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:50.162361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:50.162361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:50.162361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:22:50.162361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:50.162361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:22:50.162361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:22:50.162361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:50.162361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:22:50.162361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:50.162361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:22:50.162361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:22:50.163004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:50.163004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:22:50.163004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:50.163004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:22:50.163397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:50.163397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:22:50.163397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:22:50.163397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:22:50.163397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:22:50.163397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:22:50.163397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:50.163397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:22:50.163397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:22:50.163397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:22:50.163397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:22:50.163397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:22:50.163397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:50.163397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:50.163397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:22:50.163397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:22:50.164137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:22:50.164137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:22:50.164137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:22:50.164137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:22:50.164137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:22:50.164137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:50.164137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:50.164137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:50.164137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:22:50.164137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:50.164137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:50.164137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:50.164137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:22:50.164137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:50.164137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:22:50.164137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:22:50.164137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:50.164137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:22:50.164986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:50.164986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:22:50.164986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:22:50.164986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:50.164986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:22:50.164986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:50.164986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:22:50.181991 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:22:50.182305 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:22:50.182438 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:22:50.182589 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:22:50.182954 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:22:50.183202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:50.183710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:22:50.183710 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:50.183710 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:22:50.183710 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:22:50.183710 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:50.183710 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:22:50.183710 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:50.183710 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:22:50.184538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:50.184538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:22:50.184538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:22:50.184538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:22:50.184538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:50.184538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:22:50.184538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:50.184538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:22:50.184538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:50.184538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:22:50.184538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:50.184538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:22:50.184538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:50.184538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:22:50.184538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:22:50.185187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:50.185187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:22:50.185187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:50.185187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:22:50.185187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:22:50.185187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:50.185187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:22:50.185187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:50.185187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:22:50.185632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:50.185632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:22:50.185632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:50.185632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:22:50.185632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:22:50.185632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:22:50.185632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:22:50.185632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:22:50.185632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:22:50.185632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:50.185632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:50.186196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:50.186196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:22:50.186196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:50.186196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:50.186196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:50.186196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:22:50.186196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:50.186196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:22:50.186196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:22:50.186196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:50.186196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:22:50.186196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:50.186196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:22:50.186196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:22:50.186594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:50.186594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:22:50.186594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:50.186594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:22:50.186594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:50.186594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:22:50.186594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:22:50.186594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:22:50.186594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:22:50.186594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:22:50.186594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:50.186594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:22:50.186594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:22:50.186594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:22:50.186975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:22:50.186975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:22:50.186975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:50.186975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:50.186975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:22:50.186975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:22:50.186975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:22:50.186975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:22:50.186975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:22:50.186975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:22:50.186975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:50.186975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:50.186975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:50.186975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:22:50.186975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:50.186975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:50.186975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:50.186975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:22:50.186975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:50.187493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:22:50.187493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:22:50.187493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:50.187493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:22:50.187493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:50.187493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:22:50.187493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:22:50.187493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:50.187493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:22:50.187493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:50.187493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:22:50.190798 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:22:50.191166 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:22:50.191298 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:22:50.191463 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:22:50.191772 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:22:50.191981 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:22:50.192127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:50.192634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:22:50.192634 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:50.192634 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:22:50.192634 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:22:50.192634 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:50.192634 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:22:50.192634 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:50.192634 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:22:50.193415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:50.193415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:22:50.193415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:50.193415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:22:50.193415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:22:50.193415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:22:50.193415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:22:50.193415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:50.193415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:50.193415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:50.193415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:22:50.193415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:50.193831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:50.193831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:50.193831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:22:50.193831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:50.193831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:22:50.193831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:22:50.193831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:50.193831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:22:50.193831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:50.193831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:22:50.193831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:22:50.193831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:50.193831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:22:50.194192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:50.194192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:22:50.194388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:50.194388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:22:50.194388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:22:50.194388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:22:50.194388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:22:50.194388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:22:50.194388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:50.194388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:22:50.194388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:22:50.194388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:22:50.194388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:22:50.194388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:22:50.194388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:50.194388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:50.194388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:22:50.194388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:22:50.194872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:22:50.194872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:22:50.194872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:50.194872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:50.194872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:50.194872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:22:50.194872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:50.194872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:50.194872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:50.194872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:22:50.194872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:50.194872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:22:50.194872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:22:50.194872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:50.194872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:22:50.194872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:50.194872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:22:50.194872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:22:50.194872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:50.195400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:22:50.195400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:50.195400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:22:54.923302 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:22:54.924176 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:22:54.924176 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:22:54.924509 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:22:54.954757 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:22:54.955221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:54.956112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:22:54.956112 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:54.956112 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:22:54.956112 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:22:54.956112 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:54.956112 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:22:54.956112 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:54.956112 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:22:54.957172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:54.957172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:22:54.957172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:54.957172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:22:54.957172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:22:54.957172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:22:54.957172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:22:54.957172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:54.957172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:54.957172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:54.957172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:22:54.957172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:54.957893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:54.957893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:54.957893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:22:54.957893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:54.957893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:22:54.957893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:22:54.957893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:54.957893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:22:54.957893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:54.957893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:22:54.957893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:22:54.957893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:54.957893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:22:54.958540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:54.958540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:22:54.958540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:54.958540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:22:54.958540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:22:54.958540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:22:54.958540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:22:54.958540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:22:54.958540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:54.958540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:22:54.958540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:22:54.958540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:22:54.958540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:22:54.958540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:22:54.958540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:54.959215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:54.959215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:22:54.959215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:22:54.959215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:22:54.959215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:22:54.959215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:54.959215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:54.959215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:54.959215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:22:54.959215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:54.959215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:54.959215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:54.959215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:22:54.959215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:54.959215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:22:54.959215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:22:54.959215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:54.959215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:22:54.959215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:54.960917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:22:54.960917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:22:54.960917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:54.960917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:22:54.960917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:54.960917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:22:55.137990 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:22:55.138396 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:22:55.138625 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:22:55.139758 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:22:55.140076 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:22:55.140306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:55.141199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:22:55.141199 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:55.141199 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:22:55.141199 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:22:55.141199 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:55.141199 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:22:55.141199 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:55.141199 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:22:55.142884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:55.142884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:22:55.142884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:55.142884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:22:55.142884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:22:55.142884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:22:55.142884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:22:55.142884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:55.142884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:55.142884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:55.142884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:22:55.142884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:55.143468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:55.143468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:55.143468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:22:55.143468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:55.143468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:22:55.143468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:22:55.143468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:55.143468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:22:55.143468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:55.143468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:22:55.143468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:22:55.143468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:55.143468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:22:55.143990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:55.143990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:22:55.144437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:55.144437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:22:55.144437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:22:55.144437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:22:55.144437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:22:55.144437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:22:55.144437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:55.144437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:22:55.144437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:22:55.144437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:22:55.144437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:22:55.144437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:22:55.144437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:55.144437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:55.144437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:22:55.144437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:22:55.145732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:22:55.145732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:22:55.145732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:55.145732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:55.145732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:55.145732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:22:55.145732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:55.145732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:55.145732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:55.145732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:22:55.145732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:55.145732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:22:55.145732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:22:55.145732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:55.145732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:22:55.145732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:55.145732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:22:55.145732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:22:55.145732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:55.146560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:22:55.146560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:55.146560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:22:55.157603 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:22:55.157959 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:22:55.158126 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:22:55.158306 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:22:55.158905 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:22:55.159237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:55.160176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:22:55.160176 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:55.160176 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:22:55.160176 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:22:55.160176 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:55.160176 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:22:55.160176 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:55.160176 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:22:55.161430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:55.161430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:22:55.161430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:22:55.161430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:22:55.161430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:55.161430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:22:55.161430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:55.161430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:22:55.161430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:55.161430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:22:55.161430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:55.161430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:22:55.161430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:55.161430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:22:55.161430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:22:55.162075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:55.162075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:22:55.162075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:55.162075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:22:55.162075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:22:55.162075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:55.162075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:22:55.162075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:55.162075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:22:55.162804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:55.162804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:22:55.162804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:55.162804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:22:55.162804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:22:55.162804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:22:55.162804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:22:55.162804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:22:55.162804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:22:55.162804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:22:55.162804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:55.162804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:55.163341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:55.163341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:22:55.163341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:55.163341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:55.163341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:55.163341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:22:55.163341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:55.163341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:22:55.163341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:22:55.163341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:55.163341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:22:55.163341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:55.163341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:22:55.163341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:22:55.163965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:55.163965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:22:55.163965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:55.163965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:22:55.164523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:55.164523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:22:55.164523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:22:55.164523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:22:55.164523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:22:55.164523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:22:55.164523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:55.164523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:22:55.164523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:22:55.164523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:22:55.164523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:22:55.164523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:22:55.164523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:55.164523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:55.164523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:22:55.164523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:22:55.165258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:22:55.165258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:22:55.165258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:22:55.165258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:22:55.165258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:22:55.165258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:55.165258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:55.165258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:55.165258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:22:55.165258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:55.165258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:55.165258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:55.165258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:22:55.165258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:55.165258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:22:55.165258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:22:55.165258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:55.165258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:22:55.165792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:55.165792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:22:55.165792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:22:55.165792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:55.165792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:22:55.165792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:55.165792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:22:55.182489 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:22:55.182716 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:22:55.182879 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:22:55.183110 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:22:55.183643 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:22:55.183905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:55.184648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:22:55.184648 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:55.184648 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:22:55.184648 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:22:55.184648 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:55.184648 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:22:55.184648 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:55.184648 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:22:55.186032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:55.186032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:22:55.186032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:22:55.186032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:22:55.186032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:55.186032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:22:55.186032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:55.186032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:22:55.186032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:55.186032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:22:55.186032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:55.186032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:22:55.186032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:55.186032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:22:55.186032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:22:55.186758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:55.186758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:22:55.186758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:55.186758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:22:55.186758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:22:55.186758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:55.186758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:22:55.186758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:55.186758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:22:55.187245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:55.187245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:22:55.187245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:55.187245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:22:55.187245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:22:55.187245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:22:55.187245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:22:55.187245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:22:55.187245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:22:55.187245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:55.187245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:55.187926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:55.187926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:22:55.187926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:55.187926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:55.187926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:55.187926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:22:55.187926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:55.187926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:22:55.187926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:22:55.187926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:55.187926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:22:55.187926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:55.187926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:22:55.187926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:22:55.188648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:55.188648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:22:55.188648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:55.188648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:22:55.188907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:55.188907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:22:55.188907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:22:55.188907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:22:55.188907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:22:55.188907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:22:55.188907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:55.188907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:22:55.188907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:22:55.188907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:22:55.188907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:22:55.188907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:22:55.188907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:55.188907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:55.188907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:22:55.188907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:22:55.188907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:22:55.189596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:22:55.189596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:22:55.189596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:22:55.189596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:55.189596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:55.189596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:55.189596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:22:55.189596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:55.189596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:55.189596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:55.189596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:22:55.189596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:55.189596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:22:55.189596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:22:55.189596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:55.189596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:22:55.189596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:55.189596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:22:55.189596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:22:55.190075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:55.190075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:22:55.190075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:55.190075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:22:55.191147 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:22:55.191432 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:22:55.191554 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:22:55.191693 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:22:55.192134 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:22:55.192539 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:22:55.192753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:55.193377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:22:55.193377 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:55.193377 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:22:55.193377 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:22:55.193377 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:55.193377 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:22:55.193377 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:55.193377 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:22:55.194159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:55.194159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:22:55.194159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:55.194159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:22:55.194159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:22:55.194159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:22:55.194159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:22:55.194159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:55.194159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:55.194159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:55.194159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:22:55.194159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:55.194514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:55.194514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:55.194514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:22:55.194514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:55.194514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:22:55.194514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:22:55.194514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:55.194514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:22:55.194514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:55.194514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:22:55.194514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:22:55.194514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:55.194514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:22:55.194855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:55.194855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:22:55.195070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:55.195070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:22:55.195070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:22:55.195070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:22:55.195070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:22:55.195070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:22:55.195070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:55.195070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:22:55.195070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:22:55.195070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:22:55.195070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:22:55.195070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:22:55.195070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:55.195070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:55.195070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:22:55.195070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:22:55.195728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:22:55.195728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:22:55.195728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:55.195728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:55.195728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:55.195728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:22:55.195728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:55.195728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:55.195728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:55.195728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:22:55.195728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:55.195728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:22:55.195728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:22:55.195728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:55.195728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:22:55.195728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:55.195728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:22:55.195728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:22:55.195728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:55.196195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:22:55.196195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:55.196195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:22:59.923955 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:22:59.925004 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:22:59.925004 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:22:59.925900 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:22:59.956258 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:22:59.956799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:59.957642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:22:59.957642 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:59.957642 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:22:59.957642 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:22:59.957642 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:59.957642 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:22:59.957642 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:59.957642 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:22:59.958344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:59.958344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:22:59.958344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:59.958344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:22:59.958344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:22:59.958344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:22:59.958344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:22:59.958344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:59.958344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:59.958344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:59.958344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:22:59.958344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:59.959289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:22:59.959289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:59.959289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:22:59.959289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:59.959289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:22:59.959289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:22:59.959289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:59.959289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:22:59.959289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:59.959289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:22:59.959289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:22:59.959289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:59.959289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:22:59.959971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:59.959971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:22:59.959971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:59.959971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:22:59.959971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:22:59.959971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:22:59.959971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:22:59.959971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:22:59.959971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:59.959971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:22:59.959971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:22:59.959971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:22:59.959971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:22:59.959971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:22:59.959971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:22:59.960639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:22:59.960639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:22:59.960639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:22:59.960639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:22:59.960639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:22:59.960639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:59.960639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:59.960639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:22:59.960639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:22:59.960639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:22:59.960639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:22:59.960639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:22:59.960639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:22:59.960639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:22:59.960639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:22:59.960639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:22:59.960639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:22:59.960639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:22:59.960639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:22:59.961565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:22:59.961565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:22:59.961565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:22:59.961565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:22:59.961565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:22:59.961565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:23:00.141660 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:23:00.141660 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:23:00.141660 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:23:00.142085 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:23:00.142249 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:23:00.142631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:00.143808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:23:00.143808 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:00.143808 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:23:00.143808 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:23:00.143808 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:00.143808 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:23:00.143808 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:00.143808 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:23:00.145196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:00.145196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:23:00.145196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:00.145196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:23:00.145196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:23:00.145196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:23:00.145196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:23:00.145196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:00.145196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:00.145196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:00.145196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:23:00.145196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:00.145806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:00.145806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:00.145806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:23:00.145806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:00.145806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:23:00.145806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:23:00.145806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:00.145806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:23:00.145806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:00.145806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:23:00.145806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:23:00.145806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:00.145806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:23:00.146339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:00.146339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:23:00.150446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:00.150446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:23:00.150446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:23:00.150446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:23:00.150446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:23:00.150446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:23:00.150446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:00.150446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:23:00.150446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:23:00.150446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:23:00.150446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:23:00.150446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:23:00.150446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:00.150446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:00.150446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:23:00.150446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:23:00.150963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:23:00.150963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:23:00.150963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:00.150963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:00.150963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:00.150963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:23:00.150963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:00.150963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:00.150963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:00.150963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:23:00.150963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:00.150963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:23:00.150963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:23:00.150963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:00.150963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:23:00.150963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:00.150963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:23:00.150963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:23:00.150963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:00.151615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:23:00.151615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:00.151615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:23:00.158154 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:23:00.158442 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:23:00.158721 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:23:00.158799 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:23:00.159692 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:23:00.159769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:00.160783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:23:00.160783 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:00.160783 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:23:00.160783 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:23:00.160783 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:00.160783 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:23:00.160783 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:00.160783 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:23:00.161752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:00.161752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:23:00.161752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:23:00.161752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:23:00.161752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:00.161752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:23:00.161752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:00.161752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:23:00.161752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:00.161752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:23:00.161752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:00.161752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:23:00.161752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:00.161752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:23:00.161752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:23:00.162141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:00.162141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:23:00.162141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:00.162141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:23:00.162141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:23:00.162141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:00.162141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:23:00.162141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:00.162141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:23:00.162932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:00.162932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:23:00.162932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:00.162932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:23:00.162932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:23:00.162932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:23:00.162932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:23:00.162932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:23:00.162932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:23:00.162932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:23:00.162932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:00.162932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:00.163323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:00.163323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:23:00.163323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:00.163323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:00.163323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:00.163323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:23:00.163323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:00.163323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:23:00.163323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:23:00.163323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:00.163323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:23:00.163323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:00.163323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:23:00.163323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:23:00.163611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:00.163611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:23:00.163611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:00.163611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:23:00.164633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:00.164633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:23:00.164633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:23:00.164633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:23:00.164633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:23:00.164633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:23:00.164633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:00.164633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:23:00.164633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:23:00.164633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:23:00.164633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:23:00.164633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:23:00.164633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:00.164633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:00.164633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:23:00.164633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:23:00.165040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:23:00.165040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:23:00.165040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:23:00.165040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:23:00.165040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:23:00.165040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:00.165040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:00.165040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:00.165040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:23:00.165040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:00.165040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:00.165040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:00.165040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:23:00.165040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:00.165040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:23:00.165040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:23:00.165040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:00.165040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:23:00.165521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:00.165521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:23:00.165521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:23:00.165521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:00.165521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:23:00.165521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:00.165521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:23:00.183232 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:23:00.183232 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:23:00.183232 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:23:00.183610 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:23:00.183895 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:23:00.184251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:00.185106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:23:00.185106 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:00.185106 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:23:00.185106 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:23:00.185106 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:00.185106 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:23:00.185106 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:00.185106 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:23:00.186266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:00.186266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:23:00.186266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:23:00.186266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:23:00.186266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:00.186266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:23:00.186266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:00.186266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:23:00.186266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:00.186266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:23:00.186266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:00.186266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:23:00.186266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:00.186266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:23:00.186266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:23:00.186752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:00.186752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:23:00.186752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:00.186752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:23:00.186752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:23:00.186752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:00.186752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:23:00.186752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:00.186752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:23:00.187669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:00.187669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:23:00.187669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:00.187669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:23:00.187669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:23:00.187669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:23:00.187669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:23:00.187669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:23:00.187669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:23:00.187669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:00.187669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:00.188022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:00.188022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:23:00.188022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:00.188022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:00.188022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:00.188022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:23:00.188022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:00.188022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:23:00.188022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:23:00.188022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:00.188022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:23:00.188022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:00.188022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:23:00.188022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:23:00.188445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:00.188445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:23:00.188445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:00.188445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:23:00.189511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:00.189511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:23:00.189511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:23:00.189511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:23:00.189511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:23:00.189511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:23:00.189511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:00.189511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:23:00.189511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:23:00.189511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:23:00.189511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:23:00.189511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:23:00.189511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:00.189511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:00.189511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:23:00.189511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:23:00.189511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:23:00.190523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:23:00.190523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:23:00.190523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:23:00.190523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:00.190523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:00.190523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:00.190523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:23:00.190523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:00.190523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:00.190523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:00.190523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:23:00.190523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:00.190523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:23:00.190523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:23:00.190523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:00.190523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:23:00.190523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:00.190523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:23:00.190523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:23:00.191100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:00.191100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:23:00.191100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:00.191100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:23:00.191546 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:23:00.192054 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:23:00.192183 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:23:00.192332 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:23:00.193033 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:23:00.193117 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:23:00.193987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:00.194060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:23:00.194060 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:00.194060 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:23:00.194060 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:23:00.194060 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:00.194060 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:23:00.194060 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:00.194060 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:23:00.195129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:00.195129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:23:00.195129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:00.195129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:23:00.195129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:23:00.195129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:23:00.195129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:23:00.195129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:00.195129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:00.195129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:00.195129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:23:00.195129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:00.195605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:00.195605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:00.195605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:23:00.195605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:00.195605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:23:00.195605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:23:00.195605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:00.195605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:23:00.195605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:00.195605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:23:00.195605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:23:00.195605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:00.195605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:23:00.195983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:00.195983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:23:00.195983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:00.195983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:23:00.195983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:23:00.195983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:23:00.195983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:23:00.195983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:23:00.195983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:00.195983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:23:00.195983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:23:00.195983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:23:00.195983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:23:00.195983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:23:00.195983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:00.196542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:00.196542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:23:00.196542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:23:00.196542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:23:00.196542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:23:00.196542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:00.196542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:00.196542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:00.196542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:23:00.196542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:00.196542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:00.196542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:00.196542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:23:00.196542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:00.196542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:23:00.196542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:23:00.196542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:00.196542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:23:00.196542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:00.197114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:23:00.197114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:23:00.197114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:00.197114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:23:00.197114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:00.197114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:23:04.924701 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:23:04.925629 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:23:04.925629 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:23:04.926887 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:23:04.960137 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:23:04.960581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:04.961356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:23:04.961356 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:04.961356 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:23:04.961356 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:23:04.961356 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:04.961356 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:23:04.961356 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:04.961356 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:23:04.962200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:04.962200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:23:04.962200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:04.962200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:23:04.962200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:23:04.962200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:23:04.962200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:23:04.962200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:04.962200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:04.962200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:04.962200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:23:04.962200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:04.962886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:04.962886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:04.962886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:23:04.962886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:04.962886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:23:04.962886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:23:04.962886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:04.962886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:23:04.962886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:04.962886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:23:04.962886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:23:04.962886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:04.962886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:23:04.963534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:04.963534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:23:04.963534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:04.963534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:23:04.963534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:23:04.963534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:23:04.963534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:23:04.963534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:23:04.963534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:04.963534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:23:04.963534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:23:04.963534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:23:04.963534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:23:04.963534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:23:04.963534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:04.964280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:04.964280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:23:04.964280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:23:04.964280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:23:04.964280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:23:04.964280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:04.964280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:04.964280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:04.964280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:23:04.964280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:04.964280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:04.964280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:04.964280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:23:04.964280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:04.964280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:23:04.964280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:23:04.964280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:04.964280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:23:04.964280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:04.965255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:23:04.965255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:23:04.965255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:04.965255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:23:04.965255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:04.965255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:23:05.139296 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:23:05.139962 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:23:05.140253 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:23:05.142091 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:23:05.142091 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:23:05.142091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:05.142382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:23:05.142382 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:05.142382 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:23:05.142382 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:23:05.142382 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:05.142382 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:23:05.142382 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:05.142382 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:23:05.143958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:05.143958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:23:05.143958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:05.143958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:23:05.143958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:23:05.143958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:23:05.143958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:23:05.143958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:05.143958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:05.143958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:05.143958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:23:05.143958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:05.144331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:05.144331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:05.144331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:23:05.144331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:05.144331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:23:05.144331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:23:05.144331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:05.144331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:23:05.144331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:05.144331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:23:05.144331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:23:05.144331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:05.144331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:23:05.144712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:05.144712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:23:05.145894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:05.145894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:23:05.145894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:23:05.145894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:23:05.145894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:23:05.145894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:23:05.145894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:05.145894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:23:05.145894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:23:05.145894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:23:05.145894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:23:05.145894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:23:05.145894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:05.145894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:05.145894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:23:05.145894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:23:05.146375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:23:05.146375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:23:05.146375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:05.146375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:05.146375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:05.146375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:23:05.146375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:05.146375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:05.146375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:05.146375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:23:05.146375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:05.146375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:23:05.146375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:23:05.146375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:05.146375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:23:05.146375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:05.146375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:23:05.146375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:23:05.146375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:05.147349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:23:05.147349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:05.147349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:23:05.158609 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:23:05.158950 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:23:05.159109 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:23:05.159258 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:23:05.159772 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:23:05.160077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:05.160779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:23:05.160779 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:05.160779 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:23:05.160779 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:23:05.160779 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:05.160779 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:23:05.160779 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:05.160779 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:23:05.162152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:05.162152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:23:05.162152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:23:05.162152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:23:05.162152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:05.162152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:23:05.162152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:05.162152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:23:05.162152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:05.162152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:23:05.162152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:05.162152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:23:05.162152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:05.162152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:23:05.162152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:23:05.162710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:05.162710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:23:05.162710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:05.162710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:23:05.162710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:23:05.162710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:05.162710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:23:05.162710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:05.162710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:23:05.163809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:05.163809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:23:05.163809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:05.163809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:23:05.163809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:23:05.163809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:23:05.163809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:23:05.163809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:23:05.163809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:23:05.163809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:23:05.163809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:05.163809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:05.164209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:05.164209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:23:05.164209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:05.164209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:05.164209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:05.164209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:23:05.164209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:05.164209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:23:05.164209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:23:05.164209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:05.164209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:23:05.164209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:05.164209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:23:05.164209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:23:05.164615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:05.164615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:23:05.164615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:05.164615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:23:05.165370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:05.165370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:23:05.165370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:23:05.165370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:23:05.165370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:23:05.165370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:23:05.165370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:05.165370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:23:05.165370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:23:05.165370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:23:05.165370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:23:05.165370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:23:05.165370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:05.165370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:05.165370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:23:05.165370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:23:05.165882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:23:05.165882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:23:05.165882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:23:05.165882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:23:05.165882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:23:05.165882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:05.165882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:05.165882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:05.165882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:23:05.165882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:05.165882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:05.165882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:05.165882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:23:05.165882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:05.165882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:23:05.165882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:23:05.165882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:05.165882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:23:05.166438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:05.166438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:23:05.166438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:23:05.166438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:05.166438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:23:05.166438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:05.166438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:23:05.183335 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:23:05.183630 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:23:05.183776 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:23:05.183992 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:23:05.185123 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:23:05.185123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:05.185301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:23:05.185301 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:05.185301 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:23:05.185301 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:23:05.185301 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:05.185301 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:23:05.185301 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:05.185301 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:23:05.186156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:05.186156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:23:05.186156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:23:05.186156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:23:05.186156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:05.186156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:23:05.186156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:05.186156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:23:05.186156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:05.186156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:23:05.186156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:05.186156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:23:05.186156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:05.186156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:23:05.186156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:23:05.186501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:05.186501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:23:05.186501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:05.186501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:23:05.186501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:23:05.186501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:05.186501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:23:05.186501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:05.186501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:23:05.187270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:05.187270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:23:05.187270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:05.187270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:23:05.187270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:23:05.187270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:23:05.187270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:23:05.187270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:23:05.187270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:23:05.187270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:05.187270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:05.187568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:05.187568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:23:05.187568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:05.187568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:05.187568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:05.187568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:23:05.187568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:05.187568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:23:05.187568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:23:05.187568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:05.187568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:23:05.187568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:05.187568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:23:05.187568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:23:05.187848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:05.187848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:23:05.187848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:05.187848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:23:05.188482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:05.188482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:23:05.188482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:23:05.188482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:23:05.188482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:23:05.188482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:23:05.188482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:05.188482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:23:05.188482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:23:05.188482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:23:05.188482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:23:05.188482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:23:05.188482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:05.188482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:05.188482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:23:05.188482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:23:05.188482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:23:05.188911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:23:05.188911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:23:05.188911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:23:05.188911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:05.188911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:05.188911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:05.188911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:23:05.188911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:05.188911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:05.188911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:05.188911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:23:05.188911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:05.188911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:23:05.188911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:23:05.188911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:05.188911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:23:05.188911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:05.188911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:23:05.188911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:23:05.189249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:05.189249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:23:05.189249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:05.189249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:23:05.191987 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:23:05.192255 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:23:05.192367 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:23:05.192547 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:23:05.192908 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:23:05.193286 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:23:05.193582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:05.194218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:23:05.194218 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:05.194218 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:23:05.194218 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:23:05.194218 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:05.194218 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:23:05.194218 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:05.194218 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:23:05.195054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:05.195054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:23:05.195054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:05.195054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:23:05.195054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:23:05.195054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:23:05.195054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:23:05.195054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:05.195054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:05.195054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:05.195054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:23:05.195054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:05.195439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:05.195439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:05.195439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:23:05.195439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:05.195439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:23:05.195439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:23:05.195439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:05.195439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:23:05.195439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:05.195439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:23:05.195439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:23:05.195439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:05.195439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:23:05.195811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:05.195811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:23:05.196007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:05.196007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:23:05.196007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:23:05.196007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:23:05.196007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:23:05.196007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:23:05.196007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:05.196007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:23:05.196007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:23:05.196007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:23:05.196007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:23:05.196007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:23:05.196007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:05.196007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:05.196007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:23:05.196007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:23:05.196514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:23:05.196514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:23:05.196514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:05.196514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:05.196514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:05.196514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:23:05.196514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:05.196514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:05.196514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:05.196514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:23:05.196514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:05.196514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:23:05.196514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:23:05.196514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:05.196514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:23:05.196514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:05.196514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:23:05.196514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:23:05.196514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:05.197030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:23:05.197030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:05.197030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:23:09.925489 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:23:09.926291 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:23:09.926291 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:23:09.927477 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:23:09.959526 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:23:09.959909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:09.960777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:23:09.960777 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:09.960777 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:23:09.960777 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:23:09.960777 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:09.960777 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:23:09.960777 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:09.960777 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:23:09.961691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:09.961691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:23:09.961691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:09.961691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:23:09.961691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:23:09.961691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:23:09.961691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:23:09.961691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:09.961691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:09.961691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:09.961691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:23:09.961691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:09.962431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:09.962431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:09.962431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:23:09.962431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:09.962431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:23:09.962431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:23:09.962431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:09.962431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:23:09.962431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:09.962431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:23:09.962431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:23:09.962431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:09.962431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:23:09.963118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:09.963118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:23:09.963118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:09.963118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:23:09.963118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:23:09.963118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:23:09.963118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:23:09.963118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:23:09.963118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:09.963118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:23:09.963118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:23:09.963118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:23:09.963118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:23:09.963118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:23:09.963118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:09.963960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:09.963960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:23:09.963960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:23:09.963960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:23:09.963960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:23:09.963960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:09.963960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:09.963960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:09.963960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:23:09.963960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:09.963960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:09.963960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:09.963960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:23:09.963960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:09.963960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:23:09.963960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:23:09.963960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:09.963960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:23:09.963960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:09.965006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:23:09.965006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:23:09.965006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:09.965006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:23:09.965006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:09.965006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:23:10.141445 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:23:10.141445 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:23:10.141445 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:23:10.141973 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:23:10.142289 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:23:10.142547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:10.143393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:23:10.143393 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:10.143393 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:23:10.143393 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:23:10.143393 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:10.143393 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:23:10.143393 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:10.143393 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:23:10.145050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:10.145050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:23:10.145050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:10.145050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:23:10.145050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:23:10.145050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:23:10.145050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:23:10.145050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:10.145050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:10.145050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:10.145050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:23:10.145050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:10.145559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:10.145559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:10.145559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:23:10.145559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:10.145559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:23:10.145559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:23:10.145559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:10.145559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:23:10.145559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:10.145559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:23:10.145559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:23:10.145559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:10.145559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:23:10.145899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:10.145899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:23:10.146978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:10.146978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:23:10.146978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:23:10.146978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:23:10.146978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:23:10.146978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:23:10.146978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:10.146978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:23:10.146978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:23:10.146978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:23:10.146978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:23:10.146978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:23:10.146978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:10.146978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:10.146978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:23:10.146978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:23:10.147379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:23:10.147379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:23:10.147379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:10.147379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:10.147379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:10.147379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:23:10.147379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:10.147379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:10.147379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:10.147379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:23:10.147379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:10.147379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:23:10.147379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:23:10.147379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:10.147379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:23:10.147379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:10.147379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:23:10.147379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:23:10.147379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:10.149199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:23:10.149199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:10.149199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:23:10.159077 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:23:10.159594 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:23:10.159810 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:23:10.160209 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:23:10.160535 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:23:10.160812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:10.161597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:23:10.161597 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:10.161597 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:23:10.161597 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:23:10.161597 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:10.161597 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:23:10.161597 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:10.161597 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:23:10.162866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:10.162866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:23:10.162866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:23:10.162866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:23:10.162866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:10.162866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:23:10.162866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:10.162866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:23:10.162866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:10.162866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:23:10.162866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:10.162866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:23:10.162866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:10.162866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:23:10.162866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:23:10.163202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:10.163202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:23:10.163202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:10.163202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:23:10.163202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:23:10.163202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:10.163202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:23:10.163202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:10.163202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:23:10.164586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:10.164586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:23:10.164586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:10.164586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:23:10.164586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:23:10.164586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:23:10.164586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:23:10.164586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:23:10.164586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:23:10.164586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:23:10.164586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:10.164586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:10.164878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:10.164878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:23:10.164878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:10.164878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:10.164878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:10.164878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:23:10.164878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:10.164878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:23:10.164878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:23:10.164878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:10.164878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:23:10.164878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:10.164878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:23:10.164878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:23:10.165136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:10.165136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:23:10.165136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:10.165136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:23:10.166427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:10.166427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:23:10.166427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:23:10.166427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:23:10.166427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:23:10.166427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:23:10.166427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:10.166427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:23:10.166427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:23:10.166427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:23:10.166427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:23:10.166427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:23:10.166427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:10.166427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:10.166427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:23:10.166427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:23:10.166807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:23:10.166807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:23:10.166807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:23:10.166807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:23:10.166807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:23:10.166807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:10.166807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:10.166807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:10.166807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:23:10.166807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:10.166807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:10.166807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:10.166807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:23:10.166807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:10.166807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:23:10.166807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:23:10.166807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:10.166807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:23:10.167167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:10.167167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:23:10.167167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:23:10.167167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:10.167167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:23:10.167167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:10.167167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:23:10.183776 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:23:10.184102 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:23:10.184319 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:23:10.184638 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:23:10.185123 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:23:10.185490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:10.186236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:23:10.186236 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:10.186236 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:23:10.186236 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:23:10.186236 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:10.186236 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:23:10.186236 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:10.186236 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:23:10.187512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:10.187512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:23:10.187512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:23:10.187512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:23:10.187512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:10.187512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:23:10.187512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:10.187512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:23:10.187512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:10.187512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:23:10.187512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:10.187512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:23:10.187512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:10.187512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:23:10.187512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:23:10.188174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:10.188174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:23:10.188174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:10.188174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:23:10.188174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:23:10.188174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:10.188174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:23:10.188174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:10.188174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:23:10.189000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:10.189000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:23:10.189000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:10.189000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:23:10.189000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:23:10.189000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:23:10.189000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:23:10.189000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:23:10.189000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:23:10.189000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:10.189000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:10.189605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:10.189605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:23:10.189605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:10.189605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:10.189605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:10.189605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:23:10.189605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:10.189605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:23:10.189605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:23:10.189605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:10.189605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:23:10.189605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:10.189605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:23:10.189605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:23:10.190228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:10.190228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:23:10.190228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:10.190228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:23:10.190793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:10.190793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:23:10.190793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:23:10.190793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:23:10.190793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:23:10.190793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:23:10.190793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:10.190793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:23:10.190793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:23:10.190793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:23:10.190793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:23:10.190793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:23:10.190793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:10.190793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:10.190793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:23:10.190793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:23:10.190793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:23:10.191619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:23:10.191619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:23:10.191619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:23:10.191619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:10.191619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:10.191619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:10.191619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:23:10.191619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:10.191619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:10.191619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:10.191619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:23:10.191619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:10.191619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:23:10.191619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:23:10.191619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:10.191619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:23:10.191619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:10.191619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:23:10.191619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:23:10.192462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:10.192462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:23:10.192462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:10.192462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:23:10.192462 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:23:10.192764 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:23:10.192944 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:23:10.193246 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:23:10.193725 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:23:10.194057 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:23:10.194287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:10.194936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:23:10.194936 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:10.194936 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:23:10.194936 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:23:10.194936 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:10.194936 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:23:10.194936 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:10.194936 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:23:10.195794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:10.195794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:23:10.195794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:10.195794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:23:10.195794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:23:10.195794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:23:10.195794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:23:10.195794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:10.195794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:10.195794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:10.195794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:23:10.195794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:10.196496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:10.196496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:10.196496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:23:10.196496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:10.196496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:23:10.196496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:23:10.196496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:10.196496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:23:10.196496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:10.196496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:23:10.196496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:23:10.196496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:10.196496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:23:10.197040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:10.197040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:23:10.197040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:10.197040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:23:10.197040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:23:10.197040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:23:10.197040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:23:10.197040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:23:10.197040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:10.197040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:23:10.197040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:23:10.197040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:23:10.197040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:23:10.197040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:23:10.197040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:10.197495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:10.197495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:23:10.197495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:23:10.197495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:23:10.197495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:23:10.197495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:10.197495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:10.197495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:10.197495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:23:10.197495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:10.197495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:10.197495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:10.197495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:23:10.197495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:10.197495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:23:10.197495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:23:10.197495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:10.197495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:23:10.197495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:10.198090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:23:10.198090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:23:10.198090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:10.198090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:23:10.198090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:10.198090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:23:14.926337 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:23:14.927270 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:23:14.927270 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:23:14.928379 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:23:14.961822 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:23:14.962168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:14.963012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:23:14.963012 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:14.963012 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:23:14.963012 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:23:14.963012 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:14.963012 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:23:14.963012 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:14.963012 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:23:14.963720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:14.963720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:23:14.963720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:14.963720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:23:14.963720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:23:14.963720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:23:14.963720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:23:14.963720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:14.963720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:14.963720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:14.963720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:23:14.963720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:14.964351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:14.964351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:14.964351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:23:14.964351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:14.964351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:23:14.964351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:23:14.964351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:14.964351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:23:14.964351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:14.964351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:23:14.964351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:23:14.964351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:14.964351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:23:14.965010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:14.965010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:23:14.965010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:14.965010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:23:14.965010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:23:14.965010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:23:14.965010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:23:14.965010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:23:14.965010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:14.965010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:23:14.965010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:23:14.965010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:23:14.965010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:23:14.965010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:23:14.965010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:14.965817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:14.965817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:23:14.965817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:23:14.965817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:23:14.965817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:23:14.965817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:14.965817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:14.965817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:14.965817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:23:14.965817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:14.965817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:14.965817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:14.965817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:23:14.965817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:14.965817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:23:14.965817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:23:14.965817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:14.965817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:23:14.965817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:14.966707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:23:14.966707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:23:14.966707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:14.966707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:23:14.966707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:14.966707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:23:15.140853 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:23:15.141423 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:23:15.141637 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:23:15.142948 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:23:15.143091 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:23:15.143185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:15.143883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:23:15.143883 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:15.143883 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:23:15.143883 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:23:15.143883 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:15.143883 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:23:15.143883 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:15.143883 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:23:15.145322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:15.145322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:23:15.145322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:15.145322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:23:15.145322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:23:15.145322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:23:15.145322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:23:15.145322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:15.145322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:15.145322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:15.145322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:23:15.145322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:15.145980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:15.145980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:15.145980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:23:15.145980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:15.145980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:23:15.145980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:23:15.145980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:15.145980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:23:15.145980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:15.145980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:23:15.145980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:23:15.145980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:15.145980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:23:15.146613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:15.146613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:23:15.146989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:15.146989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:23:15.146989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:23:15.146989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:23:15.146989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:23:15.146989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:23:15.146989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:15.146989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:23:15.146989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:23:15.146989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:23:15.146989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:23:15.146989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:23:15.146989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:15.146989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:15.146989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:23:15.146989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:23:15.147837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:23:15.147837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:23:15.147837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:15.147837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:15.147837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:15.147837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:23:15.147837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:15.147837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:15.147837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:15.147837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:23:15.147837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:15.147837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:23:15.147837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:23:15.147837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:15.147837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:23:15.147837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:15.147837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:23:15.147837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:23:15.147837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:15.148750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:23:15.148750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:15.148750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:23:15.159729 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:23:15.160120 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:23:15.160241 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:23:15.160316 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:23:15.160731 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:23:15.160892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:15.161471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:23:15.161471 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:15.161471 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:23:15.161471 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:23:15.161471 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:15.161471 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:23:15.161471 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:15.161471 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:23:15.162291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:15.162291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:23:15.162291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:23:15.162291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:23:15.162291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:15.162291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:23:15.162291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:15.162291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:23:15.162291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:15.162291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:23:15.162291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:15.162291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:23:15.162291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:15.162291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:23:15.162291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:23:15.163120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:15.163120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:23:15.163120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:15.163120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:23:15.163120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:23:15.163120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:15.163120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:23:15.163120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:15.163120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:23:15.163120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:15.163120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:23:15.163120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:15.163120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:23:15.163120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:23:15.163882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:23:15.163882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:23:15.163882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:23:15.163882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:23:15.163882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:23:15.163882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:15.163882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:15.163882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:15.163882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:23:15.163882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:15.163882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:15.163882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:15.163882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:23:15.164504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:15.164504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:23:15.164504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:23:15.164504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:15.164504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:23:15.164504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:15.164504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:23:15.164504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:23:15.164504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:15.164504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:23:15.164504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:15.164504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:23:15.165034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:15.165034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:23:15.165034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:23:15.165034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:23:15.165034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:23:15.165034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:23:15.165034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:15.165034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:23:15.165034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:23:15.165034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:23:15.165034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:23:15.165034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:23:15.165034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:15.165034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:15.165034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:23:15.165034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:23:15.165768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:23:15.165768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:23:15.165768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:23:15.165768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:23:15.165768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:23:15.165768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:15.165768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:15.165768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:15.165768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:23:15.165768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:15.165768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:15.165768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:15.165768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:23:15.165768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:15.165768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:23:15.165768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:23:15.165768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:15.165768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:23:15.166679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:15.166679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:23:15.166679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:23:15.166679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:15.166679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:23:15.166679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:15.166679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:23:15.184205 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:23:15.184500 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:23:15.184587 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:23:15.184655 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:23:15.185154 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:23:15.185543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:15.186230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:23:15.186230 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:15.186230 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:23:15.186230 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:23:15.186230 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:15.186230 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:23:15.186230 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:15.186230 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:23:15.186793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:15.186793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:23:15.186793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:23:15.186793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:23:15.186793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:15.186793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:23:15.186793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:15.186793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:23:15.186793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:15.186793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:23:15.186793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:15.186793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:23:15.186793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:15.186793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:23:15.186793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:23:15.187488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:15.187488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:23:15.187488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:15.187488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:23:15.187488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:23:15.187488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:15.187488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:23:15.187488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:15.187488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:23:15.187488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:15.187488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:23:15.187488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:15.187488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:23:15.187488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:23:15.188138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:23:15.188138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:23:15.188138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:23:15.188138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:23:15.188138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:15.188138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:15.188138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:15.188138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:23:15.188138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:15.188138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:15.188138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:15.188138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:23:15.188138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:15.188741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:23:15.188741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:23:15.188741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:15.188741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:23:15.188741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:15.188741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:23:15.188741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:23:15.188741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:15.188741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:23:15.188741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:15.188741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:23:15.188741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:15.189301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:23:15.189301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:23:15.189301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:23:15.189301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:23:15.189301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:23:15.189301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:15.189301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:23:15.189301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:23:15.189301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:23:15.189301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:23:15.189301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:23:15.189301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:15.189301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:15.189301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:23:15.189301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:23:15.189301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:23:15.189301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:23:15.189301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:23:15.189301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:23:15.190134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:15.190134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:15.190134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:15.190134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:23:15.190134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:15.190134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:15.190134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:15.190134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:23:15.190134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:15.190134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:23:15.190134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:23:15.190134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:15.190134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:23:15.190134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:15.190134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:23:15.190134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:23:15.190134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:15.190134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:23:15.190134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:15.190134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:23:15.192869 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:23:15.193182 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:23:15.193292 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:23:15.193459 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:23:15.193870 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:23:15.194292 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:23:15.194536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:15.195236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:23:15.195236 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:15.195236 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:23:15.195236 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:23:15.195236 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:15.195236 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:23:15.195236 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:15.195236 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:23:15.196043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:15.196043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:23:15.196043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:15.196043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:23:15.196043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:23:15.196043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:23:15.196043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:23:15.196043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:15.196043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:15.196043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:15.196043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:23:15.196043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:15.196403 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:15.196403 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:15.196403 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:23:15.196403 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:15.196403 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:23:15.196403 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:23:15.196403 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:15.196403 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:23:15.196403 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:15.196403 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:23:15.196403 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:23:15.196403 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:15.196403 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:23:15.196785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:15.196785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:23:15.196984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:15.196984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:23:15.196984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:23:15.196984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:23:15.196984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:23:15.196984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:23:15.196984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:15.196984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:23:15.196984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:23:15.196984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:23:15.196984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:23:15.196984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:23:15.196984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:15.196984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:15.196984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:23:15.196984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:23:15.197448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:23:15.197448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:23:15.197448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:15.197448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:15.197448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:15.197448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:23:15.197448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:15.197448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:15.197448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:15.197448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:23:15.197448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:15.197448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:23:15.197448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:23:15.197448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:15.197448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:23:15.197448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:15.197448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:23:15.197448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:23:15.197448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:15.197973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:23:15.197973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:15.197973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:23:19.927133 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:23:19.928434 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:23:19.928434 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:23:19.929141 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:23:19.967434 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:23:19.967883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:19.968740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:23:19.968740 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:19.968740 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:23:19.968740 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:23:19.968740 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:19.968740 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:23:19.968740 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:19.968740 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:23:19.969737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:19.969737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:23:19.969737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:19.969737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:23:19.969737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:23:19.969737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:23:19.969737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:23:19.969737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:19.969737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:19.969737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:19.969737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:23:19.969737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:19.970305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:19.970305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:19.970305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:23:19.970305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:19.970305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:23:19.970305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:23:19.970305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:19.970305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:23:19.970305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:19.970305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:23:19.970305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:23:19.970305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:19.970305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:23:19.970863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:19.970863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:23:19.970863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:19.970863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:23:19.970863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:23:19.970863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:23:19.970863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:23:19.970863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:23:19.970863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:19.970863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:23:19.970863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:23:19.970863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:23:19.970863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:23:19.970863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:23:19.970863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:19.971631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:19.971631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:23:19.971631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:23:19.971631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:23:19.971631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:23:19.971631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:19.971631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:19.971631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:19.971631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:23:19.971631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:19.971631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:19.971631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:19.971631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:23:19.971631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:19.971631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:23:19.971631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:23:19.971631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:19.971631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:23:19.971631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:19.972534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:23:19.972534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:23:19.972534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:19.972534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:23:19.972534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:19.972534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:23:20.141630 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:23:20.142167 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:23:20.142352 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:23:20.143465 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:23:20.143778 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:23:20.144027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:20.144971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:23:20.144971 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:20.144971 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:23:20.144971 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:23:20.144971 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:20.144971 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:23:20.144971 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:20.144971 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:23:20.146506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:20.146506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:23:20.146506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:20.146506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:23:20.146506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:23:20.146506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:23:20.146506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:23:20.146506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:20.146506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:20.146506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:20.146506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:23:20.146506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:20.146901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:20.146901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:20.146901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:23:20.146901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:20.146901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:23:20.146901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:23:20.146901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:20.146901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:23:20.146901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:20.146901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:23:20.146901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:23:20.146901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:20.146901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:23:20.147397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:20.147397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:23:20.148514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:20.148514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:23:20.148514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:23:20.148514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:23:20.148514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:23:20.148514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:23:20.148514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:20.148514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:23:20.148514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:23:20.148514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:23:20.148514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:23:20.148514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:23:20.148514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:20.148514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:20.148514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:23:20.148514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:23:20.149115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:23:20.149115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:23:20.149115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:20.149115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:20.149115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:20.149115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:23:20.149115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:20.149115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:20.149115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:20.149115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:23:20.149115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:20.149115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:23:20.149115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:23:20.149115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:20.149115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:23:20.149115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:20.149115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:23:20.149115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:23:20.149115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:20.149670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:23:20.149670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:20.149670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:23:20.160205 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:23:20.160591 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:23:20.160741 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:23:20.160903 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:23:20.161390 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:23:20.161714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:20.162468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:23:20.162468 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:20.162468 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:23:20.162468 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:23:20.162468 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:20.162468 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:23:20.162468 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:20.162468 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:23:20.163379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:20.163379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:23:20.163379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:23:20.163379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:23:20.163379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:20.163379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:23:20.163379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:20.163379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:23:20.163379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:20.163379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:23:20.163379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:20.163379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:23:20.163379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:20.163379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:23:20.163379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:23:20.164189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:20.164189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:23:20.164189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:20.164189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:23:20.164189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:23:20.164189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:20.164189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:23:20.164189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:20.164189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:23:20.164819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:20.164819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:23:20.164819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:20.164819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:23:20.164819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:23:20.164819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:23:20.164819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:23:20.164819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:23:20.164819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:23:20.164819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:23:20.164819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:20.164819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:20.165468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:20.165468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:23:20.165468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:20.165468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:20.165468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:20.165468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:23:20.165468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:20.165468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:23:20.165468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:23:20.165468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:20.165468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:23:20.165468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:20.165468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:23:20.165468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:23:20.166149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:20.166149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:23:20.166149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:20.166149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:23:20.166149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:20.166149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:23:20.166149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:23:20.166149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:23:20.166149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:23:20.166149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:23:20.166149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:20.166149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:23:20.166149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:23:20.166149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:23:20.166869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:23:20.166869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:23:20.166869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:20.166869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:20.166869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:23:20.166869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:23:20.166869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:23:20.166869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:23:20.166869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:23:20.166869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:23:20.166869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:23:20.166869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:20.166869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:20.166869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:20.166869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:23:20.166869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:20.166869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:20.166869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:20.166869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:23:20.167827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:20.167827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:23:20.167827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:23:20.167827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:20.167827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:23:20.167827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:20.167827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:23:20.167827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:23:20.167827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:20.167827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:23:20.167827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:20.167827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:23:20.184686 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:23:20.185145 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:23:20.185320 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:23:20.185529 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:23:20.186028 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:23:20.186357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:20.187153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:23:20.187153 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:20.187153 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:23:20.187153 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:23:20.187153 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:20.187153 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:23:20.187153 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:20.187153 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:23:20.189854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:20.189854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:23:20.189854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:23:20.189854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:23:20.189854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:20.189854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:23:20.189854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:20.189854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:23:20.189854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:20.189854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:23:20.189854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:20.189854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:23:20.189854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:20.189854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:23:20.189854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:23:20.190308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:20.190308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:23:20.190308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:20.190308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:23:20.190308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:23:20.190308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:20.190308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:23:20.190308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:20.190308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:23:20.191295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:20.191295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:23:20.191295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:20.191295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:23:20.191295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:23:20.191295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:23:20.191295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:23:20.191295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:23:20.191295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:23:20.191295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:20.191295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:20.191684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:20.191684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:23:20.191684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:20.191684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:20.191684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:20.191684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:23:20.191684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:20.191684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:23:20.191684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:23:20.191684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:20.191684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:23:20.191684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:20.191684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:23:20.191684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:23:20.192051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:20.192051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:23:20.192051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:20.192051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:23:20.192987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:20.192987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:23:20.192987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:23:20.192987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:23:20.192987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:23:20.192987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:23:20.192987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:20.192987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:23:20.192987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:23:20.192987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:23:20.192987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:23:20.192987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:23:20.192987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:20.192987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:20.192987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:23:20.192987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:23:20.192987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:23:20.193454 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:23:20.193454 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:23:20.193454 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:23:20.193454 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:20.193454 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:20.193454 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:20.193454 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:23:20.193454 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:20.193454 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:20.193454 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:20.193454 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:23:20.193454 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:20.193454 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:23:20.193454 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:23:20.193454 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:20.193454 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:23:20.193454 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:20.193454 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:23:20.193454 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:23:20.193933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:20.193933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:23:20.193933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:20.193933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:23:20.193933 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:23:20.193933 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:23:20.193933 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:23:20.194152 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:23:20.194489 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:23:20.194845 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:23:20.195107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:20.195917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:23:20.195917 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:20.195917 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:23:20.195917 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:23:20.195917 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:20.195917 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:23:20.195917 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:20.195917 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:23:20.196982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:20.196982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:23:20.196982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:20.196982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:23:20.196982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:23:20.196982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:23:20.196982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:23:20.196982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:20.196982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:20.196982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:20.196982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:23:20.196982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:20.197326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:20.197326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:20.197326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:23:20.197326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:20.197326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:23:20.197326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:23:20.197326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:20.197326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:23:20.197326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:20.197326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:23:20.197326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:23:20.197326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:20.197326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:23:20.197696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:20.197696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:23:20.198210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:20.198210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:23:20.198210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:23:20.198210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:23:20.198210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:23:20.198210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:23:20.198210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:20.198210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:23:20.198210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:23:20.198210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:23:20.198210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:23:20.198210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:23:20.198210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:20.198210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:20.198210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:23:20.198210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:23:20.198655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:23:20.198655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:23:20.198655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:20.198655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:20.198655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:20.198655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:23:20.198655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:20.198655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:20.198655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:20.198655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:23:20.198655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:20.198655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:23:20.198655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:23:20.198655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:20.198655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:23:20.198655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:20.198655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:23:20.198655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:23:20.198655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:20.199177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:23:20.199177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:20.199177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:23:24.927903 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:23:24.928614 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:23:24.928614 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:23:24.929891 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:23:24.931822 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG dbcounter [-] [105581] Writing DB stats neutron:SELECT=240 {{(pid=105581) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 14:23:24.965625 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:23:24.965896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:24.966806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:23:24.966806 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:24.966806 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:23:24.966806 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:23:24.966806 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:24.966806 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:23:24.966806 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:24.966806 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:23:24.968426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:24.968426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:23:24.968426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:24.968426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:23:24.968426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:23:24.968426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:23:24.968426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:23:24.968426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:24.968426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:24.968426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:24.968426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:23:24.968426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:24.969279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:24.969279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:24.969279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:23:24.969279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:24.969279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:23:24.969279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:23:24.969279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:24.969279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:23:24.969279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:24.969279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:23:24.969279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:23:24.969279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:24.969279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:23:24.970060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:24.970060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:23:24.970060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:24.970060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:23:24.970060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:23:24.970060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:23:24.970060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:23:24.970060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:23:24.970060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:24.970060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:23:24.970060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:23:24.970060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:23:24.970060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:23:24.970060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:23:24.970060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:24.970919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:24.970919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:23:24.970919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:23:24.970919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:23:24.970919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:23:24.970919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:24.970919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:24.970919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:24.970919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:23:24.970919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:24.970919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:24.970919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:24.970919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:23:24.970919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:24.970919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:23:24.970919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:23:24.970919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:24.970919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:23:24.970919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:24.973375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:23:24.973375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:23:24.973375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:24.973375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:23:24.973375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:24.973375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:23:25.142167 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:23:25.142844 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:23:25.143006 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:23:25.144496 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:23:25.144496 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:23:25.144798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:25.145548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:23:25.145548 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:25.145548 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:23:25.145548 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:23:25.145548 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:25.145548 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:23:25.145548 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:25.145548 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:23:25.146982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:25.146982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:23:25.146982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:25.146982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:23:25.146982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:23:25.146982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:23:25.146982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:23:25.146982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:25.146982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:25.146982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:25.146982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:23:25.146982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:25.147344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:25.147344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:25.147344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:23:25.147344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:25.147344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:23:25.147344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:23:25.147344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:25.147344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:23:25.147344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:25.147344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:23:25.147344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:23:25.147344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:25.147344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:23:25.147734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:25.147734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:23:25.148230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:25.148230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:23:25.148230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:23:25.148230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:23:25.148230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:23:25.148230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:23:25.148230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:25.148230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:23:25.148230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:23:25.148230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:23:25.148230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:23:25.148230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:23:25.148230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:25.148230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:25.148230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:23:25.148230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:23:25.148680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:23:25.148680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:23:25.148680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:25.148680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:25.148680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:25.148680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:23:25.148680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:25.148680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:25.148680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:25.148680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:23:25.148680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:25.148680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:23:25.148680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:23:25.148680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:25.148680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:23:25.148680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:25.148680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:23:25.148680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:23:25.148680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:25.149194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:23:25.149194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:25.149194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:23:25.160767 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:23:25.161152 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:23:25.161336 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:23:25.161574 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:23:25.162786 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:23:25.163068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:25.163784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:23:25.163784 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:25.163784 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:23:25.163784 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:23:25.163784 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:25.163784 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:23:25.163784 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:25.163784 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:23:25.165013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:25.165013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:23:25.165013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:23:25.165013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:23:25.165013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:25.165013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:23:25.165013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:25.165013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:23:25.165013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:25.165013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:23:25.165013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:25.165013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:23:25.165013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:25.165013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:23:25.165013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:23:25.165807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:25.165807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:23:25.165807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:25.165807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:23:25.165807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:23:25.165807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:25.165807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:23:25.165807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:25.165807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:23:25.166366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:25.166366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:23:25.166366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:25.166366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:23:25.166366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:23:25.166366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:23:25.166366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:23:25.166366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:23:25.166366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:23:25.166366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:23:25.166366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:25.166366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:25.167018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:25.167018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:23:25.167018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:25.167018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:25.167018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:25.167018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:23:25.167018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:25.167018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:23:25.167018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:23:25.167018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:25.167018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:23:25.167018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:25.167018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:23:25.167018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:23:25.167655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:25.167655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:23:25.167655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:25.167655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:23:25.168083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:25.168083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:23:25.168083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:23:25.168083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:23:25.168083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:23:25.168083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:23:25.168083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:25.168083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:23:25.168083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:23:25.168083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:23:25.168083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:23:25.168083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:23:25.168083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:25.168083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:25.168083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:23:25.168083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:23:25.168805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:23:25.168805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:23:25.168805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:23:25.168805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:23:25.168805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:23:25.168805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:25.168805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:25.168805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:25.168805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:23:25.168805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:25.168805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:25.168805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:25.168805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:23:25.168805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:25.168805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:23:25.168805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:23:25.168805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:25.168805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:23:25.169607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:25.169607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:23:25.169607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:23:25.169607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:25.169607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:23:25.169607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:25.169607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:23:25.185069 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:23:25.185478 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:23:25.185667 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:23:25.186352 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:23:25.186352 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:23:25.186474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:25.186967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:23:25.186967 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:25.186967 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:23:25.186967 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:23:25.186967 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:25.186967 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:23:25.186967 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:25.186967 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:23:25.188187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:25.188187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:23:25.188187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:23:25.188187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:23:25.188187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:25.188187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:23:25.188187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:25.188187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:23:25.188187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:25.188187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:23:25.188187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:25.188187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:23:25.188187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:25.188187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:23:25.188187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:23:25.188866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:25.188866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:23:25.188866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:25.188866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:23:25.188866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:23:25.188866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:25.188866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:23:25.188866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:25.188866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:23:25.189779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:25.189779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:23:25.189779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:25.189779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:23:25.189779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:23:25.189779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:23:25.189779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:23:25.189779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:23:25.189779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:23:25.189779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:25.189779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:25.190472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:25.190472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:23:25.190472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:25.190472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:25.190472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:25.190472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:23:25.190472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:25.190472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:23:25.190472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:23:25.190472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:25.190472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:23:25.190472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:25.190472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:23:25.190472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:23:25.191090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:25.191090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:23:25.191090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:25.191090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:23:25.191637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:25.191637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:23:25.191637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:23:25.191637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:23:25.191637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:23:25.191637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:23:25.191637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:25.191637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:23:25.191637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:23:25.191637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:23:25.191637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:23:25.191637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:23:25.191637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:25.191637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:25.191637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:23:25.191637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:23:25.191637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:23:25.192444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:23:25.192444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:23:25.192444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:23:25.192444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:25.192444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:25.192444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:25.192444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:23:25.192444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:25.192444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:25.192444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:25.192444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:23:25.192444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:25.192444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:23:25.192444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:23:25.192444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:25.192444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:23:25.192444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:25.192444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:23:25.192444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:23:25.193243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:25.193243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:23:25.193243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:25.193243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:23:25.193682 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:23:25.193958 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:23:25.194058 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:23:25.194199 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:23:25.194451 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:23:25.194610 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:23:25.194721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:25.195153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:23:25.195153 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:25.195153 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:23:25.195153 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:23:25.195153 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:25.195153 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:23:25.195153 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:25.195153 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:23:25.195949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:25.195949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:23:25.195949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:25.195949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:23:25.195949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:23:25.195949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:23:25.195949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:23:25.195949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:25.195949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:25.195949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:25.195949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:23:25.195949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:25.196313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:25.196313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:25.196313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:23:25.196313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:25.196313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:23:25.196313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:23:25.196313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:25.196313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:23:25.196313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:25.196313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:23:25.196313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:23:25.196313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:25.196313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:23:25.196680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:25.196680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:23:25.196880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:25.196880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:23:25.196880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:23:25.196880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:23:25.196880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:23:25.196880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:23:25.196880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:25.196880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:23:25.196880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:23:25.196880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:23:25.196880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:23:25.196880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:23:25.196880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:25.196880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:25.196880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:23:25.196880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:23:25.197427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:23:25.197427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:23:25.197427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:25.197427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:25.197427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:25.197427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:23:25.197427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:25.197427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:25.197427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:25.197427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:23:25.197427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:25.197427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:23:25.197427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:23:25.197427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:25.197427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:23:25.197427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:25.197427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:23:25.197427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:23:25.197427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:25.197946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:23:25.197946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:25.197946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:23:29.928499 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:23:29.929202 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:23:29.929388 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:23:29.930655 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:23:29.958445 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:23:29.958654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:29.959250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:23:29.959250 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:29.959250 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:23:29.959250 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:23:29.959250 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:29.959250 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:23:29.959250 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:29.959250 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:23:29.960078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.030 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:29.960078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:23:29.960078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:29.960078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:23:29.960078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:23:29.960078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:23:29.960078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:23:29.960078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:29.960078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:29.960078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:29.960078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:23:29.960078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:29.960910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:29.960910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:29.960910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:23:29.960910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:29.960910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:23:29.960910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:23:29.960910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:29.960910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:23:29.960910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:29.960910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:23:29.960910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:23:29.960910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:29.960910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:23:29.961496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:29.961496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:23:29.961496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:29.961496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:23:29.961496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:23:29.961496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:23:29.961496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:23:29.961496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:23:29.961496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:29.961496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:23:29.961496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:23:29.961496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:23:29.961496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:23:29.961496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:23:29.961496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:29.962029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:29.962029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:23:29.962029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:23:29.962029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:23:29.962029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:23:29.962029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:29.962029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:29.962029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:29.962029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:23:29.962029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:29.962029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:29.962029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:29.962029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:23:29.962029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:29.962029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:23:29.962029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:23:29.962029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:29.962029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:23:29.962029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:29.962564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:23:29.962564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:23:29.962564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:29.962564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:23:29.962564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:29.962564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:23:30.142930 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:23:30.143314 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:23:30.143554 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:23:30.144740 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:23:30.145145 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:23:30.145388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:30.146231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:23:30.146231 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:30.146231 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:23:30.146231 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:23:30.146231 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:30.146231 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:23:30.146231 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:30.146231 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:23:30.147250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:30.147250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:23:30.147250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:30.147250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:23:30.147250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:23:30.147250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:23:30.147250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:23:30.147250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:30.147250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:30.147250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:30.147250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:23:30.147250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:30.147846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:30.147846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:30.147846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:23:30.147846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:30.147846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:23:30.147846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:23:30.147846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:30.147846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:23:30.147846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:30.147846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:23:30.147846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:23:30.147846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:30.147846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:23:30.148441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:30.148441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:23:30.148441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:30.148441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:23:30.148441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:23:30.148441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:23:30.148441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:23:30.148441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:23:30.148441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:30.148441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:23:30.148441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:23:30.148441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:23:30.148441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:23:30.148441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:23:30.148441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:30.149289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:30.149289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:23:30.149289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:23:30.149289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:23:30.149289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:23:30.149289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:30.149289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:30.149289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:30.149289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:23:30.149289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:30.149289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:30.149289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:30.149289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:23:30.149289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:30.149289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:23:30.149289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:23:30.149289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:30.149289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:23:30.149289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:30.150278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:23:30.150278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:23:30.150278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:30.150278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:23:30.150278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:30.150278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:23:30.161214 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:23:30.161724 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:23:30.161916 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:23:30.162110 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:23:30.162921 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:23:30.163387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:30.164266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:23:30.164266 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:30.164266 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:23:30.164266 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:23:30.164266 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:30.164266 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:23:30.164266 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:30.164266 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:23:30.165632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:30.165632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:23:30.165632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:23:30.165632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:23:30.165632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:30.165632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:23:30.165632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:30.165632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:23:30.165632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:30.165632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:23:30.165632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:30.165632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:23:30.165632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:30.165632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:23:30.165632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:23:30.166100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:30.166100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:23:30.166100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:30.166100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:23:30.166100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:23:30.166100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:30.166100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:23:30.166100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:30.166100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:23:30.167090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:30.167090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:23:30.167090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:30.167090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:23:30.167090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:23:30.167090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:23:30.167090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:23:30.167090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:23:30.167090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:23:30.167090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:23:30.167090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:30.167090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:30.167418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:30.167418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:23:30.167418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:30.167418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:30.167418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:30.167418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:23:30.167418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:30.167418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:23:30.167418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:23:30.167418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:30.167418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:23:30.167418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:30.167418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:23:30.167418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:23:30.167794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:30.167794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:23:30.167794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:30.167794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:23:30.168870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:30.168870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:23:30.168870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:23:30.168870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:23:30.168870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:23:30.168870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:23:30.168870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:30.168870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:23:30.168870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:23:30.168870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:23:30.168870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:23:30.168870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:23:30.168870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:30.168870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:30.168870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:23:30.168870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:23:30.169288 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:23:30.169288 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:23:30.169288 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:23:30.169288 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:23:30.169288 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:23:30.169288 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:30.169288 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:30.169288 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:30.169288 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:23:30.169288 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:30.169288 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:30.169288 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:30.169288 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:23:30.169288 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:30.169288 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:23:30.169288 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:23:30.169288 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:30.169288 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:23:30.169772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:30.169772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:23:30.169772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:23:30.169772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:30.169772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:23:30.169772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:30.169772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:23:30.185626 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:23:30.186025 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:23:30.186211 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:23:30.187550 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:23:30.187550 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:23:30.187550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:30.188698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:23:30.188698 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:30.188698 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:23:30.188698 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:23:30.188698 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:30.188698 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:23:30.188698 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:30.188698 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:23:30.189359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:30.189359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:23:30.189359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:23:30.189359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:23:30.189359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:30.189359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:23:30.189359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:30.189359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:23:30.189359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:30.189359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:23:30.189359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:30.189359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:23:30.189359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:30.189359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:23:30.189359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:23:30.190076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:30.190076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:23:30.190076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:30.190076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:23:30.190076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:23:30.190076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:30.190076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:23:30.190076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:30.190076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:23:30.190853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:30.190853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:23:30.190853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:30.190853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:23:30.190853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:23:30.190853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:23:30.190853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:23:30.190853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:23:30.190853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:23:30.190853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:30.190853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:30.191438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:30.191438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:23:30.191438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:30.191438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:30.191438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:30.191438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:23:30.191438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:30.191438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:23:30.191438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:23:30.191438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:30.191438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:23:30.191438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:30.191438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:23:30.191438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:23:30.192060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:30.192060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:23:30.192060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:30.192060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:23:30.192658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:30.192658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:23:30.192658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:23:30.192658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:23:30.192658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:23:30.192658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:23:30.192658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:30.192658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:23:30.192658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:23:30.192658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:23:30.192658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:23:30.192658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:23:30.192658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:30.192658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:30.192658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:23:30.192658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:23:30.192658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:23:30.193539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:23:30.193539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:23:30.193539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:23:30.193539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:30.193539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:30.193539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:30.193539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:23:30.193539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:30.193539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:30.193539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:30.193539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:23:30.193539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:30.193539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:23:30.193539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:23:30.193539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:30.193539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:23:30.193539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:30.193539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:23:30.193539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:23:30.194169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:30.194169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:23:30.194169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:30.194169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:23:30.194169 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:23:30.194326 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:23:30.194497 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:23:30.194605 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:23:30.195078 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:23:30.195434 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:23:30.195684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:30.196299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:23:30.196299 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:30.196299 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:23:30.196299 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:23:30.196299 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:30.196299 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:23:30.196299 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:30.196299 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:23:30.197144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:30.197144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:23:30.197144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:30.197144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:23:30.197144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:23:30.197144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:23:30.197144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:23:30.197144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:30.197144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:30.197144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:30.197144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:23:30.197144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:30.197545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:30.197545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:30.197545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:23:30.197545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:30.197545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:23:30.197545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:23:30.197545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:30.197545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:23:30.197545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:30.197545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:23:30.197545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:23:30.197545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:30.197545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:23:30.197890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:30.197890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:23:30.198078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:30.198078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:23:30.198078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:23:30.198078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:23:30.198078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:23:30.198078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:23:30.198078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:30.198078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:23:30.198078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:23:30.198078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:23:30.198078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:23:30.198078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:23:30.198078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:30.198078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:30.198078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:23:30.198078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:23:30.198525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:23:30.198525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:23:30.198525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:30.198525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:30.198525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:30.198525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:23:30.198525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:30.198525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:30.198525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:30.198525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:23:30.198525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:30.198525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:23:30.198525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:23:30.198525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:30.198525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:23:30.198525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:30.198525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:23:30.198525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:23:30.198525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:30.199018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:23:30.199018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:30.199018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:23:34.929374 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:23:34.930139 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:23:34.930139 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:23:34.931321 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:23:34.961877 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:23:34.962516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:34.963150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:23:34.963150 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:34.963150 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:23:34.963150 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:23:34.963150 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:34.963150 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:23:34.963150 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:34.963150 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:23:34.964521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:34.964521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:23:34.964521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:34.964521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:23:34.964521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:23:34.964521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:23:34.964521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:23:34.964521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:34.964521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:34.964521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:34.964521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:23:34.964521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:34.965573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:34.965573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:34.965573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:23:34.965573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:34.965573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:23:34.965573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:23:34.965573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:34.965573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:23:34.965573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:34.965573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:23:34.965573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:23:34.965573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:34.965573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:23:34.966350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:34.966350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:23:34.966350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:34.966350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:23:34.966350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:23:34.966350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:23:34.966350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:23:34.966350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:23:34.966350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:34.966350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:23:34.966350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:23:34.966350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:23:34.966350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:23:34.966350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:23:34.966350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:34.967072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:34.967072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:23:34.967072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:23:34.967072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:23:34.967072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:23:34.967072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:34.967072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:34.967072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:34.967072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:23:34.967072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:34.967072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:34.967072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:34.967072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:23:34.967072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:34.967072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:23:34.967072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:23:34.967072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:34.967072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:23:34.967072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:34.967825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:23:34.967825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:23:34.967825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:34.967825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:23:34.967825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:34.967825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:23:35.143674 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:23:35.144082 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:23:35.144192 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:23:35.144980 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:23:35.145169 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:23:35.145289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:35.145942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:23:35.145942 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:35.145942 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:23:35.145942 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:23:35.145942 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:35.145942 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:23:35.145942 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:35.145942 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:23:35.146957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:35.146957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:23:35.146957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:35.146957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:23:35.146957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:23:35.146957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:23:35.146957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:23:35.146957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:35.146957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:35.146957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:35.146957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:23:35.146957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:35.148933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:35.148933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:35.148933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:23:35.148933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:35.148933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:23:35.148933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:23:35.148933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:35.148933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:23:35.148933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:35.148933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:23:35.148933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:23:35.148933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:35.148933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:23:35.149571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:35.149571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:23:35.149571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:35.149571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:23:35.149571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:23:35.149571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:23:35.149571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:23:35.149571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:23:35.149571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:35.149571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:23:35.149571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:23:35.149571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:23:35.149571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:23:35.149571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:23:35.149571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:35.150307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:35.150307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:23:35.150307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:23:35.150307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:23:35.150307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:23:35.150307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:35.150307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:35.150307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:35.150307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:23:35.150307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:35.150307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:35.150307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:35.150307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:23:35.150307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:35.150307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:23:35.150307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:23:35.150307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:35.150307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:23:35.150307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:35.151147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:23:35.151147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:23:35.151147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:35.151147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:23:35.151147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:35.151147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:23:35.161635 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:23:35.161860 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:23:35.161974 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:23:35.162060 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:23:35.162532 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:23:35.162742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:35.163353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:23:35.163353 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:35.163353 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:23:35.163353 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:23:35.163353 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:35.163353 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:23:35.163353 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:35.163353 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:23:35.163935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:35.163935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:23:35.163935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:23:35.163935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:23:35.163935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:35.163935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:23:35.163935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:35.163935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:23:35.163935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:35.163935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:23:35.163935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:35.163935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:23:35.163935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:35.163935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:23:35.163935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:23:35.164421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:35.164421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:23:35.164421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:35.164421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:23:35.164421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:23:35.164421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:35.164421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:23:35.164421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:35.164421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:23:35.164704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:35.164704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:23:35.164704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:35.164704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:23:35.164704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:23:35.164704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:23:35.164704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:23:35.164704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:23:35.164704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:23:35.164704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:23:35.164704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:35.164704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:35.165066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:35.165066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:23:35.165066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:35.165066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:35.165066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:35.165066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:23:35.165066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:35.165066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:23:35.165066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:23:35.165066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:35.165066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:23:35.165066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:35.165066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:23:35.165066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:23:35.165455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:35.165455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:23:35.165455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:35.165455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:23:35.165455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:35.165455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:23:35.165455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:23:35.165455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:23:35.165455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:23:35.165455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:23:35.165455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:35.165455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:23:35.165455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:23:35.165455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:23:35.165840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:23:35.165840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:23:35.165840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:35.165840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:35.165840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:23:35.165840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:23:35.165840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:23:35.165840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:23:35.165840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:23:35.165840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:23:35.165840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:23:35.165840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:35.165840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:35.165840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:35.165840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:23:35.165840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:35.165840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:35.165840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:35.165840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:23:35.166433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:35.166433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:23:35.166433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:23:35.166433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:35.166433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:23:35.166433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:35.166433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:23:35.166433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:23:35.166433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:35.166433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:23:35.166433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:35.166433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:23:35.186140 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:23:35.186421 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:23:35.186555 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:23:35.186687 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:23:35.187033 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:23:35.187265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:35.187840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:23:35.187840 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:35.187840 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:23:35.187840 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:23:35.187840 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:35.187840 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:23:35.187840 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:35.187840 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:23:35.189052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:35.189052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:23:35.189052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:23:35.189052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:23:35.189052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:35.189052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:23:35.189052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:35.189052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:23:35.189052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:35.189052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:23:35.189052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:35.189052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:23:35.189052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:35.189052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:23:35.189052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:23:35.189758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:35.189758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:23:35.189758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:35.189758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:23:35.189758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:23:35.189758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:35.189758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:23:35.189758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:35.189758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:23:35.190472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:35.190472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:23:35.190472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:35.190472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:23:35.190472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:23:35.190472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:23:35.190472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:23:35.190472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:23:35.190472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:23:35.190472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:35.190472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:35.191021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:35.191021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:23:35.191021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:35.191021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:35.191021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:35.191021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:23:35.191021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:35.191021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:23:35.191021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:23:35.191021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:35.191021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:23:35.191021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:35.191021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:23:35.191021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:23:35.191658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:35.191658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:23:35.191658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:35.191658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:23:35.192159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:35.192159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:23:35.192159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:23:35.192159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:23:35.192159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:23:35.192159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:23:35.192159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:35.192159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:23:35.192159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:23:35.192159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:23:35.192159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:23:35.192159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:23:35.192159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:35.192159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:35.192159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:23:35.192159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:23:35.192159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:23:35.192970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:23:35.192970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:23:35.192970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:23:35.192970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:35.192970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:35.192970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:35.192970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:23:35.192970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:35.192970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:35.192970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:35.192970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:23:35.192970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:35.192970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:23:35.192970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:23:35.192970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:35.192970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:23:35.192970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:35.192970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:23:35.192970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:23:35.193488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:35.193488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:23:35.193488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:35.193488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:23:35.194593 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:23:35.195030 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:23:35.195207 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:23:35.195481 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:23:35.195945 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:23:35.196292 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:23:35.196565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:35.197320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:23:35.197320 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:35.197320 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:23:35.197320 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:23:35.197320 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:35.197320 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:23:35.197320 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:35.197320 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:23:35.198148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:35.198148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:23:35.198148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:35.198148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:23:35.198148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:23:35.198148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:23:35.198148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:23:35.198148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:35.198148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:35.198148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:35.198148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:23:35.198148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:35.198525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:35.198525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:35.198525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:23:35.198525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:35.198525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:23:35.198525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:23:35.198525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:35.198525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:23:35.198525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:35.198525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:23:35.198525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:23:35.198525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:35.198525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:23:35.198843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:35.198843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:23:35.199085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:35.199085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:23:35.199085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:23:35.199085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:23:35.199085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:23:35.199085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:23:35.199085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:35.199085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:23:35.199085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:23:35.199085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:23:35.199085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:23:35.199085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:23:35.199085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:35.199085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:35.199085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:23:35.199085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:23:35.199547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:23:35.199547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:23:35.199547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:35.199547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:35.199547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:35.199547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:23:35.199547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:35.199547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:35.199547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:35.199547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:23:35.199547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:35.199547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:23:35.199547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:23:35.199547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:35.199547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:23:35.199547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:35.199547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:23:35.199547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:23:35.199547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:35.200200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:23:35.200200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:35.200200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:23:39.930072 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:23:39.931154 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:23:39.931154 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:23:39.932457 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:23:39.969453 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:23:39.969829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:39.970658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:23:39.970658 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:39.970658 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:23:39.970658 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:23:39.970658 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:39.970658 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:23:39.970658 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:39.970658 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:23:39.971621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:39.971621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:23:39.971621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:39.971621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:23:39.971621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:23:39.971621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:23:39.971621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:23:39.971621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:39.971621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:39.971621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:39.971621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:23:39.971621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:39.972910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:39.972910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:39.972910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:23:39.972910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:39.972910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:23:39.972910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:23:39.972910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:39.972910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:23:39.972910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:39.972910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:23:39.972910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:23:39.972910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:39.972910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:23:39.973572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:39.973572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:23:39.973572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:39.973572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:23:39.973572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:23:39.973572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:23:39.973572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:23:39.973572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:23:39.973572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:39.973572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:23:39.973572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:23:39.973572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:23:39.973572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:23:39.973572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:23:39.973572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:39.975004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:39.975004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:23:39.975004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:23:39.975004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:23:39.975004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:23:39.975004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:39.975004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:39.975004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:39.975004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:23:39.975004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:39.975004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:39.975004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:39.975004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:23:39.975004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:39.975004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:23:39.975004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:23:39.975004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:39.975004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:23:39.975004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:39.975992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:23:39.975992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:23:39.975992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:39.975992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:23:39.975992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:39.975992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:23:40.144288 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:23:40.145005 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:23:40.145253 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:23:40.146612 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:23:40.146942 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:23:40.147193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:40.148590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:23:40.148590 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:40.148590 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:23:40.148590 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:23:40.148590 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:40.148590 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:23:40.148590 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:40.148590 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:23:40.150862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:40.150862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:23:40.150862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:40.150862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:23:40.150862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:23:40.150862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:23:40.150862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:23:40.150862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:40.150862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:40.150862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:40.150862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:23:40.150862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:40.151648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:40.151648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:40.151648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:23:40.151648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:40.151648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:23:40.151648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:23:40.151648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:40.151648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:23:40.151648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:40.151648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:23:40.151648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:23:40.151648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:40.151648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:23:40.152331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:40.152331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:23:40.152989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:40.152989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:23:40.152989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:23:40.152989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:23:40.152989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:23:40.152989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:23:40.152989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:40.152989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:23:40.152989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:23:40.152989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:23:40.152989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:23:40.152989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:23:40.152989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:40.152989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:40.152989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:23:40.152989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:23:40.155303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:23:40.155303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:23:40.155303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:40.155303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:40.155303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:40.155303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:23:40.155303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:40.155303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:40.155303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:40.155303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:23:40.155303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:40.155303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:23:40.155303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:23:40.155303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:40.155303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:23:40.155303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:40.155303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:23:40.155303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:23:40.155303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:40.156271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:23:40.156271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:40.156271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:23:40.162007 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:23:40.162495 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:23:40.162765 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:23:40.162972 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:23:40.163476 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:23:40.163736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:40.164866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:23:40.164866 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:40.164866 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:23:40.164866 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:23:40.164866 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:40.164866 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:23:40.164866 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:40.164866 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:23:40.166377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:40.166377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:23:40.166377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:23:40.166377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:23:40.166377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:40.166377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:23:40.166377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:40.166377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:23:40.166377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:40.166377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:23:40.166377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:40.166377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:23:40.166377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:40.166377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:23:40.166377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:23:40.166850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:40.166850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:23:40.166850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:40.166850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:23:40.166850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:23:40.166850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:40.166850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:23:40.166850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:40.166850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:23:40.167810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:40.167810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:23:40.167810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:40.167810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:23:40.167810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:23:40.167810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:23:40.167810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:23:40.167810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:23:40.167810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:23:40.167810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:23:40.167810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:40.167810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:40.168234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:40.168234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:23:40.168234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:40.168234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:40.168234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:40.168234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:23:40.168234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:40.168234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:23:40.168234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:23:40.168234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:40.168234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:23:40.168234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:40.168234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:23:40.168234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:23:40.168620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:40.168620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:23:40.168620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:40.168620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:23:40.169079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:40.169079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:23:40.169079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:23:40.169079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:23:40.169079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:23:40.169079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:23:40.169079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:40.169079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:23:40.169079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:23:40.169079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:23:40.169079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:23:40.169079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:23:40.169079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:40.169079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:40.169079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:23:40.169079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:23:40.170230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:23:40.170230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:23:40.170230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:23:40.170230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:23:40.170230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:23:40.170230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:40.170230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:40.170230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:40.170230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:23:40.170230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:40.170230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:40.170230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:40.170230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:23:40.170230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:40.170230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:23:40.170230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:23:40.170230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:40.170230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:23:40.170787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:40.170787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:23:40.170787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:23:40.170787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:40.170787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:23:40.170787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:40.170787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:23:40.186681 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:23:40.187023 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:23:40.187204 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:23:40.187399 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:23:40.187955 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:23:40.188286 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:40.189067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:23:40.189067 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:40.189067 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:23:40.189067 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:23:40.189067 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:40.189067 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:23:40.189067 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:40.189067 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:23:40.190269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:40.190269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:23:40.190269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:23:40.190269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:23:40.190269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:40.190269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:23:40.190269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:40.190269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:23:40.190269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:40.190269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:23:40.190269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:40.190269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:23:40.190269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:40.190269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:23:40.190269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:23:40.190785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:40.190785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:23:40.190785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:40.190785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:23:40.190785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:23:40.190785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:40.190785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:23:40.190785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:40.190785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:23:40.191772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:40.191772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:23:40.191772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:40.191772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:23:40.191772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:23:40.191772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:23:40.191772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:23:40.191772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:23:40.191772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:23:40.191772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:40.191772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:40.192544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:40.192544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:23:40.192544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:40.192544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:40.192544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:40.192544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:23:40.192544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:40.192544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:23:40.192544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:23:40.192544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:40.192544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:23:40.192544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:40.192544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:23:40.192544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:23:40.193510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:40.193510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:23:40.193510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:40.193510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:23:40.193510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:40.193510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:23:40.193510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:23:40.193510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:23:40.193510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:23:40.193510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:23:40.193510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:40.193510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:23:40.193510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:23:40.193510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:23:40.194831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:23:40.194831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:23:40.194831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:40.194831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:40.194831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:23:40.194831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:23:40.194831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:23:40.194831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:23:40.194831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:23:40.194831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:23:40.194831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:40.194831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:40.194831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:40.194831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:23:40.194831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:40.194831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:40.194831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:40.194831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:23:40.194831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:40.196904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:23:40.196904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:23:40.196904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:40.196904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:23:40.196904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:40.196904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:23:40.196904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:23:40.196904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:40.196904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:23:40.196904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:40.196904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:23:40.196904 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:23:40.196904 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:23:40.198637 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:23:40.198637 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:23:40.198637 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:23:40.198637 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:23:40.198637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:40.198637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:23:40.198637 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:40.198637 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:23:40.198637 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:23:40.199016 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:40.199016 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:23:40.199016 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:40.199016 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:23:40.199016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:40.199016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:23:40.199016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:40.199016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:23:40.199016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:23:40.199016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:23:40.199016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:23:40.199016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:40.199016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:40.199016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:40.199906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:23:40.199906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:40.199906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:40.199906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:40.199906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:23:40.199906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:40.199906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:23:40.199906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:23:40.199906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:40.199906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:23:40.199906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:40.199906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:23:40.199906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:23:40.199906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:40.200922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:23:40.200922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:40.200922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:23:40.200922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:40.200922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:23:40.200922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:23:40.200922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:23:40.200922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:23:40.200922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:23:40.200922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:40.200922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:23:40.200922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:23:40.200922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:23:40.200922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:23:40.200922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:23:40.200922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:40.201521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:40.201521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:23:40.201521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:23:40.201521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:23:40.201521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:23:40.201521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:40.201521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:40.201521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:40.201521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:23:40.201521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:40.201521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:40.201521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:40.201521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:23:40.201521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:40.201521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:23:40.201521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:23:40.201521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:40.201521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:23:40.201521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:40.201991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:23:40.201991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:23:40.201991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:40.201991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:23:40.201991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:40.201991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:23:44.930709 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:23:44.931681 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:23:44.931681 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:23:44.932517 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:23:44.996153 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:23:44.996153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:44.996153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:23:44.996153 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:44.996153 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:23:44.996153 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:23:44.996153 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:44.996153 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:23:44.996153 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:44.996153 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:23:44.996153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:44.996153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:23:44.996153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:44.996496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:23:44.996496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:23:44.996496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:23:44.996496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:23:44.996496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:44.996496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:44.996496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:44.996496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:23:44.996496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:44.996496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:44.996496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:44.996496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:23:44.996496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:44.996496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:23:44.996789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:23:44.996789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:44.996789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:23:44.996789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:44.996789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:23:44.996789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:23:44.996789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:44.996789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:23:44.996789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:44.996789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:23:44.996789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:44.996789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:23:44.997082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:23:44.997082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:23:44.997082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:23:44.997082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:23:44.997082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:44.997082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:23:44.997082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:23:44.997082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:23:44.997082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:23:44.997082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:23:44.997082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:44.997082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:44.997082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:23:44.997082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:23:44.997082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:23:44.997082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:23:44.997082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:44.997082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:44.997082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:44.997496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:23:44.997496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:44.997496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:44.997496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:44.997496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:23:44.997496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:44.997496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:23:44.997496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:23:44.997496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:44.997496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:23:44.997496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:44.997496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:23:44.997496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:23:44.997496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:44.997496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:23:44.997496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:44.997496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:23:45.146895 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:23:45.148162 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:23:45.148162 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:23:45.149020 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:23:45.149330 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:23:45.149614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:45.152162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:23:45.152162 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:45.152162 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:23:45.152162 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:23:45.152162 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:45.152162 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:23:45.152162 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:45.152162 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:23:45.153146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:45.153146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:23:45.153146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:45.153146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:23:45.153146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:23:45.153146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:23:45.153146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:23:45.153146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:45.153146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:45.153146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:45.153146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:23:45.153146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:45.154635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:45.154635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:45.154635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:23:45.154635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:45.154635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:23:45.154635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:23:45.154635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:45.154635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:23:45.154635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:45.154635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:23:45.154635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:23:45.154635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:45.154635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:23:45.156246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:45.156246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:23:45.156246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:45.156246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:23:45.156246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:23:45.156246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:23:45.156246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:23:45.156246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:23:45.156246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:45.156246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:23:45.156246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:23:45.156246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:23:45.156246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:23:45.156246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:23:45.156246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:45.158545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:45.158545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:23:45.158545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:23:45.158545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:23:45.158545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:23:45.158545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:45.158545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:45.158545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:45.158545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:23:45.158545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:45.158545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:45.158545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:45.158545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:23:45.158545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:45.158545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:23:45.158545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:23:45.158545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:45.158545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:23:45.158545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:45.159530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:23:45.159530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:23:45.159530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:45.159530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:23:45.159530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:45.159530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:23:45.162480 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:23:45.162722 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:23:45.162799 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:23:45.162893 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:23:45.163536 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:23:45.163905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:45.165033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:23:45.165033 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:45.165033 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:23:45.165033 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:23:45.165033 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:45.165033 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:23:45.165033 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:45.165033 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:23:45.166287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:45.166287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:23:45.166287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:23:45.166287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:23:45.166287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:45.166287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:23:45.166287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:45.166287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:23:45.166287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:45.166287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:23:45.166287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:45.166287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:23:45.166287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:45.166287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:23:45.166287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:23:45.166834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:45.166834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:23:45.166834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:45.166834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:23:45.166834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:23:45.166834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:45.166834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:23:45.166834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:45.166834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:23:45.167809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:45.167809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:23:45.167809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:45.167809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:23:45.167809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:23:45.167809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:23:45.167809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:23:45.167809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:23:45.167809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:23:45.167809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:23:45.167809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:45.167809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:45.168212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:45.168212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:23:45.168212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:45.168212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:45.168212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:45.168212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:23:45.168212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:45.168212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:23:45.168212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:23:45.168212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:45.168212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:23:45.168212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:45.168212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:23:45.168212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:23:45.168587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:45.168587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:23:45.168587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:45.168587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:23:45.169677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:45.169677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:23:45.169677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:23:45.169677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:23:45.169677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:23:45.169677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:23:45.169677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:45.169677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:23:45.169677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:23:45.169677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:23:45.169677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:23:45.169677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:23:45.169677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:45.169677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:45.169677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:23:45.169677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:23:45.170126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:23:45.170126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:23:45.170126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:23:45.170126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:23:45.170126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:23:45.170126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:45.170126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:45.170126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:45.170126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:23:45.170126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:45.170126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:45.170126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:45.170126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:23:45.170126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:45.170126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:23:45.170126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:23:45.170126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:45.170126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:23:45.170646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:45.170646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:23:45.170646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:23:45.170646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:45.170646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:23:45.170646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:45.170646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:23:45.187069 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:23:45.187446 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:23:45.187695 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:23:45.187903 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:23:45.188445 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:23:45.188872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:45.189681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:23:45.189681 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:45.189681 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:23:45.189681 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:23:45.189681 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:45.189681 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:23:45.189681 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:45.189681 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:23:45.190978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:45.190978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:23:45.190978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:23:45.190978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:23:45.190978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:45.190978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:23:45.190978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:45.190978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:23:45.190978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:45.190978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:23:45.190978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:45.190978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:23:45.190978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:45.190978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:23:45.190978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:23:45.191772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:45.191772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:23:45.191772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:45.191772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:23:45.191772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:23:45.191772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:45.191772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:23:45.191772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:45.191772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:23:45.192715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:45.192715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:23:45.192715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:45.192715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:23:45.192715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:23:45.192715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:23:45.192715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:23:45.192715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:23:45.192715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:23:45.192715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:45.192715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:45.193321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:45.193321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:23:45.193321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:45.193321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:45.193321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:45.193321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:23:45.193321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:45.193321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:23:45.193321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:23:45.193321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:45.193321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:23:45.193321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:45.193321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:23:45.193321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:23:45.194006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:45.194006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:23:45.194006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:45.194006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:23:45.195133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:45.195133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:23:45.195133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:23:45.195133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:23:45.195133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:23:45.195133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:23:45.195133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:45.195133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:23:45.195133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:23:45.195133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:23:45.195133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:23:45.195133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:23:45.195133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:45.195133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:45.195133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:23:45.195133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:23:45.195133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:23:45.195984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:23:45.195984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:23:45.195984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:23:45.195984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:45.195984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:45.195984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:45.195984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:23:45.195984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:45.195984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:45.195984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:45.195984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:23:45.195984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:45.195984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:23:45.195984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:23:45.195984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:45.195984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:23:45.195984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:45.195984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:23:45.195984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:23:45.196892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:45.196892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:23:45.196892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:45.196892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:23:45.196892 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:23:45.196892 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:23:45.196892 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:23:45.196892 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:23:45.197332 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:23:45.197332 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:23:45.197589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:45.199578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:23:45.199578 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:45.199578 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:23:45.199578 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:23:45.199578 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:45.199578 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:23:45.199578 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:45.199578 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:23:45.200344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:45.200344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:23:45.200344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:45.200344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:23:45.200344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:23:45.200344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:23:45.200344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:23:45.200344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:45.200344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:45.200344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:45.200344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:23:45.200344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:45.200737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:45.200737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:45.200737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:23:45.200737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:45.200737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:23:45.200737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:23:45.200737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:45.200737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:23:45.200737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:45.200737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:23:45.200737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:23:45.200737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:45.200737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:23:45.201087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:45.201087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:23:45.201369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:45.201369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:23:45.201369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:23:45.201369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:23:45.201369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:23:45.201369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:23:45.201369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:45.201369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:23:45.201369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:23:45.201369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:23:45.201369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:23:45.201369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:23:45.201369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:45.201369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:45.201369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:23:45.201369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:23:45.201844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:23:45.201844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:23:45.201844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:45.201844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:45.201844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:45.201844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:23:45.201844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:45.201844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:45.201844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:45.201844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:23:45.201844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:45.201844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:23:45.201844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:23:45.201844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:45.201844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:23:45.201844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:45.201844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:23:45.201844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:23:45.201844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:45.202396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:23:45.202396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:45.202396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:23:49.931359 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:23:49.932248 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:23:49.932248 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:23:49.933362 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:23:49.965327 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:23:49.965803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:49.966698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:23:49.966698 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:49.966698 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:23:49.966698 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:23:49.966698 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:49.966698 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:23:49.966698 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:49.966698 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:23:49.967415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:49.967415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:23:49.967415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:49.967415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:23:49.967415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:23:49.967415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:23:49.967415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:23:49.967415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:49.967415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:49.967415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:49.967415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:23:49.967415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:49.967805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:49.967805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:49.967805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:23:49.967805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:49.967805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:23:49.967805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:23:49.967805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:49.967805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:23:49.967805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:49.967805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:23:49.967805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:23:49.967805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:49.967805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:23:49.968193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:49.968193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:23:49.968193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:49.968193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:23:49.968193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:23:49.968193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:23:49.968193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:23:49.968193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:23:49.968193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:49.968193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:23:49.968193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:23:49.968193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:23:49.968193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:23:49.968193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:23:49.968193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:49.968671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:49.968671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:23:49.968671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:23:49.968671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:23:49.968671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:23:49.968671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:49.968671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:49.968671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:49.968671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:23:49.968671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:49.968671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:49.968671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:49.968671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:23:49.968671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:49.968671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:23:49.968671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:23:49.968671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:49.968671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:23:49.968671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:49.969191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:23:49.969191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:23:49.969191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:49.969191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:23:49.969191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:49.969191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:23:50.148018 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:23:50.148616 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:23:50.148780 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:23:50.149824 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:23:50.150219 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:23:50.150423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:50.151275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:23:50.151275 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:50.151275 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:23:50.151275 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:23:50.151275 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:50.151275 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:23:50.151275 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:50.151275 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:23:50.152860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:50.152860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:23:50.152860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:50.152860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:23:50.152860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:23:50.152860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:23:50.152860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:23:50.152860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:50.152860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:50.152860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:50.152860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:23:50.152860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:50.153513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:50.153513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:50.153513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:23:50.153513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:50.153513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:23:50.153513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:23:50.153513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:50.153513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:23:50.153513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:50.153513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:23:50.153513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:23:50.153513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:50.153513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:23:50.154065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:50.154065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:23:50.154711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:50.154711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:23:50.154711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:23:50.154711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:23:50.154711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:23:50.154711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:23:50.154711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:50.154711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:23:50.154711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:23:50.154711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:23:50.154711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:23:50.154711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:23:50.154711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:50.154711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:50.154711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:23:50.154711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:23:50.156031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:23:50.156031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:23:50.156031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:50.156031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:50.156031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:50.156031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:23:50.156031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:50.156031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:50.156031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:50.156031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:23:50.156031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:50.156031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:23:50.156031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:23:50.156031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:50.156031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:23:50.156031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:50.156031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:23:50.156031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:23:50.156031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:50.156873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:23:50.156873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:50.156873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:23:50.162808 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:23:50.163086 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:23:50.163199 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:23:50.163315 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:23:50.163830 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:23:50.164156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:50.164889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:23:50.164889 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:50.164889 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:23:50.164889 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:23:50.164889 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:50.164889 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:23:50.164889 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:50.164889 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:23:50.165826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:50.165826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:23:50.165826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:23:50.165826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:23:50.165826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:50.165826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:23:50.165826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:50.165826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:23:50.165826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:50.165826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:23:50.165826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:50.165826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:23:50.165826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:50.165826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:23:50.165826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:23:50.166339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:50.166339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:23:50.166339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:50.166339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:23:50.166339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:23:50.166339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:50.166339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:23:50.166339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:50.166339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:23:50.168034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:50.168034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:23:50.168034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:50.168034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:23:50.168034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:23:50.168034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:23:50.168034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:23:50.168034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:23:50.168034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:23:50.168034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:23:50.168034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:50.168034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:50.168676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:50.168676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:23:50.168676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:50.168676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:50.168676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:50.168676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:23:50.168676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:50.168676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:23:50.168676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:23:50.168676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:50.168676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:23:50.168676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:50.168676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:23:50.168676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:23:50.169280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:50.169280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:23:50.169280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:50.169280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:23:50.170182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:50.170182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:23:50.170182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:23:50.170182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:23:50.170182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:23:50.170182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:23:50.170182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:50.170182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:23:50.170182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:23:50.170182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:23:50.170182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:23:50.170182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:23:50.170182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:50.170182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:50.170182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:23:50.170182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:23:50.170930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:23:50.170930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:23:50.170930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:23:50.170930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:23:50.170930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:23:50.170930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:50.170930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:50.170930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:50.170930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:23:50.170930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:50.170930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:50.170930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:50.170930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:23:50.170930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:50.170930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:23:50.170930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:23:50.170930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:50.170930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:23:50.171772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:50.171772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:23:50.171772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:23:50.171772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:50.171772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:23:50.171772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:50.171772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:23:50.187581 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:23:50.187823 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:23:50.187908 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:23:50.188365 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:23:50.188622 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:23:50.189005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:50.189771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:23:50.189771 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:50.189771 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:23:50.189771 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:23:50.189771 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:50.189771 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:23:50.189771 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:50.189771 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:23:50.190548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:50.190548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:23:50.190548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:23:50.190548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:23:50.190548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:50.190548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:23:50.190548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:50.190548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:23:50.190548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:50.190548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:23:50.190548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:50.190548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:23:50.190548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:50.190548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:23:50.190548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:23:50.191219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:50.191219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:23:50.191219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:50.191219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:23:50.191219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:23:50.191219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:50.191219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:23:50.191219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:50.191219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:23:50.191626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:50.191626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:23:50.191626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:50.191626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:23:50.191626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:23:50.191626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:23:50.191626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:23:50.191626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:23:50.191626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:23:50.191626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:50.191626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:50.192142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:50.192142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:23:50.192142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:50.192142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:50.192142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:50.192142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:23:50.192142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:50.192142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:23:50.192142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:23:50.192142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:50.192142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:23:50.192142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:50.192142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:23:50.192142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:23:50.192784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:50.192784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:23:50.192784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:50.192784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:23:50.192784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:50.192784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:23:50.192784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:23:50.192784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:23:50.192784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:23:50.192784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:23:50.192784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:50.192784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:23:50.192784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:23:50.192784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:23:50.193461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:23:50.193461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:23:50.193461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:50.193461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:50.193461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:23:50.193461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:23:50.193461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:23:50.193461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:23:50.193461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:23:50.193461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:23:50.193461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:50.193461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:50.193461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:50.193461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:23:50.193461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:50.193461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:50.193461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:50.193461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:23:50.193461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:50.194368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:23:50.194368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:23:50.194368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:50.194368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:23:50.194368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:50.194368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:23:50.194368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:23:50.194368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:50.194368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:23:50.194368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:50.194368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:23:50.196106 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:23:50.196378 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:23:50.196503 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:23:50.196651 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:23:50.197123 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:23:50.197496 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:23:50.197720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:50.198419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:23:50.198419 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:50.198419 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:23:50.198419 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:23:50.198419 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:50.198419 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:23:50.198419 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:50.198419 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:23:50.199239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:50.199239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:23:50.199239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:50.199239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:23:50.199239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:23:50.199239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:23:50.199239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:23:50.199239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:50.199239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:50.199239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:50.199239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:23:50.199239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:50.199632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:50.199632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:50.199632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:23:50.199632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:50.199632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:23:50.199632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:23:50.199632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:50.199632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:23:50.199632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:50.199632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:23:50.199632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:23:50.199632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:50.199632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:23:50.199976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:50.199976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:23:50.200179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:50.200179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:23:50.200179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:23:50.200179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:23:50.200179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:23:50.200179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:23:50.200179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:50.200179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:23:50.200179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:23:50.200179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:23:50.200179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:23:50.200179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:23:50.200179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:50.200179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:50.200179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:23:50.200179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:23:50.200654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:23:50.200654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:23:50.200654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:50.200654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:50.200654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:50.200654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:23:50.200654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:50.200654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:50.200654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:50.200654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:23:50.200654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:50.200654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:23:50.200654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:23:50.200654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:50.200654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:23:50.200654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:50.200654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:23:50.200654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:23:50.200654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:50.201160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:23:50.201160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:50.201160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:23:54.932124 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:23:54.933266 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:23:54.933266 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:23:54.933954 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:23:54.970015 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:23:54.970413 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:54.971325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:23:54.971325 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:54.971325 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:23:54.971325 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:23:54.971325 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:54.971325 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:23:54.971325 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:54.971325 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:23:54.972477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:54.972477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:23:54.972477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:54.972477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:23:54.972477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:23:54.972477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:23:54.972477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:23:54.972477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:54.972477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:54.972477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:54.972477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:23:54.972477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:54.973145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:54.973145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:54.973145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:23:54.973145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:54.973145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:23:54.973145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:23:54.973145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:54.973145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:23:54.973145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:54.973145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:23:54.973145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:23:54.973145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:54.973145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:23:54.973857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:54.973857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:23:54.973980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:54.973980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:23:54.973980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:23:54.973980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:23:54.973980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:23:54.973980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:23:54.973980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:54.973980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:23:54.973980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:23:54.973980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:23:54.973980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:23:54.973980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:23:54.973980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:54.973980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:54.973980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:23:54.973980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:23:54.974544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:23:54.974544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:23:54.974544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:54.974544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:54.974544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:54.974544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:23:54.974544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:54.974544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:54.974544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:54.974544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:23:54.974544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:54.974544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:23:54.974544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:23:54.974544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:54.974544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:23:54.974544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:54.974544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:23:54.974544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:23:54.974544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:54.975110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:23:54.975110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:54.975110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:23:55.148767 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:23:55.149054 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:23:55.149267 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:23:55.150065 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:23:55.150385 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:23:55.150674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:55.152689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:23:55.152689 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:55.152689 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:23:55.152689 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:23:55.152689 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:55.152689 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:23:55.152689 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:55.152689 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:23:55.152689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:55.152689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:23:55.152689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:55.152689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:23:55.152689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:23:55.152689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:23:55.152689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:23:55.153182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:55.153182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:55.153182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:55.153182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:23:55.153182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:55.153182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:55.153182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:55.153182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:23:55.153182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:55.153182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:23:55.153182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:23:55.153182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:55.153182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:23:55.153182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:55.153495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:23:55.153495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:23:55.153495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:55.153495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:23:55.153495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:55.153495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:23:55.153495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:55.153495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:23:55.153495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:23:55.153495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:23:55.153495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:23:55.153495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:23:55.153495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:55.153495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:23:55.153778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:23:55.153778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:23:55.153778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:23:55.153778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:23:55.153778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:55.153778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:55.153778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:23:55.153778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:23:55.153778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:23:55.153778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:23:55.153778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:55.153778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:55.153778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:55.153778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:23:55.153778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:55.153778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:55.153778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:55.153778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:23:55.153778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:55.153778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:23:55.154154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:23:55.154154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:55.154154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:23:55.154154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:55.154154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:23:55.154154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:23:55.154154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:55.154154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:23:55.154154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:55.154154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:23:55.163639 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:23:55.163639 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:23:55.163639 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:23:55.163639 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:23:55.164098 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:23:55.164098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:55.164783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:23:55.164783 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:55.164783 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:23:55.164783 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:23:55.164783 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:55.164783 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:23:55.164783 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:55.164783 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:23:55.166108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:55.166108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:23:55.166108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:23:55.166108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:23:55.166108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:55.166108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:23:55.166108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:55.166108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:23:55.166108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:55.166108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:23:55.166108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:55.166108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:23:55.166108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:55.166108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:23:55.166108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:23:55.166971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:55.166971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:23:55.166971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:55.166971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:23:55.166971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:23:55.166971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:55.166971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:23:55.166971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:55.166971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:23:55.167739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:55.167739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:23:55.167739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:55.167739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:23:55.167739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:23:55.167739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:23:55.167739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:23:55.167739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:23:55.167739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:23:55.167739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:23:55.167739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:55.167739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:55.168364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:55.168364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:23:55.168364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:55.168364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:55.168364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:55.168364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:23:55.168364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:55.168364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:23:55.168364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:23:55.168364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:55.168364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:23:55.168364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:55.168364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:23:55.168364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:23:55.169037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:55.169037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:23:55.169037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:55.169037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:23:55.169574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:55.169574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:23:55.169574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:23:55.169574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:23:55.169574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:23:55.169574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:23:55.169574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:55.169574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:23:55.169574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:23:55.169574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:23:55.169574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:23:55.169574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:23:55.169574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:55.169574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:55.169574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:23:55.169574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:23:55.170312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:23:55.170312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:23:55.170312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:23:55.170312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:23:55.170312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:23:55.170312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:55.170312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:55.170312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:55.170312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:23:55.170312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:55.170312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:55.170312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:55.170312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:23:55.170312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:55.170312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:23:55.170312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:23:55.170312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:55.170312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:23:55.171129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:55.171129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:23:55.171129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:23:55.171129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:55.171129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:23:55.171129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:55.171129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:23:55.188039 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:23:55.188299 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:23:55.188495 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:23:55.188687 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:23:55.189067 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:23:55.189337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:55.190044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:23:55.190044 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:55.190044 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:23:55.190044 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:23:55.190044 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:55.190044 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:23:55.190044 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:55.190044 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:23:55.191431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:55.191431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:23:55.191431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:23:55.191431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:23:55.191431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:55.191431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:23:55.191431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:55.191431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:23:55.191431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:55.191431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:23:55.191431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:55.191431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:23:55.191431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:55.191431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:23:55.191431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:23:55.192085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:55.192085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:23:55.192085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:55.192085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:23:55.192085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:23:55.192085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:55.192085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:23:55.192085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:55.192085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:23:55.193084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:55.193084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:23:55.193084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:55.193084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:23:55.193084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:23:55.193084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:23:55.193084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:23:55.193084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:23:55.193084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:23:55.193084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:55.193084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:55.193684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:55.193684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:23:55.193684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:55.193684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:55.193684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:55.193684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:23:55.193684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:55.193684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:23:55.193684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:23:55.193684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:55.193684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:23:55.193684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:55.193684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:23:55.193684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:23:55.194060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:55.194060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:23:55.194060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:55.194060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:23:55.194532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:55.194532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:23:55.194532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:23:55.194532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:23:55.194532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:23:55.194532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:23:55.194532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:55.194532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:23:55.194532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:23:55.194532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:23:55.194532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:23:55.194532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:23:55.194532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:55.194532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:55.194532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:23:55.194532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:23:55.194532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:23:55.195029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:23:55.195029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:23:55.195029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:23:55.195029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:55.195029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:55.195029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:55.195029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:23:55.195029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:55.195029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:55.195029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:55.195029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:23:55.195029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:55.195029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:23:55.195029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:23:55.195029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:55.195029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:23:55.195029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:55.195029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:23:55.195029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:23:55.195526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:55.195526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:23:55.195526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:55.195526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:23:55.196506 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:23:55.197102 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:23:55.197102 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:23:55.197102 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:23:55.197482 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:23:55.197702 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:23:55.197843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:55.198355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:23:55.198355 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:55.198355 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:23:55.198355 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:23:55.198355 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:55.198355 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:23:55.198355 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:55.198355 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:23:55.199243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:55.199243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:23:55.199243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:55.199243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:23:55.199243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:23:55.199243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:23:55.199243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:23:55.199243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:55.199243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:55.199243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:55.199243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:23:55.199243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:55.199754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:55.199754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:55.199754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:23:55.199754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:55.199754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:23:55.199754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:23:55.199754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:55.199754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:23:55.199754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:55.199754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:23:55.199754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:23:55.199754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:55.199754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:23:55.200145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:55.200145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:23:55.200256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:55.200256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:23:55.200256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:23:55.200256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:23:55.200256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:23:55.200256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:23:55.200256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:55.200256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:23:55.200256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:23:55.200256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:23:55.200256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:23:55.200256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:23:55.200256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:55.200256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:55.200256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:23:55.200256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:23:55.200711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:23:55.200711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:23:55.200711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:55.200711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:55.200711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:55.200711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:23:55.200711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:55.200711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:55.200711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:55.200711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:23:55.200711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:55.200711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:23:55.200711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:23:55.200711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:55.200711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:23:55.200711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:55.200711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:23:55.200711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:23:55.200711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:55.201200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:23:55.201200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:55.201200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:23:59.932812 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:23:59.933603 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:23:59.933603 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:23:59.934395 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:23:59.967773 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:23:59.968172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:59.969640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:23:59.969640 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:59.969640 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:23:59.969640 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:23:59.969640 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:59.969640 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:23:59.969640 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:59.969640 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:23:59.970422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:59.970422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:23:59.970422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:59.970422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:23:59.970422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:23:59.970422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:23:59.970422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:23:59.970422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:59.970422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:59.970422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:59.970422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:23:59.970422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:59.970937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:23:59.970937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:59.970937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:23:59.970937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:59.970937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:23:59.970937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:23:59.970937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:59.970937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:23:59.970937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:59.970937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:23:59.970937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:23:59.970937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:59.970937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:23:59.971503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:59.971503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:23:59.971503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:59.971503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:23:59.971503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:23:59.971503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:23:59.971503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:23:59.971503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:23:59.971503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:59.971503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:23:59.971503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:23:59.971503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:23:59.971503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:23:59.971503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:23:59.971503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:23:59.972181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:23:59.972181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:23:59.972181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:23:59.972181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:23:59.972181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:23:59.972181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:59.972181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:59.972181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:23:59.972181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:23:59.972181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:23:59.972181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:23:59.972181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:23:59.972181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:23:59.972181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:23:59.972181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:23:59.972181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:23:59.972181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:23:59.972181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:23:59.972181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:23:59.973080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:23:59.973080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:23:59.973080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:23:59.973080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:23:59.973080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:23:59.973080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:24:00.149339 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:24:00.150025 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:24:00.150259 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:24:00.151364 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:00.151760 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:00.152021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:00.152996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:24:00.152996 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:00.152996 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:24:00.152996 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:24:00.152996 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:00.152996 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:24:00.152996 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:00.152996 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:24:00.156014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:00.156014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:24:00.156014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:00.156014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:24:00.156014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:24:00.156014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:24:00.156014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:24:00.156014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:00.156014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:00.156014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:00.156014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:24:00.156014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:00.156603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:00.156603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:00.156603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:24:00.156603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:00.156603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:24:00.156603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:24:00.156603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:00.156603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:24:00.156603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:00.156603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:24:00.156603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:24:00.156603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:00.156603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:24:00.157147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:00.157147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:24:00.157757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:00.157757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:24:00.157757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:24:00.157757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:24:00.157757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:24:00.157757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:24:00.157757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:00.157757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:24:00.157757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:24:00.157757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:24:00.157757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:24:00.157757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:24:00.157757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:00.157757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:00.157757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:24:00.157757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:24:00.158920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:24:00.158920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:24:00.158920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:00.158920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:00.158920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:00.158920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:24:00.158920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:00.158920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:00.158920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:00.158920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:24:00.158920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:00.158920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:24:00.158920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:24:00.158920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:00.158920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:24:00.158920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:00.158920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:24:00.158920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:24:00.158920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:00.159457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:24:00.159457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:00.159457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:24:00.163626 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:24:00.163910 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:24:00.163987 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:24:00.164080 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:24:00.164480 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:00.164683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:00.165696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:24:00.165696 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:00.165696 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:24:00.165696 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:24:00.165696 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:00.165696 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:24:00.165696 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:00.165696 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:24:00.167034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:00.167034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:24:00.167034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:24:00.167034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:24:00.167034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:00.167034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:24:00.167034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:00.167034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:24:00.167034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:00.167034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:24:00.167034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:00.167034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:24:00.167034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:00.167034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:24:00.167034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:24:00.167637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:00.167637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:24:00.167637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:00.167637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:24:00.167637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:24:00.167637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:00.167637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:24:00.167637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:00.167637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:24:00.168383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:00.168383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:24:00.168383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:00.168383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:24:00.168383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:24:00.168383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:24:00.168383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:24:00.168383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:24:00.168383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:24:00.168383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:24:00.168383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:00.168383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:00.168933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:00.168933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:24:00.168933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:00.168933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:00.168933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:00.168933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:24:00.168933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:00.168933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:24:00.168933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:24:00.168933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:00.168933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:24:00.168933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:00.168933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:24:00.168933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:24:00.169498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:00.169498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:24:00.169498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:00.169498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:24:00.171677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:00.171677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:24:00.171677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:24:00.171677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:24:00.171677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:24:00.171677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:24:00.171677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:00.171677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:24:00.171677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:24:00.171677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:24:00.171677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:24:00.171677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:24:00.171677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:00.171677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:00.171677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:24:00.171677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:24:00.172368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:24:00.172368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:24:00.172368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:24:00.172368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:24:00.172368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:24:00.172368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:00.172368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:00.172368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:00.172368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:24:00.172368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:00.172368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:00.172368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:00.172368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:24:00.172368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:00.172368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:24:00.172368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:24:00.172368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:00.172368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:24:00.173184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:00.173184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:24:00.173184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:24:00.173184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:00.173184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:24:00.173184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:00.173184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:24:00.188559 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:24:00.189033 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:24:00.189296 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:24:00.189598 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:24:00.190201 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:00.190617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:00.191309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:24:00.191309 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:00.191309 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:24:00.191309 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:24:00.191309 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:00.191309 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:24:00.191309 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:00.191309 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:24:00.192560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:00.192560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:24:00.192560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:24:00.192560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:24:00.192560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:00.192560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:24:00.192560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:00.192560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:24:00.192560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:00.192560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:24:00.192560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:00.192560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:24:00.192560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:00.192560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:24:00.192560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:24:00.193150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:00.193150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:24:00.193150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:00.193150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:24:00.193150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:24:00.193150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:00.193150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:24:00.193150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:00.193150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:24:00.193944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:00.193944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:24:00.193944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:00.193944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:24:00.193944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:24:00.193944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:24:00.193944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:24:00.193944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:24:00.193944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:24:00.193944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:00.193944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:00.194376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:00.194376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:24:00.194376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:00.194376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:00.194376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:00.194376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:24:00.194376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:00.194376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:24:00.194376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:24:00.194376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:00.194376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:24:00.194376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:00.194376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:24:00.194376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:24:00.194949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:00.194949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:24:00.194949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:00.194949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:24:00.196983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:00.196983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:24:00.196983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:24:00.196983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:24:00.196983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:24:00.196983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:24:00.196983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:00.196983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:24:00.196983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:24:00.196983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:24:00.196983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:24:00.196983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:24:00.196983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:00.196983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:00.196983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:24:00.196983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:24:00.196983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:24:00.197842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:24:00.197842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:24:00.197842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:24:00.197842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:00.197842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:00.197842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:00.197842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:24:00.197842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:00.197842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:00.197842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:00.197842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:24:00.197842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:00.197842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:24:00.197842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:24:00.197842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:00.197842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:24:00.197842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:00.197842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:24:00.197842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:24:00.198592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:00.198592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:24:00.198592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:00.198592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:24:00.198592 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:24:00.198592 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:24:00.198592 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:24:00.198592 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:24:00.198923 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:00.199570 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:00.199570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:00.200172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:24:00.200172 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:00.200172 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:24:00.200172 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:24:00.200172 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:00.200172 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:24:00.200172 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:00.200172 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:24:00.201397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:00.201397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:24:00.201397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:00.201397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:24:00.201397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:24:00.201397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:24:00.201397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:24:00.201397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:00.201397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:00.201397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:00.201397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:24:00.201397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:00.201775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:00.201775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:00.201775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:24:00.201775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:00.201775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:24:00.201775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:24:00.201775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:00.201775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:24:00.201775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:00.201775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:24:00.201775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:24:00.201775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:00.201775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:24:00.202146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:00.202146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:24:00.202647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:00.202647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:24:00.202647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:24:00.202647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:24:00.202647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:24:00.202647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:24:00.202647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:00.202647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:24:00.202647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:24:00.202647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:24:00.202647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:24:00.202647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:24:00.202647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:00.202647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:00.202647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:24:00.202647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:24:00.203087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:24:00.203087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:24:00.203087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:00.203087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:00.203087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:00.203087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:24:00.203087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:00.203087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:00.203087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:00.203087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:24:00.203087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:00.203087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:24:00.203087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:24:00.203087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:00.203087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:24:00.203087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:00.203087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:24:00.203087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:24:00.203087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:00.203645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:24:00.203645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:00.203645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:24:04.933619 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:24:04.934555 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:24:04.934555 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:24:04.935541 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:24:04.970322 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:04.970466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:04.970926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:24:04.970926 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:04.970926 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:24:04.970926 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:24:04.970926 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:04.970926 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:24:04.970926 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:04.970926 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:24:04.971612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:04.971612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:24:04.971612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:04.971612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:24:04.971612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:24:04.971612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:24:04.971612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:24:04.971612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:04.971612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:04.971612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:04.971612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:24:04.971612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:04.972388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:04.972388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:04.972388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:24:04.972388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:04.972388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:24:04.972388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:24:04.972388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:04.972388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:24:04.972388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:04.972388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:24:04.972388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:24:04.972388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:04.972388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:24:04.973143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:04.973143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:24:04.973143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:04.973143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:24:04.973143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:24:04.973143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:24:04.973143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:24:04.973143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:24:04.973143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:04.973143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:24:04.973143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:24:04.973143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:24:04.973143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:24:04.973143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:24:04.973143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:04.973996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:04.973996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:24:04.973996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:24:04.973996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:24:04.973996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:24:04.973996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:04.973996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:04.973996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:04.973996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:24:04.973996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:04.973996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:04.973996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:04.973996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:24:04.973996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:04.973996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:24:04.973996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:24:04.973996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:04.973996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:24:04.973996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:04.975358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:24:04.975358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:24:04.975358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:04.975358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:24:04.975358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:04.975358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:24:05.150121 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:24:05.150743 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:24:05.150911 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:24:05.151750 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:05.151928 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:05.152056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:05.152677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:24:05.152677 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:05.152677 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:24:05.152677 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:24:05.152677 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:05.152677 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:24:05.152677 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:05.152677 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:24:05.153873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:05.153873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:24:05.153873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:05.153873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:24:05.153873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:24:05.153873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:24:05.153873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:24:05.153873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:05.153873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:05.153873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:05.153873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:24:05.153873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:05.154487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:05.154487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:05.154487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:24:05.154487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:05.154487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:24:05.154487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:24:05.154487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:05.154487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:24:05.154487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:05.154487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:24:05.154487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:24:05.154487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:05.154487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:24:05.155060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:05.155060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:24:05.155228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:05.155228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:24:05.155228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:24:05.155228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:24:05.155228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:24:05.155228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:24:05.155228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:05.155228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:24:05.155228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:24:05.155228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:24:05.155228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:24:05.155228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:24:05.155228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:05.155228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:05.155228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:24:05.155228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:24:05.155968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:24:05.155968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:24:05.155968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:05.155968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:05.155968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:05.155968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:24:05.155968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:05.155968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:05.155968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:05.155968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:24:05.155968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:05.155968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:24:05.155968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:24:05.155968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:05.155968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:24:05.155968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:05.155968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:24:05.155968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:24:05.155968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:05.158324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:24:05.158324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:05.158324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:24:05.164110 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:24:05.164493 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:24:05.164622 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:24:05.164783 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:24:05.165097 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:05.165309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:05.165786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:24:05.165786 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:05.165786 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:24:05.165786 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:24:05.165786 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:05.165786 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:24:05.165786 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:05.165786 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:24:05.166607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:05.166607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:24:05.166607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:24:05.166607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:24:05.166607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:05.166607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:24:05.166607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:05.166607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:24:05.166607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:05.166607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:24:05.166607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:05.166607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:24:05.166607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:05.166607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:24:05.166607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:24:05.167089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:05.167089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:24:05.167089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:05.167089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:24:05.167089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:24:05.167089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:05.167089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:24:05.167089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:05.167089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:24:05.167713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:05.167713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:24:05.167713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:05.167713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:24:05.167713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:24:05.167713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:24:05.167713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:24:05.167713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:24:05.167713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:24:05.167713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:24:05.167713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:05.167713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:05.168052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:05.168052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:24:05.168052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:05.168052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:05.168052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:05.168052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:24:05.168052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:05.168052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:24:05.168052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:24:05.168052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:05.168052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:24:05.168052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:05.168052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:24:05.168052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:24:05.168641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:05.168641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:24:05.168641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:05.168641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:24:05.169019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:05.169019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:24:05.169019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:24:05.169019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:24:05.169019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:24:05.169019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:24:05.169019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:05.169019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:24:05.169019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:24:05.169019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:24:05.169019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:24:05.169019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:24:05.169019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:05.169019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:05.169019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:24:05.169019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:24:05.169793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:24:05.169793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:24:05.169793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:24:05.169793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:24:05.169793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:24:05.169793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:05.169793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:05.169793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:05.169793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:24:05.169793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:05.169793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:05.169793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:05.169793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:24:05.169793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:05.169793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:24:05.169793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:24:05.169793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:05.169793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:24:05.170632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:05.170632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:24:05.170632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:24:05.170632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:05.170632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:24:05.170632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:05.170632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:24:05.189261 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:24:05.189704 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:24:05.189970 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:24:05.190251 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:24:05.190618 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:05.190892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:05.191554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:24:05.191554 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:05.191554 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:24:05.191554 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:24:05.191554 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:05.191554 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:24:05.191554 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:05.191554 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:24:05.192990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:05.192990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:24:05.192990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:24:05.192990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:24:05.192990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:05.192990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:24:05.192990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:05.192990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:24:05.192990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:05.192990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:24:05.192990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:05.192990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:24:05.192990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:05.192990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:24:05.192990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:24:05.193671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:05.193671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:24:05.193671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:05.193671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:24:05.193671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:24:05.193671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:05.193671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:24:05.193671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:05.193671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:24:05.194880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:05.194880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:24:05.194880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:05.194880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:24:05.194880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:24:05.194880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:24:05.194880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:24:05.194880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:24:05.194880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:24:05.194880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:05.194880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:05.195494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:05.195494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:24:05.195494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:05.195494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:05.195494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:05.195494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:24:05.195494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:05.195494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:24:05.195494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:24:05.195494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:05.195494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:24:05.195494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:05.195494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:24:05.195494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:24:05.196091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:05.196091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:24:05.196091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:05.196091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:24:05.196516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:05.196516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:24:05.196516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:24:05.196516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:24:05.196516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:24:05.196516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:24:05.196516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:05.196516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:24:05.196516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:24:05.196516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:24:05.196516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:24:05.196516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:24:05.196516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:05.196516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:05.196516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:24:05.196516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:24:05.196516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:24:05.197012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:24:05.197012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:24:05.197012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:24:05.197012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:05.197012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:05.197012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:05.197012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:24:05.197012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:05.197012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:05.197012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:05.197012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:24:05.197012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:05.197012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:24:05.197012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:24:05.197012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:05.197012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:24:05.197012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:05.197012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:24:05.197012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:24:05.197548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:05.197548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:24:05.197548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:05.197548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:24:05.198188 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:24:05.198522 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:24:05.198629 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:24:05.198770 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:24:05.199035 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:05.199251 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:05.199447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:05.199852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:24:05.199852 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:05.199852 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:24:05.199852 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:24:05.199852 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:05.199852 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:24:05.199852 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:05.199852 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:24:05.200666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:05.200666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:24:05.200666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:05.200666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:24:05.200666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:24:05.200666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:24:05.200666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:24:05.200666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:05.200666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:05.200666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:05.200666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:24:05.200666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:05.201008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:05.201008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:05.201008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:24:05.201008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:05.201008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:24:05.201008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:24:05.201008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:05.201008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:24:05.201008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:05.201008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:24:05.201008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:24:05.201008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:05.201008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:24:05.201360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:05.201360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:24:05.201607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:05.201607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:24:05.201607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:24:05.201607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:24:05.201607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:24:05.201607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:24:05.201607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:05.201607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:24:05.201607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:24:05.201607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:24:05.201607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:24:05.201607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:24:05.201607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:05.201607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:05.201607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:24:05.201607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:24:05.202047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:24:05.202047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:24:05.202047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:05.202047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:05.202047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:05.202047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:24:05.202047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:05.202047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:05.202047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:05.202047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:24:05.202047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:05.202047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:24:05.202047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:24:05.202047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:05.202047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:24:05.202047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:05.202047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:24:05.202047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:24:05.202047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:05.202597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:24:05.202597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:05.202597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:24:09.934212 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:24:09.935161 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:24:09.935161 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:24:09.936386 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:24:09.968529 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:09.968529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:09.969267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:24:09.969267 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:09.969267 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:24:09.969267 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:24:09.969267 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:09.969267 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:24:09.969267 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:09.969267 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:24:09.970007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:09.970007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:24:09.970007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:09.970007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:24:09.970007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:24:09.970007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:24:09.970007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:24:09.970007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:09.970007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:09.970007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:09.970007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:24:09.970007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:09.970993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:09.970993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:09.970993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:24:09.970993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:09.970993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:24:09.970993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:24:09.970993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:09.970993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:24:09.970993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:09.970993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:24:09.970993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:24:09.970993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:09.970993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:24:09.971674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:09.971674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:24:09.971674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:09.971674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:24:09.971674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:24:09.971674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:24:09.971674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:24:09.971674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:24:09.971674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:09.971674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:24:09.971674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:24:09.971674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:24:09.971674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:24:09.971674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:24:09.971674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:09.972374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:09.972374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:24:09.972374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:24:09.972374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:24:09.972374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:24:09.972374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:09.972374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:09.972374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:09.972374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:24:09.972374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:09.972374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:09.972374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:09.972374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:24:09.972374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:09.972374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:24:09.972374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:24:09.972374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:09.972374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:24:09.972374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:09.973252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:24:09.973252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:24:09.973252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:09.973252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:24:09.973252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:09.973252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:24:10.150814 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:24:10.151438 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:24:10.151641 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:24:10.152700 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:10.153044 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:10.153311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:10.154190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:24:10.154190 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:10.154190 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:24:10.154190 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:24:10.154190 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:10.154190 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:24:10.154190 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:10.154190 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:24:10.155681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:10.155681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:24:10.155681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:10.155681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:24:10.155681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:24:10.155681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:24:10.155681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:24:10.155681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:10.155681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:10.155681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:10.155681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:24:10.155681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:10.156263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:10.156263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:10.156263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:24:10.156263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:10.156263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:24:10.156263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:24:10.156263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:10.156263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:24:10.156263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:10.156263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:24:10.156263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:24:10.156263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:10.156263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:24:10.156898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:10.156898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:24:10.157309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:10.157309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:24:10.157309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:24:10.157309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:24:10.157309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:24:10.157309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:24:10.157309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:10.157309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:24:10.157309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:24:10.157309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:24:10.157309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:24:10.157309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:24:10.157309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:10.157309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:10.157309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:24:10.157309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:24:10.158115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:24:10.158115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:24:10.158115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:10.158115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:10.158115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:10.158115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:24:10.158115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:10.158115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:10.158115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:10.158115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:24:10.158115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:10.158115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:24:10.158115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:24:10.158115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:10.158115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:24:10.158115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:10.158115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:24:10.158115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:24:10.158115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:10.159121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:24:10.159121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:10.159121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:24:10.164696 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:24:10.165260 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:24:10.165433 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:24:10.165560 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:24:10.165984 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:10.166287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:10.166982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:24:10.166982 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:10.166982 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:24:10.166982 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:24:10.166982 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:10.166982 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:24:10.166982 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:10.166982 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:24:10.167769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:10.167769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:24:10.167769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:24:10.167769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:24:10.167769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:10.167769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:24:10.167769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:10.167769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:24:10.167769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:10.167769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:24:10.167769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:10.167769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:24:10.167769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:10.167769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:24:10.167769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:24:10.168239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:10.168239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:24:10.168239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:10.168239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:24:10.168239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:24:10.168239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:10.168239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:24:10.168239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:10.168239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:24:10.168543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:10.168543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:24:10.168543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:10.168543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:24:10.168543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:24:10.168543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:24:10.168543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:24:10.168543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:24:10.168543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:24:10.168543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:24:10.168543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:10.168543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:10.169004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:10.169004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:24:10.169004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:10.169004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:10.169004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:10.169004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:24:10.169004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:10.169004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:24:10.169004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:24:10.169004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:10.169004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:24:10.169004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:10.169004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:24:10.169004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:24:10.169375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:10.169375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:24:10.169375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:10.169375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:24:10.169509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:10.169509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:24:10.169509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:24:10.169509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:24:10.169509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:24:10.169509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:24:10.169509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:10.169509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:24:10.169509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:24:10.169509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:24:10.169509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:24:10.169509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:24:10.169509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:10.169509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:10.169509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:24:10.169509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:24:10.169949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:24:10.169949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:24:10.169949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:24:10.169949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:24:10.169949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:24:10.169949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:10.169949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:10.169949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:10.169949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:24:10.169949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:10.169949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:10.169949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:10.169949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:24:10.169949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:10.169949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:24:10.169949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:24:10.169949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:10.169949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:24:10.170509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:10.170509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:24:10.170509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:24:10.170509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:10.170509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:24:10.170509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:10.170509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:24:10.189762 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:24:10.190088 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:24:10.190333 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:24:10.190560 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:24:10.191025 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:10.191384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:10.192132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:24:10.192132 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:10.192132 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:24:10.192132 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:24:10.192132 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:10.192132 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:24:10.192132 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:10.192132 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:24:10.193273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:10.193273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:24:10.193273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:24:10.193273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:24:10.193273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:10.193273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:24:10.193273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:10.193273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:24:10.193273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:10.193273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:24:10.193273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:10.193273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:24:10.193273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:10.193273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:24:10.193273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:24:10.193976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:10.193976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:24:10.193976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:10.193976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:24:10.193976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:24:10.193976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:10.193976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:24:10.193976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:10.193976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:24:10.194694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:10.194694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:24:10.194694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:10.194694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:24:10.194694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:24:10.194694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:24:10.194694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:24:10.194694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:24:10.194694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:24:10.194694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:10.194694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:10.195235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:10.195235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:24:10.195235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:10.195235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:10.195235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:10.195235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:24:10.195235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:10.195235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:24:10.195235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:24:10.195235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:10.195235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:24:10.195235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:10.195235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:24:10.195235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:24:10.195831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:10.195831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:24:10.195831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:10.195831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:24:10.196487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:10.196487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:24:10.196487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:24:10.196487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:24:10.196487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:24:10.196487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:24:10.196487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:10.196487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:24:10.196487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:24:10.196487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:24:10.196487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:24:10.196487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:24:10.196487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:10.196487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:10.196487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:24:10.196487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:24:10.196487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:24:10.197209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:24:10.197209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:24:10.197209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:24:10.197209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:10.197209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:10.197209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:10.197209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:24:10.197209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:10.197209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:10.197209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:10.197209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:24:10.197209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:10.197209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:24:10.197209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:24:10.197209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:10.197209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:24:10.197209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:10.197209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:24:10.197209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:24:10.198140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:10.198140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:24:10.198140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:10.198140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:24:10.199975 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:24:10.200422 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:24:10.200613 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:24:10.200845 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:24:10.201261 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:10.201633 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:10.201855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:10.202600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:24:10.202600 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:10.202600 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:24:10.202600 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:24:10.202600 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:10.202600 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:24:10.202600 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:10.202600 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:24:10.203400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:10.203400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:24:10.203400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:10.203400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:24:10.203400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:24:10.203400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:24:10.203400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:24:10.203400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:10.203400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:10.203400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:10.203400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:24:10.203400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:10.203816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:10.203816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:10.203816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:24:10.203816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:10.203816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:24:10.203816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:24:10.203816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:10.203816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:24:10.203816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:10.203816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:24:10.203816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:24:10.203816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:10.203816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:24:10.204176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:10.204176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:24:10.204348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:10.204348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:24:10.204348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:24:10.204348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:24:10.204348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:24:10.204348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:24:10.204348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:10.204348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:24:10.204348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:24:10.204348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:24:10.204348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:24:10.204348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:24:10.204348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:10.204348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:10.204348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:24:10.204348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:24:10.204810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:24:10.204810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:24:10.204810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:10.204810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:10.204810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:10.204810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:24:10.204810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:10.204810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:10.204810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:10.204810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:24:10.204810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:10.204810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:24:10.204810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:24:10.204810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:10.204810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:24:10.204810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:10.204810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:24:10.204810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:24:10.204810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:10.205348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:24:10.205348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:10.205348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:24:14.935432 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:24:14.936748 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:24:14.936748 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:24:14.937390 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:24:14.969346 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:14.969806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:14.970628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:24:14.970628 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:14.970628 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:24:14.970628 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:24:14.970628 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:14.970628 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:24:14.970628 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:14.970628 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:24:14.971356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:14.971356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:24:14.971356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:14.971356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:24:14.971356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:24:14.971356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:24:14.971356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:24:14.971356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:14.971356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:14.971356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:14.971356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:24:14.971356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:14.971975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:14.971975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:14.971975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:24:14.971975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:14.971975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:24:14.971975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:24:14.971975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:14.971975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:24:14.971975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:14.971975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:24:14.971975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:24:14.971975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:14.971975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:24:14.972644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:14.972644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:24:14.972644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:14.972644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:24:14.972644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:24:14.972644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:24:14.972644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:24:14.972644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:24:14.972644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:14.972644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:24:14.972644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:24:14.972644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:24:14.972644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:24:14.972644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:24:14.972644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:14.973436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:14.973436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:24:14.973436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:24:14.973436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:24:14.973436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:24:14.973436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:14.973436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:14.973436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:14.973436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:24:14.973436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:14.973436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:14.973436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:14.973436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:24:14.973436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:14.973436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:24:14.973436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:24:14.973436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:14.973436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:24:14.973436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:14.974042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:24:14.974042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:24:14.974042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:14.974042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:24:14.974042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:14.974042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:24:15.151448 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:24:15.152344 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:24:15.152344 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:24:15.153223 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:15.153564 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:15.153788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:15.154631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:24:15.154631 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:15.154631 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:24:15.154631 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:24:15.154631 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:15.154631 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:24:15.154631 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:15.154631 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:24:15.156051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:15.156051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:24:15.156051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:15.156051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:24:15.156051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:24:15.156051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:24:15.156051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:24:15.156051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:15.156051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:15.156051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:15.156051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:24:15.156051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:15.157843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:15.157843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:15.157843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:24:15.157843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:15.157843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:24:15.157843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:24:15.157843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:15.157843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:24:15.157843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:15.157843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:24:15.157843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:24:15.157843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:15.157843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:24:15.160134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:15.160134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:24:15.160824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:15.160824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:24:15.160824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:24:15.160824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:24:15.160824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:24:15.160824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:24:15.160824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:15.160824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:24:15.160824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:24:15.160824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:24:15.160824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:24:15.160824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:24:15.160824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:15.160824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:15.160824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:24:15.160824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:24:15.161883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:24:15.161883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:24:15.161883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:15.161883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:15.161883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:15.161883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:24:15.161883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:15.161883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:15.161883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:15.161883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:24:15.161883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:15.161883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:24:15.161883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:24:15.161883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:15.161883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:24:15.161883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:15.161883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:24:15.161883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:24:15.161883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:15.162755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:24:15.162755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:15.162755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:24:15.165242 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:24:15.165581 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:24:15.165691 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:24:15.165814 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:24:15.166224 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:15.166550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:15.167175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:24:15.167175 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:15.167175 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:24:15.167175 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:24:15.167175 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:15.167175 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:24:15.167175 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:15.167175 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:24:15.167910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:15.167910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:24:15.167910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:24:15.167910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:24:15.167910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:15.167910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:24:15.167910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:15.167910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:24:15.167910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:15.167910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:24:15.167910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:15.167910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:24:15.167910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:15.167910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:24:15.167910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:24:15.168357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:15.168357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:24:15.168357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:15.168357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:24:15.168357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:24:15.168357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:15.168357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:24:15.168357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:15.168357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:24:15.168700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:15.168700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:24:15.168700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:15.168700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:24:15.168700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:24:15.168700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:24:15.168700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:24:15.168700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:24:15.168700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:24:15.168700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:24:15.168700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:15.168700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:15.169045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:15.169045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:24:15.169045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:15.169045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:15.169045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:15.169045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:24:15.169045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:15.169045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:24:15.169045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:24:15.169045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:15.169045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:24:15.169045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:15.169045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:24:15.169045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:24:15.169753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:15.169753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:24:15.169753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:15.169753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:24:15.169753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:15.169753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:24:15.169753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:24:15.169753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:24:15.169753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:24:15.169753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:24:15.169753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:15.169753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:24:15.169753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:24:15.169753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:24:15.170318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:24:15.170318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:24:15.170318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:15.170318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:15.170318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:24:15.170318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:24:15.170318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:24:15.170318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:24:15.170318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:24:15.170318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:24:15.170318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:24:15.170318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:15.170318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:15.170318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:15.170318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:24:15.170318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:15.170318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:15.170318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:15.170318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:24:15.170820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:15.170820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:24:15.170820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:24:15.170820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:15.170820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:24:15.170820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:15.170820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:24:15.170820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:24:15.170820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:15.170820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:24:15.170820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:15.170820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:24:15.190236 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:24:15.190616 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:24:15.190802 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:24:15.191034 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:24:15.191364 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:15.191680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:15.192611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:24:15.192611 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:15.192611 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:24:15.192611 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:24:15.192611 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:15.192611 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:24:15.192611 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:15.192611 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:24:15.193774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:15.193774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:24:15.193774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:24:15.193774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:24:15.193774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:15.193774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:24:15.193774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:15.193774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:24:15.193774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:15.193774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:24:15.193774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:15.193774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:24:15.193774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:15.193774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:24:15.193774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:24:15.194365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:15.194365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:24:15.194365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:15.194365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:24:15.194365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:24:15.194365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:15.194365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:24:15.194365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:15.194365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:24:15.195130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:15.195130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:24:15.195130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:15.195130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:24:15.195130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:24:15.195130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:24:15.195130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:24:15.195130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:24:15.195130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:24:15.195130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:15.195130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:15.196588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:15.196588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:24:15.196588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:15.196588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:15.196588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:15.196588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:24:15.196588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:15.196588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:24:15.196588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:24:15.196588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:15.196588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:24:15.196588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:15.196588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:24:15.196588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:24:15.197178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:15.197178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:24:15.197178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:15.197178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:24:15.197178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:15.197178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:24:15.197178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:24:15.197178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:24:15.197178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:24:15.197178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:24:15.197178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:15.197178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:24:15.197178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:24:15.197178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:24:15.197560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:24:15.197560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:24:15.197560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:15.197560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:15.197560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:24:15.197560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:24:15.197560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:24:15.197560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:24:15.197560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:24:15.197560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:24:15.197560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:15.197560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:15.197560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:15.197560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:24:15.197560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:15.197560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:15.197560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:15.197560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:24:15.197560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:15.198066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:24:15.198066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:24:15.198066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:15.198066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:24:15.198066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:15.198066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:24:15.198066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:24:15.198066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:15.198066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:24:15.198066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:15.198066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:24:15.200388 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:24:15.200645 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:24:15.200740 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:24:15.200894 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:24:15.201748 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:15.202029 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:15.202256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:15.202933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:24:15.202933 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:15.202933 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:24:15.202933 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:24:15.202933 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:15.202933 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:24:15.202933 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:15.202933 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:24:15.203717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:15.203717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:24:15.203717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:15.203717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:24:15.203717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:24:15.203717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:24:15.203717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:24:15.203717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:15.203717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:15.203717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:15.203717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:24:15.203717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:15.204094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:15.204094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:15.204094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:24:15.204094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:15.204094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:24:15.204094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:24:15.204094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:15.204094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:24:15.204094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:15.204094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:24:15.204094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:24:15.204094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:15.204094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:24:15.204635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:15.204635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:24:15.204635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:15.204635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:24:15.204635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:24:15.204635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:24:15.204635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:24:15.204635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:24:15.204635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:15.204635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:24:15.204635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:24:15.204635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:24:15.204635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:24:15.204635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:24:15.204635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:15.205027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:15.205027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:24:15.205027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:24:15.205027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:24:15.205027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:24:15.205027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:15.205027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:15.205027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:15.205027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:24:15.205027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:15.205027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:15.205027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:15.205027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:24:15.205027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:15.205027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:24:15.205027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:24:15.205027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:15.205027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:24:15.205027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:15.205593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:24:15.205593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:24:15.205593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:15.205593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:24:15.205593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:15.205593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:24:19.936758 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:24:19.937583 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:24:19.937813 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:24:19.939142 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:24:19.974633 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:19.975067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:19.976229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:24:19.976229 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:19.976229 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:24:19.976229 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:24:19.976229 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:19.976229 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:24:19.976229 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:19.976229 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:24:19.977902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:19.977902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:24:19.977902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:19.977902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:24:19.977902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:24:19.977902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:24:19.977902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:24:19.977902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:19.977902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:19.977902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:19.977902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:24:19.977902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:19.978195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:19.978195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:19.978195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:24:19.978195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:19.978195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:24:19.978195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:24:19.978195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:19.978195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:24:19.978195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:19.978195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:24:19.978195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:24:19.978195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:19.978195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:24:19.978451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:19.978451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:24:19.979907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:19.979907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:24:19.979907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:24:19.979907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:24:19.979907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:24:19.979907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:24:19.979907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:19.979907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:24:19.979907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:24:19.979907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:24:19.979907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:24:19.979907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:24:19.979907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:19.979907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:19.979907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:24:19.979907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:24:19.980316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:24:19.980316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:24:19.980316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:19.980316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:19.980316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:19.980316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:24:19.980316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:19.980316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:19.980316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:19.980316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:24:19.980316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:19.980316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:24:19.980316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:24:19.980316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:19.980316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:24:19.980316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:19.980316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:24:19.980316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:24:19.980316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:19.980697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:24:19.980697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:19.980697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:24:20.152340 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:24:20.152729 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:24:20.152961 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:24:20.153709 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:20.154056 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:20.154292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:20.155139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:24:20.155139 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:20.155139 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:24:20.155139 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:24:20.155139 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:20.155139 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:24:20.155139 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:20.155139 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:24:20.156264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:20.156264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:24:20.156264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:20.156264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:24:20.156264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:24:20.156264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:24:20.156264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:24:20.156264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:20.156264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:20.156264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:20.156264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:24:20.156264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:20.157049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:20.157049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:20.157049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:24:20.157049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:20.157049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:24:20.157049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:24:20.157049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:20.157049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:24:20.157049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:20.157049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:24:20.157049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:24:20.157049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:20.157049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:24:20.157682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:20.157682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:24:20.158518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:20.158518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:24:20.158518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:24:20.158518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:24:20.158518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:24:20.158518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:24:20.158518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:20.158518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:24:20.158518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:24:20.158518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:24:20.158518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:24:20.158518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:24:20.158518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:20.158518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:20.158518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:24:20.158518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:24:20.159268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:24:20.159268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:24:20.159268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:20.159268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:20.159268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:20.159268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:24:20.159268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:20.159268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:20.159268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:20.159268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:24:20.159268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:20.159268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:24:20.159268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:24:20.159268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:20.159268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:24:20.159268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:20.159268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:24:20.159268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:24:20.159268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:20.160183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:24:20.160183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:20.160183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:24:20.165602 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:24:20.165860 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:24:20.165970 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:24:20.166086 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:24:20.166574 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:20.166875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:20.167602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:24:20.167602 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:20.167602 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:24:20.167602 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:24:20.167602 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:20.167602 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:24:20.167602 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:20.167602 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:24:20.168392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:20.168392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:24:20.168392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:24:20.168392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:24:20.168392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:20.168392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:24:20.168392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:20.168392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:24:20.168392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:20.168392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:24:20.168392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:20.168392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:24:20.168392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:20.168392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:24:20.168392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:24:20.168797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:20.168797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:24:20.168797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:20.168797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:24:20.168797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:24:20.168797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:20.168797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:24:20.168797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:20.168797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:24:20.169298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:20.169298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:24:20.169298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:20.169298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:24:20.169298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:24:20.169298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:24:20.169298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:24:20.169298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:24:20.169298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:24:20.169298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:24:20.169298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:20.169298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:20.169891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:20.169891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:24:20.169891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:20.169891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:20.169891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:20.169891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:24:20.169891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:20.169891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:24:20.169891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:24:20.169891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:20.169891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:24:20.169891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:20.169891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:24:20.169891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:24:20.170468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:20.170468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:24:20.170468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:20.170468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:24:20.170468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:20.170468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:24:20.170468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:24:20.170468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:24:20.170468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:24:20.170468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:24:20.170468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:20.170468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:24:20.170468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:24:20.170468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:24:20.171083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:24:20.171083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:24:20.171083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:20.171083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:20.171083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:24:20.171083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:24:20.171083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:24:20.171083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:24:20.171083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:24:20.171083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:24:20.171083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:24:20.171083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:20.171083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:20.171083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:20.171083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:24:20.171083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:20.171083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:20.171083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:20.171083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:24:20.171974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:20.171974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:24:20.171974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:24:20.171974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:20.171974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:24:20.171974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:20.171974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:24:20.171974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:24:20.171974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:20.171974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:24:20.171974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:20.171974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:24:20.190750 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:24:20.191139 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:24:20.191326 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:24:20.191552 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:24:20.192136 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:20.192414 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:20.193370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:24:20.193370 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:20.193370 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:24:20.193370 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:24:20.193370 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:20.193370 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:24:20.193370 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:20.193370 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:24:20.194457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:20.194457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:24:20.194457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:24:20.194457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:24:20.194457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:20.194457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:24:20.194457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:20.194457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:24:20.194457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:20.194457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:24:20.194457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:20.194457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:24:20.194457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:20.194457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:24:20.194457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:24:20.195132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:20.195132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:24:20.195132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:20.195132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:24:20.195132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:24:20.195132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:20.195132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:24:20.195132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:20.195132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:24:20.195836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:20.195836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:24:20.195836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:20.195836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:24:20.195836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:24:20.195836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:24:20.195836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:24:20.195836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:24:20.195836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:24:20.195836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:20.195836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:20.196313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:20.196313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:24:20.196313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:20.196313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:20.196313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:20.196313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:24:20.196313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:20.196313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:24:20.196313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:24:20.196313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:20.196313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:24:20.196313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:20.196313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:24:20.196313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:24:20.196910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:20.196910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:24:20.196910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:20.196910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:24:20.197495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:20.197495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:24:20.197495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:24:20.197495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:24:20.197495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:24:20.197495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:24:20.197495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:20.197495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:24:20.197495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:24:20.197495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:24:20.197495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:24:20.197495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:24:20.197495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:20.197495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:20.197495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:24:20.197495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:24:20.197495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:24:20.198220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:24:20.198220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:24:20.198220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:24:20.198220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:20.198220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:20.198220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:20.198220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:24:20.198220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:20.198220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:20.198220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:20.198220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:24:20.198220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:20.198220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:24:20.198220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:24:20.198220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:20.198220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:24:20.198220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:20.198220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:24:20.198220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:24:20.199179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:20.199179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:24:20.199179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:20.199179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:24:20.200905 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:24:20.201204 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:24:20.201311 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:24:20.201473 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:24:20.201871 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:20.202220 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:20.202427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:20.203106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:24:20.203106 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:20.203106 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:24:20.203106 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:24:20.203106 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:20.203106 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:24:20.203106 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:20.203106 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:24:20.203916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:20.203916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:24:20.203916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:20.203916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:24:20.203916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:24:20.203916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:24:20.203916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:24:20.203916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:20.203916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:20.203916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:20.203916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:24:20.203916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:20.204303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:20.204303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:20.204303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:24:20.204303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:20.204303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:24:20.204303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:24:20.204303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:20.204303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:24:20.204303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:20.204303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:24:20.204303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:24:20.204303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:20.204303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:24:20.204703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:20.204703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:24:20.204839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:20.204839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:24:20.204839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:24:20.204839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:24:20.204839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:24:20.204839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:24:20.204839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:20.204839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:24:20.204839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:24:20.204839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:24:20.204839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:24:20.204839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:24:20.204839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:20.204839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:20.204839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:24:20.204839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:24:20.205293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:24:20.205293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:24:20.205293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:20.205293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:20.205293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:20.205293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:24:20.205293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:20.205293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:20.205293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:20.205293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:24:20.205293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:20.205293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:24:20.205293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:24:20.205293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:20.205293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:24:20.205293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:20.205293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:24:20.205293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:24:20.205293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:20.205848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:24:20.205848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:20.205848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:24:24.937986 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:24:24.938885 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:24:24.938885 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:24:24.940189 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:24:24.942520 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG dbcounter [-] [105581] Writing DB stats neutron:SELECT=240 {{(pid=105581) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 14:24:24.972179 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:24.972664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:24.973544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:24:24.973544 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:24.973544 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:24:24.973544 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:24:24.973544 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:24.973544 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:24:24.973544 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:24.973544 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:24:24.974305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:24.974305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:24:24.974305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:24.974305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:24:24.974305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:24:24.974305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:24:24.974305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:24:24.974305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:24.974305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:24.974305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:24.974305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:24:24.974305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:24.975276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:24.975276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:24.975276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:24:24.975276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:24.975276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:24:24.975276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:24:24.975276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:24.975276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:24:24.975276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:24.975276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:24:24.975276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:24:24.975276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:24.975276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:24:24.975962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:24.975962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:24:24.975962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:24.975962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:24:24.975962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:24:24.975962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:24:24.975962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:24:24.975962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:24:24.975962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:24.975962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:24:24.975962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:24:24.975962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:24:24.975962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:24:24.975962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:24:24.975962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:24.976837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:24.976837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:24:24.976837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:24:24.976837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:24:24.976837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:24:24.976837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:24.976837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:24.976837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:24.976837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:24:24.976837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:24.976837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:24.976837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:24.976837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:24:24.976837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:24.976837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:24:24.976837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:24:24.976837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:24.976837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:24:24.976837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:24.977777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:24:24.977777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:24:24.977777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:24.977777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:24:24.977777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:24.977777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:24:25.152863 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:24:25.153365 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:24:25.153595 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:24:25.154917 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:25.155271 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:25.155576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:25.156484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:24:25.156484 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:25.156484 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:24:25.156484 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:24:25.156484 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:25.156484 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:24:25.156484 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:25.156484 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:24:25.157968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:25.157968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:24:25.157968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:25.157968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:24:25.157968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:24:25.157968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:24:25.157968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:24:25.157968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:25.157968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:25.157968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:25.157968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:24:25.157968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:25.158685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:25.158685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:25.158685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:24:25.158685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:25.158685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:24:25.158685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:24:25.158685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:25.158685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:24:25.158685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:25.158685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:24:25.158685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:24:25.158685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:25.158685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:24:25.159278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:25.159278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:24:25.159774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:25.159774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:24:25.159774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:24:25.159774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:24:25.159774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:24:25.159774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:24:25.159774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:25.159774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:24:25.159774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:24:25.159774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:24:25.159774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:24:25.159774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:24:25.159774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:25.159774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:25.159774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:24:25.159774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:24:25.160793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:24:25.160793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:24:25.160793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:25.160793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:25.160793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:25.160793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:24:25.160793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:25.160793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:25.160793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:25.160793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:24:25.160793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:25.160793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:24:25.160793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:24:25.160793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:25.160793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:24:25.160793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:25.160793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:24:25.160793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:24:25.160793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:25.161727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:24:25.161727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:25.161727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:24:25.166216 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:24:25.166643 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:24:25.166751 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:24:25.166877 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:24:25.167323 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:25.167683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:25.168240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:24:25.168240 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:25.168240 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:24:25.168240 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:24:25.168240 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:25.168240 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:24:25.168240 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:25.168240 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:24:25.169533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:25.169533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:24:25.169533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:24:25.169533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:24:25.169533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:25.169533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:24:25.169533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:25.169533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:24:25.169533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:25.169533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:24:25.169533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:25.169533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:24:25.169533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:25.169533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:24:25.169533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:24:25.170070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:25.170070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:24:25.170070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:25.170070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:24:25.170070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:24:25.170070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:25.170070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:24:25.170070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:25.170070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:24:25.170977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:25.170977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:24:25.170977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:25.170977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:24:25.170977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:24:25.170977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:24:25.170977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:24:25.170977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:24:25.170977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:24:25.170977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:24:25.170977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:25.170977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:25.171548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:25.171548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:24:25.171548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:25.171548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:25.171548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:25.171548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:24:25.171548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:25.171548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:24:25.171548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:24:25.171548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:25.171548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:24:25.171548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:25.171548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:24:25.171548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:24:25.172097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:25.172097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:24:25.172097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:25.172097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:24:25.173826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:25.173826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:24:25.173826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:24:25.173826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:24:25.173826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:24:25.173826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:24:25.173826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:25.173826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:24:25.173826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:24:25.173826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:24:25.173826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:24:25.173826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:24:25.173826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:25.173826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:25.173826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:24:25.173826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:24:25.174572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:24:25.174572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:24:25.174572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:24:25.174572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:24:25.174572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:24:25.174572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:25.174572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:25.174572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:25.174572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:24:25.174572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:25.174572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:25.174572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:25.174572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:24:25.174572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:25.174572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:24:25.174572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:24:25.174572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:25.174572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:24:25.175432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:25.175432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:24:25.175432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:24:25.175432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:25.175432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:24:25.175432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:25.175432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:24:25.191262 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:24:25.191658 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:24:25.191884 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:24:25.192092 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:24:25.192823 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:25.193278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:25.194200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:24:25.194200 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:25.194200 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:24:25.194200 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:24:25.194200 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:25.194200 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:24:25.194200 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:25.194200 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:24:25.195520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:25.195520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:24:25.195520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:24:25.195520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:24:25.195520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:25.195520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:24:25.195520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:25.195520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:24:25.195520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:25.195520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:24:25.195520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:25.195520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:24:25.195520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:25.195520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:24:25.195520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:24:25.196196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:25.196196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:24:25.196196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:25.196196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:24:25.196196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:24:25.196196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:25.196196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:24:25.196196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:25.196196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:24:25.196958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:25.196958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:24:25.196958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:25.196958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:24:25.196958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:24:25.196958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:24:25.196958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:24:25.196958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:24:25.196958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:24:25.196958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:25.196958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:25.197700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:25.197700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:24:25.197700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:25.197700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:25.197700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:25.197700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:24:25.197700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:25.197700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:24:25.197700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:24:25.197700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:25.197700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:24:25.197700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:25.197700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:24:25.197700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:24:25.198300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:25.198300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:24:25.198300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:25.198300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:24:25.198922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:25.198922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:24:25.198922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:24:25.198922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:24:25.198922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:24:25.198922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:24:25.198922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:25.198922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:24:25.198922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:24:25.198922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:24:25.198922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:24:25.198922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:24:25.198922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:25.198922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:25.198922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:24:25.198922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:24:25.198922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:24:25.199738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:24:25.199738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:24:25.199738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:24:25.199738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:25.199738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:25.199738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:25.199738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:24:25.199738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:25.199738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:25.199738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:25.199738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:24:25.199738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:25.199738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:24:25.199738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:24:25.199738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:25.199738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:24:25.199738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:25.199738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:24:25.199738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:24:25.200572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:25.200572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:24:25.200572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:25.200572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:24:25.201287 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:24:25.201656 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:24:25.201841 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:24:25.202082 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:24:25.202536 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:25.202793 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:25.202944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:25.203424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:24:25.203424 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:25.203424 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:24:25.203424 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:24:25.203424 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:25.203424 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:24:25.203424 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:25.203424 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:24:25.204256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:25.204256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:24:25.204256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:25.204256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:24:25.204256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:24:25.204256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:24:25.204256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:24:25.204256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:25.204256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:25.204256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:25.204256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:24:25.204256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:25.204856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:25.204856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:25.204856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:24:25.204856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:25.204856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:24:25.204856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:24:25.204856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:25.204856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:24:25.204856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:25.204856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:24:25.204856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:24:25.204856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:25.204856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:24:25.205192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:25.205192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:24:25.205192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:25.205192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:24:25.205192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:24:25.205192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:24:25.205192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:24:25.205192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:24:25.205192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:25.205192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:24:25.205192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:24:25.205192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:24:25.205192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:24:25.205192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:24:25.205192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:25.205610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:25.205610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:24:25.205610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:24:25.205610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:24:25.205610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:24:25.205610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:25.205610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:25.205610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:25.205610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:24:25.205610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:25.205610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:25.205610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:25.205610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:24:25.205610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:25.205610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:24:25.205610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:24:25.205610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:25.205610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:24:25.205610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:25.206191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:24:25.206191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:24:25.206191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:25.206191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:24:25.206191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:25.206191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:24:29.938750 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:24:29.939919 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:24:29.940002 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:24:29.941273 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:24:29.973677 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:29.974012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:29.974705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:24:29.974705 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:29.974705 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:24:29.974705 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:24:29.974705 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:29.974705 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:24:29.974705 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:29.974705 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:24:29.975698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:29.975698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:24:29.975698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:29.975698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:24:29.975698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:24:29.975698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:24:29.975698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:24:29.975698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:29.975698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:29.975698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:29.975698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:24:29.975698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:29.976574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:29.976574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:29.976574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:24:29.976574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:29.976574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:24:29.976574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:24:29.976574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:29.976574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:24:29.976574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:29.976574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:24:29.976574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:24:29.976574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:29.976574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:24:29.977230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:29.977230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:24:29.977230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:29.977230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:24:29.977230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:24:29.977230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:24:29.977230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:24:29.977230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:24:29.977230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:29.977230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:24:29.977230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:24:29.977230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:24:29.977230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:24:29.977230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:24:29.977230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:29.978037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:29.978037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:24:29.978037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:24:29.978037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:24:29.978037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:24:29.978037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:29.978037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:29.978037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:29.978037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:24:29.978037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:29.978037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:29.978037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:29.978037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:24:29.978037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:29.978037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:24:29.978037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:24:29.978037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:29.978037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:24:29.978037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:29.978895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:24:29.978895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:24:29.978895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:29.978895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:24:29.978895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:29.978895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:24:30.153687 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:24:30.154209 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:24:30.154332 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:24:30.155329 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:30.155700 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:30.155960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:30.156841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:24:30.156841 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:30.156841 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:24:30.156841 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:24:30.156841 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:30.156841 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:24:30.156841 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:30.156841 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:24:30.158195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:30.158195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:24:30.158195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:30.158195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:24:30.158195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:24:30.158195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:24:30.158195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:24:30.158195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:30.158195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:30.158195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:30.158195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:24:30.158195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:30.158990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:30.158990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:30.158990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:24:30.158990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:30.158990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:24:30.158990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:24:30.158990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:30.158990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:24:30.158990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:30.158990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:24:30.158990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:24:30.158990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:30.158990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:24:30.159692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:30.159692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:24:30.159692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:30.159692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:24:30.159692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:24:30.159692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:24:30.159692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:24:30.159692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:24:30.159692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:30.159692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:24:30.159692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:24:30.159692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:24:30.159692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:24:30.159692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:24:30.159692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:30.160506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:30.160506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:24:30.160506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:24:30.160506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:24:30.160506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:24:30.160506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:30.160506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:30.160506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:30.160506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:24:30.160506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:30.160506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:30.160506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:30.160506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:24:30.160506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:30.160506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:24:30.160506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:24:30.160506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:30.160506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:24:30.160506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:30.161446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:24:30.161446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:24:30.161446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:30.161446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:24:30.161446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:30.161446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:24:30.166710 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:24:30.167166 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:24:30.167351 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:24:30.167563 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:24:30.168087 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:30.168400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:30.169243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:24:30.169243 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:30.169243 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:24:30.169243 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:24:30.169243 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:30.169243 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:24:30.169243 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:30.169243 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:24:30.170433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:30.170433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:24:30.170433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:24:30.170433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:24:30.170433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:30.170433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:24:30.170433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:30.170433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:24:30.170433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:30.170433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:24:30.170433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:30.170433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:24:30.170433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:30.170433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:24:30.170433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:24:30.171249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:30.171249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:24:30.171249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:30.171249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:24:30.171249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:24:30.171249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:30.171249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:24:30.171249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:30.171249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:24:30.171249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:30.171249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:24:30.171249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:30.171249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:24:30.171249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:24:30.171982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:24:30.171982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:24:30.171982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:24:30.171982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:24:30.171982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:24:30.171982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:30.171982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:30.171982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:30.171982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:24:30.171982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:30.171982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:30.171982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:30.171982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:24:30.172800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:30.172800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:24:30.172800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:24:30.172800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:30.172800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:24:30.172800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:30.172800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:24:30.172800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:24:30.172800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:30.172800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:24:30.172800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:30.172800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:24:30.173427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:30.173427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:24:30.173427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:24:30.173427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:24:30.173427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:24:30.173427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:24:30.173427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:30.173427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:24:30.173427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:24:30.173427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:24:30.173427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:24:30.173427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:24:30.173427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:30.173427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:30.173427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:24:30.173427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:24:30.174246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:24:30.174246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:24:30.174246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:24:30.174246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:24:30.174246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:24:30.174246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:30.174246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:30.174246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:30.174246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:24:30.174246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:30.174246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:30.174246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:30.174246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:24:30.174246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:30.174246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:24:30.174246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:24:30.174246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:30.174246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:24:30.175269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:30.175269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:24:30.175269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:24:30.175269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:30.175269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:24:30.175269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:30.175269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:24:30.191790 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:24:30.192055 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:24:30.192233 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:24:30.192424 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:24:30.192963 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:30.193383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:30.194166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:24:30.194166 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:30.194166 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:24:30.194166 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:24:30.194166 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:30.194166 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:24:30.194166 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:30.194166 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:24:30.194947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:30.194947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:24:30.194947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:24:30.194947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:24:30.194947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:30.194947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:24:30.194947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:30.194947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:24:30.194947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:30.194947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:24:30.194947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:30.194947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:24:30.194947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:30.194947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:24:30.194947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:24:30.195614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:30.195614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:24:30.195614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:30.195614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:24:30.195614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:24:30.195614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:30.195614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:24:30.195614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:30.195614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:24:30.196034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:30.196034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:24:30.196034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:30.196034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:24:30.196034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:24:30.196034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:24:30.196034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:24:30.196034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:24:30.196034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:24:30.196034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:30.196034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:30.196541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:30.196541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:24:30.196541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:30.196541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:30.196541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:30.196541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:24:30.196541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:30.196541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:24:30.196541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:24:30.196541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:30.196541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:24:30.196541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:30.196541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:24:30.196541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:24:30.197166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:30.197166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:24:30.197166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:30.197166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:24:30.197166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:30.197166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:24:30.197166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:24:30.197166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:24:30.197166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:24:30.197166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:24:30.197166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:30.197166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:24:30.197166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:24:30.197166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:24:30.197784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:24:30.197784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:24:30.197784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:30.197784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:30.197784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:24:30.197784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:24:30.197784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:24:30.197784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:24:30.197784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:24:30.197784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:24:30.197784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:30.197784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:30.197784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:30.197784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:24:30.197784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:30.197784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:30.197784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:30.197784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:24:30.197784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:30.198590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:24:30.198590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:24:30.198590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:30.198590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:24:30.198590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:30.198590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:24:30.198590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:24:30.198590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:30.198590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:24:30.198590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:30.198590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:24:30.201713 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:24:30.202090 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:24:30.202259 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:24:30.202520 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:24:30.202994 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:30.203347 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:30.203672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:30.204514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:24:30.204514 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:30.204514 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:24:30.204514 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:24:30.204514 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:30.204514 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:24:30.204514 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:30.204514 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:24:30.205477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:30.205477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:24:30.205477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:30.205477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:24:30.205477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:24:30.205477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:24:30.205477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:24:30.205477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:30.205477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:30.205477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:30.205477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:24:30.205477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:30.205801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:30.205801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:30.205801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:24:30.205801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:30.205801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:24:30.205801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:24:30.205801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:30.205801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:24:30.205801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:30.205801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:24:30.205801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:24:30.205801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:30.205801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:24:30.206238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:30.206238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:24:30.206389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:30.206389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:24:30.206389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:24:30.206389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:24:30.206389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:24:30.206389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:24:30.206389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:30.206389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:24:30.206389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:24:30.206389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:24:30.206389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:24:30.206389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:24:30.206389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:30.206389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:30.206389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:24:30.206389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:24:30.206824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:24:30.206824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:24:30.206824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:30.206824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:30.206824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:30.206824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:24:30.206824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:30.206824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:30.206824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:30.206824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:24:30.206824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:30.206824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:24:30.206824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:24:30.206824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:30.206824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:24:30.206824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:30.206824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:24:30.206824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:24:30.206824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:30.207375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:24:30.207375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:30.207375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:24:34.271289 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:24:34.272142 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_for_inconsistencies {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:24:34.272142 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Checking Neutron and OVN revision consistency. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:24:34.282266 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Synchronizing Neutron and OVN databases started {{(pid=105581) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:440}}
May 09 14:24:34.282425 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Number of inconsistencies found at create/update: networks=4, routers=1, router_ports=1 {{(pid=105581) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:419}}
May 09 14:24:34.282546 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Fixing resource 0b4a5724-2518-4e33-a6d3-0d92c749f303 (type: networks) at create/update {{(pid=105581) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 14:24:34.314351 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-0b4a5724-2518-4e33-a6d3-0d92c749f303, resource={'id': '0b4a5724-2518-4e33-a6d3-0d92c749f303', 'name': 'private', 'tenant_id': '5796380176014a32af8ff613b146faed', 'project_id': '5796380176014a32af8ff613b146faed', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['23eb2e2e-ec2f-4f98-bc47-7125a2f92952', 'db2636e9-0880-488d-9aa3-e100ce5eeb9f'], 'standard_attr_id': 12, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-09T14:04:20Z', 'updated_at': '2026-05-09T14:04:23Z', 'revision_number': 3, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 52278}, resource_type=networks, if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:34.314829 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-0b4a5724-2518-4e33-a6d3-0d92c749f303, col_values=(('external_ids', {'neutron:network_name': 'private', 'neutron:mtu': '1372', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:34.315806 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-f71b8cb1-fe70-438c-84f9-bbb7008a596d, external_ids_update=None, columns={'tag': 52278}, if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:34.316023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:34.317075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:24:34.317075 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:34.317075 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:24:34.317075 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:24:34.317075 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:34.317075 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:24:34.317075 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:34.317075 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:24:34.318284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Failed to fix resource 0b4a5724-2518-4e33-a6d3-0d92c749f303 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:34.318284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:24:34.318284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 14:24:34.318284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 14:24:34.318284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 14:24:34.318284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 14:24:34.318284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 09 14:24:34.318284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:24:34.318284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:34.318284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:34.318284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:34.318668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:24:34.318668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:34.318668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:34.318668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:34.318668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:24:34.318668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:34.318668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:24:34.318668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:24:34.318668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:34.318668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:24:34.318668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:34.318668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:24:34.318668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:24:34.318668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:34.319030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:24:34.319030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:34.319030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:24:34.319030 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Fixing resource a7d094cd-23f3-4e0c-9e88-24be6ee17b2c (type: networks) at create/update {{(pid=105581) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 14:24:34.347708 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-a7d094cd-23f3-4e0c-9e88-24be6ee17b2c, resource={'id': 'a7d094cd-23f3-4e0c-9e88-24be6ee17b2c', 'name': 'public', 'tenant_id': '5796380176014a32af8ff613b146faed', 'project_id': '5796380176014a32af8ff613b146faed', 'admin_state_up': True, 'mtu': 1430, 'status': 'ACTIVE', 'subnets': ['2f0821e3-c6a2-49d6-80ee-3480a17ba8d2', '644269b7-6b84-4d8a-ad28-d59a7c6b7c5e'], 'standard_attr_id': 18, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': True, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'is_default': True, 'tags': [], 'created_at': '2026-05-09T14:04:26Z', 'updated_at': '2026-05-09T14:04:34Z', 'revision_number': 3, 'provider:network_type': 'flat', 'provider:physical_network': 'public', 'provider:segmentation_id': None}, resource_type=networks, if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:34.348152 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-a7d094cd-23f3-4e0c-9e88-24be6ee17b2c, col_values=(('external_ids', {'neutron:network_name': 'public', 'neutron:mtu': '1430', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'flat', 'neutron:provnet-physical-network': 'public'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false', 'fdb_age_threshold': '300', 'broadcast-arps-to-all-routers': 'true'})), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:34.348711 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-08fea115-9b05-4667-a0d0-487c303084a5, external_ids_update=None, columns={'tag': []}, if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:34.349285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:34.349990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:24:34.349990 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:34.349990 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:24:34.349990 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:24:34.349990 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:34.349990 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:24:34.349990 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:34.349990 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:24:34.351534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Failed to fix resource a7d094cd-23f3-4e0c-9e88-24be6ee17b2c (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:34.351534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:24:34.351534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 14:24:34.351534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 14:24:34.351534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 14:24:34.351534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 14:24:34.351534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 09 14:24:34.351534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:24:34.351534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:34.351534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:34.351534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:34.352276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:24:34.352276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:34.352276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:34.352276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:34.352276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:24:34.352276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:34.352276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:24:34.352276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:24:34.352276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:34.352276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:24:34.352276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:34.352276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:24:34.352276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:24:34.352276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:34.352813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:24:34.352813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:34.352813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:24:34.352813 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Fixing resource 3e80d551-6de5-4370-806b-36d1db5f6fda (type: networks) at create/update {{(pid=105581) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 14:24:34.380684 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-3e80d551-6de5-4370-806b-36d1db5f6fda, resource={'id': '3e80d551-6de5-4370-806b-36d1db5f6fda', 'name': 'admin_net', 'tenant_id': '590c644b03ae4bed862ecf492c6b2eb2', 'project_id': '590c644b03ae4bed862ecf492c6b2eb2', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['453849e5-22a7-43ca-911e-9401ed2b0d74'], 'standard_attr_id': 42, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-09T14:07:41Z', 'updated_at': '2026-05-09T14:07:42Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 35952}, resource_type=networks, if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:34.381053 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-3e80d551-6de5-4370-806b-36d1db5f6fda, col_values=(('external_ids', {'neutron:network_name': 'admin_net', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:34.381620 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-f84050fb-ff65-46ce-b744-016e1ff15440, external_ids_update=None, columns={'tag': 35952}, if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:34.381782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:34.382499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:24:34.382499 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:34.382499 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:24:34.382499 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:24:34.382499 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:34.382499 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:24:34.382499 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:34.382499 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:24:34.383298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Failed to fix resource 3e80d551-6de5-4370-806b-36d1db5f6fda (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:34.383298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:24:34.383298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 14:24:34.383298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 14:24:34.383298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 14:24:34.383298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 14:24:34.383298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 09 14:24:34.383298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:24:34.383298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:34.383298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:34.383298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:34.384253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:24:34.384253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:34.384253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:34.384253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:34.384253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:24:34.384253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:34.384253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:24:34.384253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:24:34.384253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:34.384253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:24:34.384253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:34.384253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:24:34.384253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:24:34.384253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:34.385119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:24:34.385119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:34.385119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:24:34.385119 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Fixing resource 632a2ddb-5634-4bb3-aad0-f05d97abd45e (type: networks) at create/update {{(pid=105581) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 14:24:34.409932 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-632a2ddb-5634-4bb3-aad0-f05d97abd45e, resource={'id': '632a2ddb-5634-4bb3-aad0-f05d97abd45e', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-5urlz', 'tenant_id': '1f17f4a2a7ac4543a1253522e61993d2', 'project_id': '1f17f4a2a7ac4543a1253522e61993d2', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['32b523fe-135f-4f9e-a98b-fc0537fcdb35'], 'standard_attr_id': 52, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-09T14:08:15Z', 'updated_at': '2026-05-09T14:08:16Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 39245}, resource_type=networks, if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:34.410197 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-632a2ddb-5634-4bb3-aad0-f05d97abd45e, col_values=(('external_ids', {'neutron:network_name': 'k8s-clusterapi-cluster-magnum-system-kube-5urlz', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:34.410799 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-bca44f95-fe96-432e-84b8-1edc05e10ce0, external_ids_update=None, columns={'tag': 39245}, if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:34.410921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:34.411699 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:24:34.411699 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:34.411699 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:24:34.411699 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:24:34.411699 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:34.411699 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:24:34.411699 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:34.411699 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:24:34.412470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Failed to fix resource 632a2ddb-5634-4bb3-aad0-f05d97abd45e (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:34.412470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:24:34.412470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 14:24:34.412470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 14:24:34.412470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 14:24:34.412470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 14:24:34.412470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 09 14:24:34.412470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:24:34.412470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:34.412470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:34.412470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:34.413021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:24:34.413021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:34.413021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:34.413021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:34.413021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:24:34.413021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:34.413021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:24:34.413021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:24:34.413021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:34.413021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:24:34.413021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:34.413021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:24:34.413021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:24:34.413021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:34.413710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:24:34.413710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:34.413710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:24:34.413710 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Fixing resource 09e042cf-39a9-42c8-b683-b29ffc42b049 (type: routers) at create/update {{(pid=105581) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 14:24:34.476511 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, resource={'id': '09e042cf-39a9-42c8-b683-b29ffc42b049', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-5urlz', 'project_id': '1f17f4a2a7ac4543a1253522e61993d2', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': 'a7d094cd-23f3-4e0c-9e88-24be6ee17b2c', 'external_fixed_ips': [{'subnet_id': '2f0821e3-c6a2-49d6-80ee-3480a17ba8d2', 'ip_address': '172.24.5.175'}, {'subnet_id': '644269b7-6b84-4d8a-ad28-d59a7c6b7c5e', 'ip_address': '2001:db8::d8'}], 'enable_snat': True}, 'gw_port_id': '71ded037-e812-40c0-8d8f-70c141c74323', 'description': 'Created by cluster-api-provider-openstack cluster magnum-system-kube-5urlz', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': 'a7d094cd-23f3-4e0c-9e88-24be6ee17b2c', 'external_fixed_ips': [{'ip_address': '172.24.5.175', 'subnet_id': '2f0821e3-c6a2-49d6-80ee-3480a17ba8d2'}, {'ip_address': '2001:db8::d8', 'subnet_id': '644269b7-6b84-4d8a-ad28-d59a7c6b7c5e'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2026-05-09T14:08:17Z', 'updated_at': '2026-05-09T14:08:19Z', 'revision_number': 3, 'tenant_id': '1f17f4a2a7ac4543a1253522e61993d2'}, resource_type=routers, if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:34.478117 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, columns={'external_ids': {'neutron:router_name': 'k8s-clusterapi-cluster-magnum-system-kube-5urlz', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:34.478117 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-a7d094cd-23f3-4e0c-9e88-24be6ee17b2c, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "71ded037-e812-40c0-8d8f-70c141c74323"')], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:34.478117 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-a7d094cd-23f3-4e0c-9e88-24be6ee17b2c, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "71ded037-e812-40c0-8d8f-70c141c74323"')], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:34.478117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:34.478808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:24:34.478808 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:34.478808 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:24:34.478808 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:24:34.478808 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:34.478808 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:24:34.478808 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:34.478808 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:24:34.478808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Unable to update router 09e042cf-39a9-42c8-b683-b29ffc42b049. Error: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:34.480074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Failed to fix resource 09e042cf-39a9-42c8-b683-b29ffc42b049 (type: routers): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:34.480074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:24:34.480074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 14:24:34.480074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 14:24:34.480074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 14:24:34.480074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 14:24:34.480074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1635, in update_router
May 09 14:24:34.480074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with excutils.save_and_reraise_exception():
May 09 14:24:34.480074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 271, in __exit__
May 09 14:24:34.480074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.force_reraise()
May 09 14:24:34.480074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 233, in force_reraise
May 09 14:24:34.480870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise self.value
May 09 14:24:34.480870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1569, in update_router
May 09 14:24:34.480870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:24:34.480870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:34.480870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:34.480870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:34.480870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:24:34.480870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:34.480870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:34.480870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:34.480870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:24:34.480870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:34.480870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:24:34.480870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:24:34.482018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:34.482018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:24:34.482018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:34.482018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:24:34.482018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:24:34.482018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:34.482018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:24:34.482018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:34.482018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:24:34.482018 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Fixing resource 71ded037-e812-40c0-8d8f-70c141c74323 (type: router_ports) at create/update {{(pid=105581) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 14:24:34.579678 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-71ded037-e812-40c0-8d8f-70c141c74323, resource={'id': '71ded037-e812-40c0-8d8f-70c141c74323', 'name': '', 'network_id': 'a7d094cd-23f3-4e0c-9e88-24be6ee17b2c', 'project_id': '', 'mac_address': 'fa:16:3e:c7:86:37', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '09e042cf-39a9-42c8-b683-b29ffc42b049', 'device_owner': 'network:router_gateway', 'standard_attr_id': 57, 'fixed_ips': [{'subnet_id': '2f0821e3-c6a2-49d6-80ee-3480a17ba8d2', 'ip_address': '172.24.5.175'}, {'subnet_id': '644269b7-6b84-4d8a-ad28-d59a7c6b7c5e', 'ip_address': '2001:db8::d8'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0000173183', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'ovs_create_tap': False, 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2026-05-09T14:08:17Z', 'updated_at': '2026-05-09T14:08:19Z', 'revision_number': 5, 'tenant_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:34.580021 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-71ded037-e812-40c0-8d8f-70c141c74323, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': '2f0821e3-c6a2-49d6-80ee-3480a17ba8d2 644269b7-6b84-4d8a-ad28-d59a7c6b7c5e', 'neutron:network_name': 'neutron-a7d094cd-23f3-4e0c-9e88-24be6ee17b2c', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-09e042cf-39a9-42c8-b683-b29ffc42b049'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['2001:db8::d8/64', '172.24.5.175/24'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:34.580628 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=71ded037-e812-40c0-8d8f-70c141c74323, lrouter_port=lrp-71ded037-e812-40c0-8d8f-70c141c74323, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:34.581050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:34.582030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:24:34.582030 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:34.582030 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:24:34.582030 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:24:34.582030 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:34.582030 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:24:34.582030 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:34.582030 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:24:34.583122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Failed to fix resource 71ded037-e812-40c0-8d8f-70c141c74323 (type: router_ports): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:34.583122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:24:34.583122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 14:24:34.583122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 14:24:34.583122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 14:24:34.583122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 14:24:34.583122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1976, in update_router_port
May 09 14:24:34.583122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:24:34.583122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:34.583122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:34.583122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:34.583872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:24:34.583872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:34.583872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:34.583872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:34.583872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:24:34.583872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:34.583872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:24:34.583872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:24:34.583872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:34.583872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:24:34.583872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:34.583872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:24:34.583872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:24:34.583872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:34.584468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:24:34.584468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:34.584468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:24:34.584468 np0000173183 neutron-ovn-maintenance-worker[105581]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_for_inconsistencies finished in 0.311 seconds
May 09 14:24:34.940014 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:24:34.940302 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:24:34.940548 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:24:34.941697 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:24:34.973119 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:34.973533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:34.974333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:24:34.974333 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:34.974333 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:24:34.974333 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:24:34.974333 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:34.974333 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:24:34.974333 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:34.974333 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:24:34.975293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:34.975293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:24:34.975293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:34.975293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:24:34.975293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:24:34.975293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:24:34.975293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:24:34.975293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:34.975293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:34.975293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:34.975293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:24:34.975293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:34.975824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:34.975824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:34.975824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:24:34.975824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:34.975824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:24:34.975824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:24:34.975824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:34.975824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:24:34.975824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:34.975824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:24:34.975824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:24:34.975824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:34.975824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:24:34.976190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:34.976190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:24:34.976322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:34.976322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:24:34.976322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:24:34.976322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:24:34.976322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:24:34.976322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:24:34.976322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:34.976322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:24:34.976322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:24:34.976322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:24:34.976322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:24:34.976322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:24:34.976322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:34.976322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:34.976322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:24:34.976322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:24:34.977030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:24:34.977030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:24:34.977030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:34.977030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:34.977030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:34.977030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:24:34.977030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:34.977030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:34.977030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:34.977030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:24:34.977030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:34.977030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:24:34.977030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:24:34.977030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:34.977030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:24:34.977030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:34.977030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:24:34.977030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:24:34.977030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:34.977759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:24:34.977759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:34.977759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:24:35.154447 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:24:35.155157 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:24:35.155157 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:24:35.156426 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:35.156775 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:35.157017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:35.157877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:24:35.157877 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:35.157877 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:24:35.157877 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:24:35.157877 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:35.157877 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:24:35.157877 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:35.157877 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:24:35.159356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:35.159356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:24:35.159356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:35.159356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:24:35.159356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:24:35.159356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:24:35.159356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:24:35.159356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:35.159356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:35.159356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:35.159356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:24:35.159356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:35.160201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:35.160201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:35.160201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:24:35.160201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:35.160201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:24:35.160201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:24:35.160201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:35.160201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:24:35.160201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:35.160201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:24:35.160201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:24:35.160201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:35.160201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:24:35.160807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:35.160807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:24:35.161250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:35.161250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:24:35.161250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:24:35.161250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:24:35.161250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:24:35.161250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:24:35.161250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:35.161250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:24:35.161250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:24:35.161250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:24:35.161250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:24:35.161250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:24:35.161250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:35.161250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:35.161250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:24:35.161250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:24:35.162042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:24:35.162042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:24:35.162042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:35.162042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:35.162042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:35.162042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:24:35.162042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:35.162042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:35.162042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:35.162042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:24:35.162042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:35.162042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:24:35.162042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:24:35.162042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:35.162042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:24:35.162042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:35.162042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:24:35.162042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:24:35.162042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:35.162890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:24:35.162890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:35.162890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:24:35.167249 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:24:35.167675 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:24:35.167864 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:24:35.168057 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:24:35.168420 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:35.168624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:35.169090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:24:35.169090 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:35.169090 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:24:35.169090 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:24:35.169090 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:35.169090 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:24:35.169090 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:35.169090 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:24:35.169832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:35.169832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:24:35.169832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:24:35.169832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:24:35.169832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:35.169832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:24:35.169832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:35.169832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:24:35.169832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:35.169832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:24:35.169832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:35.169832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:24:35.169832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:35.169832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:24:35.169832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:24:35.170302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:35.170302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:24:35.170302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:35.170302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:24:35.170302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:24:35.170302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:35.170302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:24:35.170302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:35.170302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:24:35.170664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:35.170664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:24:35.170664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:35.170664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:24:35.170664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:24:35.170664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:24:35.170664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:24:35.170664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:24:35.170664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:24:35.170664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:24:35.170664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:35.170664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:35.171023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:35.171023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:24:35.171023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:35.171023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:35.171023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:35.171023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:24:35.171023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:35.171023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:24:35.171023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:24:35.171023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:35.171023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:24:35.171023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:35.171023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:24:35.171023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:24:35.171421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:35.171421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:24:35.171421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:35.171421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:24:35.171698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:35.171698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:24:35.171698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:24:35.171698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:24:35.171698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:24:35.171698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:24:35.171698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:35.171698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:24:35.171698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:24:35.171698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:24:35.171698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:24:35.171698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:24:35.171698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:35.171698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:35.171698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:24:35.171698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:24:35.172151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:24:35.172151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:24:35.172151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:24:35.172151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:24:35.172151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:24:35.172151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:35.172151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:35.172151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:35.172151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:24:35.172151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:35.172151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:35.172151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:35.172151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:24:35.172151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:35.172151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:24:35.172151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:24:35.172151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:35.172151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:24:35.172677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:35.172677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:24:35.172677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:24:35.172677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:35.172677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:24:35.172677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:35.172677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:24:35.192254 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:24:35.192502 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:24:35.192585 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:24:35.192737 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:24:35.193134 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:35.193505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:35.194168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:24:35.194168 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:35.194168 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:24:35.194168 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:24:35.194168 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:35.194168 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:24:35.194168 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:35.194168 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:24:35.195058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:35.195058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:24:35.195058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:24:35.195058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:24:35.195058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:35.195058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:24:35.195058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:35.195058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:24:35.195058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:35.195058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:24:35.195058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:35.195058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:24:35.195058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:35.195058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:24:35.195058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:24:35.195815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:35.195815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:24:35.195815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:35.195815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:24:35.195815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:24:35.195815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:35.195815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:24:35.195815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:35.195815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:24:35.196264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:35.196264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:24:35.196264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:35.196264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:24:35.196264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:24:35.196264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:24:35.196264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:24:35.196264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:24:35.196264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:24:35.196264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:35.196264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:35.196819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:35.196819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:24:35.196819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:35.196819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:35.196819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:35.196819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:24:35.196819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:35.196819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:24:35.196819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:24:35.196819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:35.196819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:24:35.196819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:35.196819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:24:35.196819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:24:35.197522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:35.197522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:24:35.197522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:35.197522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:24:35.197522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:35.197522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:24:35.197522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:24:35.197522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:24:35.197522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:24:35.197522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:24:35.197522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:35.197522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:24:35.197522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:24:35.197522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:24:35.198202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:24:35.198202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:24:35.198202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:35.198202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:35.198202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:24:35.198202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:24:35.198202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:24:35.198202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:24:35.198202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:24:35.198202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:24:35.198202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:35.198202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:35.198202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:35.198202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:24:35.198202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:35.198202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:35.198202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:35.198202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:24:35.198202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:35.199096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:24:35.199096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:24:35.199096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:35.199096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:24:35.199096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:35.199096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:24:35.199096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:24:35.199096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:35.199096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:24:35.199096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:35.199096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:24:35.202054 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:24:35.202281 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:24:35.202425 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:24:35.202580 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:24:35.202982 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:35.203312 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:35.203553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:35.204212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:24:35.204212 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:35.204212 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:24:35.204212 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:24:35.204212 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:35.204212 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:24:35.204212 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:35.204212 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:24:35.205046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:35.205046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:24:35.205046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:35.205046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:24:35.205046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:24:35.205046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:24:35.205046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:24:35.205046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:35.205046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:35.205046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:35.205046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:24:35.205046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:35.205440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:35.205440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:35.205440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:24:35.205440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:35.205440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:24:35.205440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:24:35.205440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:35.205440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:24:35.205440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:35.205440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:24:35.205440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:24:35.205440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:35.205440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:24:35.205858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:35.205858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:24:35.206026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:35.206026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:24:35.206026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:24:35.206026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:24:35.206026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:24:35.206026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:24:35.206026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:35.206026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:24:35.206026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:24:35.206026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:24:35.206026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:24:35.206026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:24:35.206026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:35.206026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:35.206026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:24:35.206026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:24:35.206504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:24:35.206504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:24:35.206504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:35.206504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:35.206504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:35.206504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:24:35.206504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:35.206504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:35.206504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:35.206504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:24:35.206504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:35.206504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:24:35.206504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:24:35.206504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:35.206504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:24:35.206504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:35.206504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:24:35.206504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:24:35.206504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:35.207023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:24:35.207023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:35.207023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:24:39.940707 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:24:39.941869 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:24:39.942042 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:24:39.943280 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:24:39.973881 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:39.974202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:39.974998 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:24:39.974998 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:39.974998 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:24:39.974998 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:24:39.974998 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:39.974998 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:24:39.974998 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:39.974998 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:24:39.976125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:39.976125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:24:39.976125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:39.976125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:24:39.976125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:24:39.976125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:24:39.976125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:24:39.976125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:39.976125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:39.976125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:39.976125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:24:39.976125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:39.976876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:39.976876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:39.976876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:24:39.976876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:39.976876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:24:39.976876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:24:39.976876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:39.976876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:24:39.976876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:39.976876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:24:39.976876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:24:39.976876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:39.976876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:24:39.977572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:39.977572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:24:39.978028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:39.978028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:24:39.978028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:24:39.978028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:24:39.978028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:24:39.978028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:24:39.978028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:39.978028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:24:39.978028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:24:39.978028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:24:39.978028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:24:39.978028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:24:39.978028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:39.978028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:39.978028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:24:39.978028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:24:39.978731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:24:39.978731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:24:39.978731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:39.978731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:39.978731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:39.978731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:24:39.978731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:39.978731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:39.978731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:39.978731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:24:39.978731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:39.978731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:24:39.978731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:24:39.978731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:39.978731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:24:39.978731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:39.978731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:24:39.978731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:24:39.978731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:39.979224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:24:39.979224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:39.979224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:24:40.155107 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:24:40.155693 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:24:40.155892 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:24:40.156863 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:40.157306 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:40.157428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:40.158379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:24:40.158379 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:40.158379 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:24:40.158379 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:24:40.158379 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:40.158379 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:24:40.158379 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:40.158379 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:24:40.159789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:40.159789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:24:40.159789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:40.159789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:24:40.159789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:24:40.159789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:24:40.159789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:24:40.159789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:40.159789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:40.159789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:40.159789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:24:40.159789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:40.160338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:40.160338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:40.160338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:24:40.160338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:40.160338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:24:40.160338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:24:40.160338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:40.160338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:24:40.160338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:40.160338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:24:40.160338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:24:40.160338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:40.160338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:24:40.161154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:40.161154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:24:40.161358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:40.161358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:24:40.161358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:24:40.161358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:24:40.161358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:24:40.161358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:24:40.161358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:40.161358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:24:40.161358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:24:40.161358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:24:40.161358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:24:40.161358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:24:40.161358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:40.161358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:40.161358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:24:40.161358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:24:40.162062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:24:40.162062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:24:40.162062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:40.162062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:40.162062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:40.162062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:24:40.162062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:40.162062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:40.162062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:40.162062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:24:40.162062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:40.162062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:24:40.162062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:24:40.162062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:40.162062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:24:40.162062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:40.162062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:24:40.162062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:24:40.162062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:40.163040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:24:40.163040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:40.163040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:24:40.167719 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:24:40.168079 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:24:40.168243 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:24:40.168428 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:24:40.168932 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:40.169228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:40.170191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:24:40.170191 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:40.170191 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:24:40.170191 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:24:40.170191 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:40.170191 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:24:40.170191 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:40.170191 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:24:40.170900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:40.170900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:24:40.170900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:24:40.170900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:24:40.170900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:40.170900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:24:40.170900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:40.170900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:24:40.170900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:40.170900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:24:40.170900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:40.170900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:24:40.170900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:40.170900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:24:40.170900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:24:40.171320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:40.171320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:24:40.171320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:40.171320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:24:40.171320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:24:40.171320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:40.171320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:24:40.171320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:40.171320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:24:40.171684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:40.171684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:24:40.171684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:40.171684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:24:40.171684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:24:40.171684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:24:40.171684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:24:40.171684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:24:40.171684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:24:40.171684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:24:40.171684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:40.171684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:40.172079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:40.172079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:24:40.172079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:40.172079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:40.172079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:40.172079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:24:40.172079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:40.172079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:24:40.172079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:24:40.172079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:40.172079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:24:40.172079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:40.172079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:24:40.172079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:24:40.172465 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:40.172465 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:24:40.172465 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:40.172465 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:24:40.172627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:40.172627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:24:40.172627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:24:40.172627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:24:40.172627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:24:40.172627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:24:40.172627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:40.172627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:24:40.172627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:24:40.172627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:24:40.172627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:24:40.172627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:24:40.172627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:40.172627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:40.172627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:24:40.172627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:24:40.173048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:24:40.173048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:24:40.173048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:24:40.173048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:24:40.173048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:24:40.173048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:40.173048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:40.173048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:40.173048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:24:40.173048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:40.173048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:40.173048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:40.173048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:24:40.173048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:40.173048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:24:40.173048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:24:40.173048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:40.173048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:24:40.173566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:40.173566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:24:40.173566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:24:40.173566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:40.173566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:24:40.173566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:40.173566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:24:40.192749 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:24:40.193028 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:24:40.193196 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:24:40.193387 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:24:40.193986 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:40.194195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:40.194906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:24:40.194906 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:40.194906 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:24:40.194906 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:24:40.194906 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:40.194906 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:24:40.194906 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:40.194906 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:24:40.196002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:40.196002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:24:40.196002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:24:40.196002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:24:40.196002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:40.196002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:24:40.196002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:40.196002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:24:40.196002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:40.196002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:24:40.196002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:40.196002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:24:40.196002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:40.196002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:24:40.196002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:24:40.196693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:40.196693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:24:40.196693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:40.196693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:24:40.196693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:24:40.196693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:40.196693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:24:40.196693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:40.196693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:24:40.197431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:40.197431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:24:40.197431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:40.197431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:24:40.197431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:24:40.197431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:24:40.197431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:24:40.197431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:24:40.197431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:24:40.197431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:40.197431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:40.197991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:40.197991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:24:40.197991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:40.197991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:40.197991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:40.197991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:24:40.197991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:40.197991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:24:40.197991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:24:40.197991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:40.197991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:24:40.197991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:40.197991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:24:40.197991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:24:40.198624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:40.198624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:24:40.198624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:40.198624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:24:40.198985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:40.198985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:24:40.198985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:24:40.198985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:24:40.198985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:24:40.198985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:24:40.198985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:40.198985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:24:40.198985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:24:40.198985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:24:40.198985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:24:40.198985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:24:40.198985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:40.198985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:40.198985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:24:40.198985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:24:40.198985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:24:40.199747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:24:40.199747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:24:40.199747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:24:40.199747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:40.199747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:40.199747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:40.199747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:24:40.199747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:40.199747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:40.199747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:40.199747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:24:40.199747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:40.199747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:24:40.199747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:24:40.199747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:40.199747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:24:40.199747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:40.199747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:24:40.199747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:24:40.200582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:40.200582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:24:40.200582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:40.200582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:24:40.202543 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:24:40.202786 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:24:40.202880 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:24:40.203028 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:24:40.203517 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:40.203810 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:40.204031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:40.204737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:24:40.204737 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:40.204737 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:24:40.204737 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:24:40.204737 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:40.204737 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:24:40.204737 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:40.204737 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:24:40.205503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:40.205503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:24:40.205503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:40.205503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:24:40.205503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:24:40.205503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:24:40.205503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:24:40.205503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:40.205503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:40.205503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:40.205503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:24:40.205503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:40.205837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:40.205837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:40.205837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:24:40.205837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:40.205837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:24:40.205837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:24:40.205837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:40.205837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:24:40.205837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:40.205837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:24:40.205837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:24:40.205837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:40.205837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:24:40.206190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:40.206190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:24:40.206455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:40.206455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:24:40.206455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:24:40.206455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:24:40.206455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:24:40.206455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:24:40.206455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:40.206455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:24:40.206455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:24:40.206455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:24:40.206455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:24:40.206455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:24:40.206455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:40.206455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:40.206455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:24:40.206455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:24:40.206882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:24:40.206882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:24:40.206882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:40.206882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:40.206882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:40.206882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:24:40.206882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:40.206882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:40.206882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:40.206882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:24:40.206882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:40.206882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:24:40.206882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:24:40.206882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:40.206882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:24:40.206882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:40.206882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:24:40.206882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:24:40.206882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:40.207419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:24:40.207419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:40.207419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:24:44.941780 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:24:44.942797 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:24:44.942797 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:24:44.943113 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:24:44.977654 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:44.977976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:44.978810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:24:44.978810 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:44.978810 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:24:44.978810 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:24:44.978810 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:44.978810 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:24:44.978810 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:44.978810 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:24:44.979810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:44.979810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:24:44.979810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:44.979810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:24:44.979810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:24:44.979810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:24:44.979810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:24:44.979810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:44.979810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:44.979810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:44.979810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:24:44.979810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:44.980481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:44.980481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:44.980481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:24:44.980481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:44.980481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:24:44.980481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:24:44.980481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:44.980481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:24:44.980481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:44.980481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:24:44.980481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:24:44.980481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:44.980481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:24:44.981119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:44.981119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:24:44.981119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:44.981119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:24:44.981119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:24:44.981119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:24:44.981119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:24:44.981119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:24:44.981119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:44.981119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:24:44.981119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:24:44.981119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:24:44.981119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:24:44.981119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:24:44.981119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:44.981843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:44.981843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:24:44.981843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:24:44.981843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:24:44.981843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:24:44.981843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:44.981843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:44.981843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:44.981843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:24:44.981843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:44.981843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:44.981843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:44.981843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:24:44.981843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:44.981843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:24:44.981843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:24:44.981843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:44.981843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:24:44.981843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:44.982703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:24:44.982703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:24:44.982703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:44.982703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:24:44.982703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:44.982703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:24:45.156048 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:24:45.156364 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:24:45.156440 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:24:45.157868 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:45.158205 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:45.158737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:45.159101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:24:45.159101 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:45.159101 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:24:45.159101 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:24:45.159101 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:45.159101 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:24:45.159101 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:45.159101 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:24:45.161036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:45.161036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:24:45.161036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:45.161036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:24:45.161036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:24:45.161036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:24:45.161036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:24:45.161036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:45.161036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:45.161036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:45.161036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:24:45.161036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:45.161453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:45.161453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:45.161453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:24:45.161453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:45.161453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:24:45.161453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:24:45.161453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:45.161453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:24:45.161453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:45.161453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:24:45.161453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:24:45.161453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:45.161453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:24:45.161840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:45.161840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:24:45.162539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:45.162539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:24:45.162539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:24:45.162539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:24:45.162539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:24:45.162539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:24:45.162539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:45.162539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:24:45.162539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:24:45.162539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:24:45.162539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:24:45.162539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:24:45.162539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:45.162539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:45.162539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:24:45.162539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:24:45.163032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:24:45.163032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:24:45.163032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:45.163032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:45.163032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:45.163032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:24:45.163032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:45.163032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:45.163032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:45.163032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:24:45.163032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:45.163032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:24:45.163032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:24:45.163032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:45.163032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:24:45.163032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:45.163032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:24:45.163032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:24:45.163032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:45.163612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:24:45.163612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:45.163612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:24:45.168105 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:24:45.168335 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:24:45.168463 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:24:45.168590 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:24:45.169034 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:45.169238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:45.169817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:24:45.169817 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:45.169817 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:24:45.169817 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:24:45.169817 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:45.169817 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:24:45.169817 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:45.169817 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:24:45.170554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:45.170554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:24:45.170554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:24:45.170554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:24:45.170554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:45.170554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:24:45.170554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:45.170554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:24:45.170554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:45.170554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:24:45.170554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:45.170554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:24:45.170554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:45.170554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:24:45.170554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:24:45.171064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:45.171064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:24:45.171064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:45.171064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:24:45.171064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:24:45.171064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:45.171064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:24:45.171064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:45.171064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:24:45.171352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:45.171352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:24:45.171352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:45.171352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:24:45.171352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:24:45.171352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:24:45.171352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:24:45.171352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:24:45.171352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:24:45.171352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:24:45.171352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:45.171352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:45.171736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:45.171736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:24:45.171736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:45.171736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:45.171736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:45.171736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:24:45.171736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:45.171736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:24:45.171736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:24:45.171736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:45.171736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:24:45.171736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:45.171736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:24:45.171736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:24:45.172110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:45.172110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:24:45.172110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:45.172110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:24:45.172298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:45.172298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:24:45.172298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:24:45.172298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:24:45.172298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:24:45.172298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:24:45.172298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:45.172298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:24:45.172298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:24:45.172298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:24:45.172298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:24:45.172298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:24:45.172298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:45.172298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:45.172298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:24:45.172298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:24:45.172759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:24:45.172759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:24:45.172759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:24:45.172759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:24:45.172759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:24:45.172759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:45.172759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:45.172759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:45.172759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:24:45.172759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:45.172759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:45.172759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:45.172759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:24:45.172759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:45.172759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:24:45.172759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:24:45.172759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:45.172759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:24:45.173292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:45.173292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:24:45.173292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:24:45.173292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:45.173292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:24:45.173292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:45.173292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:24:45.193238 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:24:45.193545 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:24:45.193756 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:24:45.193969 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:24:45.194423 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:45.194622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:45.195215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:24:45.195215 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:45.195215 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:24:45.195215 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:24:45.195215 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:45.195215 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:24:45.195215 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:45.195215 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:24:45.196423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:45.196423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:24:45.196423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:24:45.196423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:24:45.196423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:45.196423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:24:45.196423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:45.196423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:24:45.196423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:45.196423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:24:45.196423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:45.196423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:24:45.196423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:45.196423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:24:45.196423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:24:45.197077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:45.197077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:24:45.197077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:45.197077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:24:45.197077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:24:45.197077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:45.197077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:24:45.197077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:45.197077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:24:45.198012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:45.198012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:24:45.198012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:45.198012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:24:45.198012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:24:45.198012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:24:45.198012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:24:45.198012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:24:45.198012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:24:45.198012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:45.198012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:45.198713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:45.198713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:24:45.198713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:45.198713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:45.198713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:45.198713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:24:45.198713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:45.198713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:24:45.198713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:24:45.198713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:45.198713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:24:45.198713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:45.198713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:24:45.198713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:24:45.199300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:45.199300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:24:45.199300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:45.199300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:24:45.199844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:45.199844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:24:45.199844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:24:45.199844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:24:45.199844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:24:45.199844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:24:45.199844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:45.199844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:24:45.199844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:24:45.199844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:24:45.199844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:24:45.199844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:24:45.199844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:45.199844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:45.199844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:24:45.199844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:24:45.199844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:24:45.200640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:24:45.200640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:24:45.200640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:24:45.200640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:45.200640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:45.200640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:45.200640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:24:45.200640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:45.200640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:45.200640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:45.200640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:24:45.200640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:45.200640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:24:45.200640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:24:45.200640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:45.200640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:24:45.200640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:45.200640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:24:45.200640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:24:45.201455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:45.201455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:24:45.201455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:45.201455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:24:45.202880 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:24:45.203213 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:24:45.203517 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:24:45.203668 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:24:45.204001 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:45.204198 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:45.204396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:45.204929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:24:45.204929 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:45.204929 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:24:45.204929 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:24:45.204929 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:45.204929 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:24:45.204929 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:45.204929 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:24:45.206329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:45.206329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:24:45.206329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:45.206329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:24:45.206329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:24:45.206329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:24:45.206329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:24:45.206329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:45.206329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:45.206329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:45.206329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:24:45.206329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:45.206720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:45.206720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:45.206720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:24:45.206720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:45.206720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:24:45.206720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:24:45.206720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:45.206720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:24:45.206720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:45.206720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:24:45.206720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:24:45.206720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:45.206720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:24:45.207052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:45.207052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:24:45.207979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:45.207979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:24:45.207979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:24:45.207979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:24:45.207979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:24:45.207979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:24:45.207979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:45.207979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:24:45.207979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:24:45.207979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:24:45.207979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:24:45.207979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:24:45.207979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:45.207979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:45.207979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:24:45.207979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:24:45.208441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:24:45.208441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:24:45.208441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:45.208441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:45.208441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:45.208441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:24:45.208441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:45.208441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:45.208441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:45.208441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:24:45.208441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:45.208441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:24:45.208441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:24:45.208441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:45.208441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:24:45.208441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:45.208441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:24:45.208441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:24:45.208441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:45.208984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:24:45.208984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:45.208984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:24:49.942381 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:24:49.942960 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:24:49.943206 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:24:49.944417 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:24:49.979263 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:49.979612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:49.980261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:24:49.980261 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:49.980261 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:24:49.980261 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:24:49.980261 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:49.980261 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:24:49.980261 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:49.980261 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:24:49.981274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:49.981274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:24:49.981274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:49.981274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:24:49.981274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:24:49.981274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:24:49.981274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:24:49.981274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:49.981274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:49.981274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:49.981274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:24:49.981274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:49.981653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:49.981653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:49.981653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:24:49.981653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:49.981653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:24:49.981653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:24:49.981653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:49.981653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:24:49.981653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:49.981653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:24:49.981653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:24:49.981653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:49.981653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:24:49.981983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:49.981983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:24:49.982413 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:49.982413 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:24:49.982413 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:24:49.982413 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:24:49.982413 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:24:49.982413 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:24:49.982413 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:49.982413 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:24:49.982413 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:24:49.982413 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:24:49.982413 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:24:49.982413 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:24:49.982413 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:49.982413 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:49.982413 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:24:49.982413 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:24:49.982828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:24:49.982828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:24:49.982828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:49.982828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:49.982828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:49.982828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:24:49.982828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:49.982828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:49.982828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:49.982828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:24:49.982828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:49.982828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:24:49.982828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:24:49.982828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:49.982828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:24:49.982828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:49.982828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:24:49.982828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:24:49.982828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:49.983292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:24:49.983292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:49.983292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:24:50.156923 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:24:50.156923 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:24:50.157145 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:24:50.158298 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:50.158633 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:50.158633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:50.159528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:24:50.159528 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:50.159528 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:24:50.159528 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:24:50.159528 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:50.159528 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:24:50.159528 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:50.159528 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:24:50.160762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:50.160762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:24:50.160762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:50.160762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:24:50.160762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:24:50.160762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:24:50.160762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:24:50.160762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:50.160762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:50.160762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:50.160762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:24:50.160762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:50.162099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:50.162099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:50.162099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:24:50.162099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:50.162099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:24:50.162099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:24:50.162099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:50.162099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:24:50.162099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:50.162099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:24:50.162099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:24:50.162099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:50.162099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:24:50.162764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:50.162764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:24:50.162764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:50.162764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:24:50.162764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:24:50.162764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:24:50.162764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:24:50.162764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:24:50.162764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:50.162764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:24:50.162764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:24:50.162764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:24:50.162764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:24:50.162764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:24:50.162764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:50.163777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:50.163777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:24:50.163777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:24:50.163777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:24:50.163777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:24:50.163777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:50.163777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:50.163777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:50.163777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:24:50.163777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:50.163777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:50.163777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:50.163777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:24:50.163777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:50.163777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:24:50.163777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:24:50.163777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:50.163777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:24:50.163777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:50.164351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:24:50.164351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:24:50.164351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:50.164351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:24:50.164351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:50.164351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:24:50.168461 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:24:50.168620 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:24:50.168792 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:24:50.168851 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:24:50.169325 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:50.169655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:50.170360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:24:50.170360 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:50.170360 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:24:50.170360 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:24:50.170360 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:50.170360 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:24:50.170360 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:50.170360 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:24:50.171424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:50.171424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:24:50.171424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:24:50.171424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:24:50.171424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:50.171424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:24:50.171424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:50.171424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:24:50.171424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:50.171424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:24:50.171424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:50.171424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:24:50.171424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:50.171424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:24:50.171424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:24:50.171885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:50.171885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:24:50.171885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:50.171885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:24:50.171885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:24:50.171885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:50.171885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:24:50.171885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:50.171885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:24:50.172880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:50.172880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:24:50.172880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:50.172880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:24:50.172880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:24:50.172880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:24:50.172880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:24:50.172880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:24:50.172880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:24:50.172880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:24:50.172880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:50.172880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:50.173641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:50.173641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:24:50.173641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:50.173641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:50.173641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:50.173641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:24:50.173641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:50.173641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:24:50.173641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:24:50.173641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:50.173641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:24:50.173641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:50.173641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:24:50.173641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:24:50.174357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:50.174357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:24:50.174357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:50.174357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:24:50.174976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:50.174976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:24:50.174976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:24:50.174976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:24:50.174976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:24:50.174976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:24:50.174976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:50.174976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:24:50.174976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:24:50.174976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:24:50.174976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:24:50.174976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:24:50.174976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:50.174976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:50.174976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:24:50.174976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:24:50.176034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:24:50.176034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:24:50.176034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:24:50.176034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:24:50.176034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:24:50.176034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:50.176034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:50.176034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:50.176034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:24:50.176034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:50.176034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:50.176034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:50.176034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:24:50.176034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:50.176034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:24:50.176034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:24:50.176034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:50.176034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:24:50.176870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:50.176870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:24:50.176870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:24:50.176870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:50.176870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:24:50.176870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:50.176870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:24:50.193533 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:24:50.193739 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:24:50.193898 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:24:50.194097 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:24:50.194629 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:50.194963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:50.195709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:24:50.195709 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:50.195709 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:24:50.195709 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:24:50.195709 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:50.195709 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:24:50.195709 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:50.195709 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:24:50.196981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:50.196981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:24:50.196981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:24:50.196981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:24:50.196981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:50.196981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:24:50.196981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:50.196981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:24:50.196981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:50.196981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:24:50.196981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:50.196981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:24:50.196981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:50.196981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:24:50.196981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:24:50.197421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:50.197421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:24:50.197421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:50.197421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:24:50.197421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:24:50.197421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:50.197421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:24:50.197421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:50.197421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:24:50.198681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:50.198681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:24:50.198681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:50.198681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:24:50.198681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:24:50.198681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:24:50.198681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:24:50.198681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:24:50.198681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:24:50.198681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:50.198681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:50.199342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:50.199342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:24:50.199342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:50.199342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:50.199342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:50.199342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:24:50.199342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:50.199342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:24:50.199342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:24:50.199342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:50.199342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:24:50.199342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:50.199342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:24:50.199342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:24:50.200245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:50.200245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:24:50.200245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:50.200245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:24:50.200477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:50.200477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:24:50.200477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:24:50.200477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:24:50.200477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:24:50.200477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:24:50.200477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:50.200477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:24:50.200477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:24:50.200477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:24:50.200477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:24:50.200477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:24:50.200477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:50.200477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:50.200477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:24:50.200477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:24:50.200477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:24:50.201428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:24:50.201428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:24:50.201428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:24:50.201428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:50.201428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:50.201428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:50.201428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:24:50.201428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:50.201428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:50.201428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:50.201428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:24:50.201428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:50.201428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:24:50.201428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:24:50.201428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:50.201428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:24:50.201428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:50.201428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:24:50.201428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:24:50.202525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:50.202525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:24:50.202525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:50.202525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:24:50.203134 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:24:50.203347 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:24:50.203499 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:24:50.203660 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:24:50.204122 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:50.204527 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:50.204759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:50.205471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:24:50.205471 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:50.205471 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:24:50.205471 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:24:50.205471 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:50.205471 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:24:50.205471 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:50.205471 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:24:50.206297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:50.206297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:24:50.206297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:50.206297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:24:50.206297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:24:50.206297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:24:50.206297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:24:50.206297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:50.206297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:50.206297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:50.206297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:24:50.206297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:50.206692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:50.206692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:50.206692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:24:50.206692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:50.206692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:24:50.206692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:24:50.206692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:50.206692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:24:50.206692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:50.206692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:24:50.206692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:24:50.206692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:50.206692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:24:50.207044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:50.207044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:24:50.207223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:50.207223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:24:50.207223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:24:50.207223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:24:50.207223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:24:50.207223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:24:50.207223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:50.207223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:24:50.207223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:24:50.207223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:24:50.207223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:24:50.207223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:24:50.207223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:50.207223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:50.207223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:24:50.207223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:24:50.207765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:24:50.207765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:24:50.207765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:50.207765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:50.207765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:50.207765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:24:50.207765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:50.207765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:50.207765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:50.207765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:24:50.207765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:50.207765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:24:50.207765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:24:50.207765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:50.207765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:24:50.207765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:50.207765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:24:50.207765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:24:50.207765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:50.208248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:24:50.208248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:50.208248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:24:54.943124 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:24:54.944045 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:24:54.944131 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:24:54.945435 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:24:54.978039 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:54.978328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:54.978956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:24:54.978956 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:54.978956 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:24:54.978956 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:24:54.978956 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:54.978956 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:24:54.978956 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:54.978956 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:24:54.979695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:54.979695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:24:54.979695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:54.979695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:24:54.979695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:24:54.979695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:24:54.979695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:24:54.979695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:54.979695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:54.979695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:54.979695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:24:54.979695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:54.980369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:54.980369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:54.980369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:24:54.980369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:54.980369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:24:54.980369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:24:54.980369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:54.980369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:24:54.980369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:54.980369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:24:54.980369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:24:54.980369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:54.980369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:24:54.981060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:54.981060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:24:54.981060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:54.981060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:24:54.981060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:24:54.981060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:24:54.981060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:24:54.981060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:24:54.981060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:54.981060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:24:54.981060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:24:54.981060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:24:54.981060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:24:54.981060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:24:54.981060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:54.981854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:54.981854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:24:54.981854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:24:54.981854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:24:54.981854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:24:54.981854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:54.981854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:54.981854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:54.981854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:24:54.981854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:54.981854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:54.981854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:54.981854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:24:54.981854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:54.981854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:24:54.981854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:24:54.981854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:54.981854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:24:54.981854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:54.982830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:24:54.982830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:24:54.982830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:54.982830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:24:54.982830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:54.982830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:24:55.157111 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:24:55.157802 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:24:55.158010 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:24:55.159056 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:55.159376 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:55.159670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:55.160530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:24:55.160530 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:55.160530 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:24:55.160530 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:24:55.160530 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:55.160530 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:24:55.160530 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:55.160530 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:24:55.162254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:55.162254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:24:55.162254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:55.162254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:24:55.162254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:24:55.162254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:24:55.162254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:24:55.162254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:55.162254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:55.162254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:55.162254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:24:55.162254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:55.162895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:55.162895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:55.162895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:24:55.162895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:55.162895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:24:55.162895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:24:55.162895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:55.162895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:24:55.162895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:55.162895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:24:55.162895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:24:55.162895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:55.162895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:24:55.163505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:55.163505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:24:55.164168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:55.164168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:24:55.164168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:24:55.164168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:24:55.164168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:24:55.164168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:24:55.164168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:55.164168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:24:55.164168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:24:55.164168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:24:55.164168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:24:55.164168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:24:55.164168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:55.164168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:55.164168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:24:55.164168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:24:55.164954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:24:55.164954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:24:55.164954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:55.164954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:55.164954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:55.164954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:24:55.164954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:55.164954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:55.164954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:55.164954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:24:55.164954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:55.164954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:24:55.164954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:24:55.164954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:55.164954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:24:55.164954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:55.164954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:24:55.164954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:24:55.164954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:55.165942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:24:55.165942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:55.165942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:24:55.168858 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:24:55.169233 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:24:55.169422 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:24:55.169644 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:24:55.170090 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:55.170360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:55.171044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:24:55.171044 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:55.171044 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:24:55.171044 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:24:55.171044 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:55.171044 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:24:55.171044 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:55.171044 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:24:55.171786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:55.171786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:24:55.171786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:24:55.171786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:24:55.171786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:55.171786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:24:55.171786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:55.171786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:24:55.171786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:55.171786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:24:55.171786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:55.171786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:24:55.171786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:55.171786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:24:55.171786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:24:55.172286 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:55.172286 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:24:55.172286 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:55.172286 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:24:55.172286 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:24:55.172286 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:55.172286 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:24:55.172286 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:55.172286 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:24:55.172587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:55.172587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:24:55.172587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:55.172587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:24:55.172587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:24:55.172587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:24:55.172587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:24:55.172587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:24:55.172587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:24:55.172587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:24:55.172587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:55.172587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:55.172937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:55.172937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:24:55.172937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:55.172937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:55.172937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:55.172937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:24:55.172937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:55.172937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:24:55.172937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:24:55.172937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:55.172937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:24:55.172937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:55.172937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:24:55.172937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:24:55.173345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:55.173345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:24:55.173345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:55.173345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:24:55.173612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:55.173612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:24:55.173612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:24:55.173612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:24:55.173612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:24:55.173612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:24:55.173612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:55.173612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:24:55.173612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:24:55.173612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:24:55.173612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:24:55.173612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:24:55.173612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:55.173612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:55.173612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:24:55.173612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:24:55.174054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:24:55.174054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:24:55.174054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:24:55.174054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:24:55.174054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:24:55.174054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:55.174054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:55.174054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:55.174054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:24:55.174054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:55.174054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:55.174054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:55.174054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:24:55.174054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:55.174054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:24:55.174054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:24:55.174054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:55.174054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:24:55.174527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:55.174527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:24:55.174527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:24:55.174527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:55.174527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:24:55.174527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:55.174527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:24:55.193957 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:24:55.194260 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:24:55.194458 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:24:55.194661 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:24:55.195111 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:55.195471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:55.196195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:24:55.196195 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:55.196195 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:24:55.196195 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:24:55.196195 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:55.196195 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:24:55.196195 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:55.196195 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:24:55.197499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:55.197499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:24:55.197499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:24:55.197499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:24:55.197499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:55.197499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:24:55.197499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:55.197499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:24:55.197499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:55.197499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:24:55.197499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:55.197499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:24:55.197499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:55.197499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:24:55.197499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:24:55.198535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:55.198535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:24:55.198535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:55.198535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:24:55.198535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:24:55.198535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:55.198535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:24:55.198535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:55.198535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:24:55.199060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:55.199060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:24:55.199060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:55.199060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:24:55.199060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:24:55.199060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:24:55.199060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:24:55.199060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:24:55.199060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:24:55.199060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:55.199060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:55.199786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:55.199786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:24:55.199786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:55.199786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:55.199786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:55.199786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:24:55.199786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:55.199786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:24:55.199786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:24:55.199786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:55.199786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:24:55.199786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:55.199786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:24:55.199786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:24:55.201046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:55.201046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:24:55.201046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:55.201046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:24:55.201542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:55.201542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:24:55.201542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:24:55.201542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:24:55.201542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:24:55.201542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:24:55.201542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:55.201542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:24:55.201542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:24:55.201542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:24:55.201542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:24:55.201542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:24:55.201542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:55.201542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:55.201542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:24:55.201542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:24:55.201542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:24:55.202207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:24:55.202207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:24:55.202207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:24:55.202207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:55.202207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:55.202207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:55.202207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:24:55.202207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:55.202207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:55.202207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:55.202207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:24:55.202207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:55.202207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:24:55.202207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:24:55.202207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:55.202207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:24:55.202207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:55.202207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:24:55.202207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:24:55.203049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:55.203049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:24:55.203049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:55.203049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:24:55.203544 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:24:55.203931 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:24:55.204061 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:24:55.204202 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:24:55.204687 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:55.205051 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:55.205261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:55.206496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:24:55.206496 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:55.206496 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:24:55.206496 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:24:55.206496 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:55.206496 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:24:55.206496 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:55.206496 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:24:55.206847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:55.206847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:24:55.206847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:55.206847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:24:55.206847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:24:55.206847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:24:55.206847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:24:55.206847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:55.206847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:55.206847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:55.206847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:24:55.206847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:55.207247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:55.207247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:55.207247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:24:55.207247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:55.207247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:24:55.207247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:24:55.207247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:55.207247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:24:55.207247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:55.207247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:24:55.207247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:24:55.207247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:55.207247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:24:55.207607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:55.207607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:24:55.207909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:55.207909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:24:55.207909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:24:55.207909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:24:55.207909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:24:55.207909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:24:55.207909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:55.207909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:24:55.207909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:24:55.207909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:24:55.207909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:24:55.207909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:24:55.207909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:55.207909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:55.207909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:24:55.207909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:24:55.208369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:24:55.208369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:24:55.208369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:55.208369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:55.208369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:55.208369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:24:55.208369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:55.208369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:55.208369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:55.208369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:24:55.208369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:55.208369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:24:55.208369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:24:55.208369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:55.208369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:24:55.208369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:55.208369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:24:55.208369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:24:55.208369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:55.208914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:24:55.208914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:55.208914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:24:59.944892 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:24:59.945762 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:24:59.945762 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:24:59.946984 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:24:59.986692 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:24:59.986950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:59.987701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:24:59.987701 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:59.987701 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:24:59.987701 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:24:59.987701 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:59.987701 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:24:59.987701 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:59.987701 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:24:59.988506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:59.988506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:24:59.988506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:59.988506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:24:59.988506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:24:59.988506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:24:59.988506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:24:59.988506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:59.988506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:59.988506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:59.988506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:24:59.988506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:59.989336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:24:59.989336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:59.989336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:24:59.989336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:59.989336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:24:59.989336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:24:59.989336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:59.989336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:24:59.989336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:59.989336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:24:59.989336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:24:59.989336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:59.989336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:24:59.990342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:59.990342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:24:59.990342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:59.990342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:24:59.990342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:24:59.990342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:24:59.990342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:24:59.990342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:24:59.990342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:59.990342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:24:59.990342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:24:59.990342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:24:59.990342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:24:59.990342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:24:59.990342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:24:59.991362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:24:59.991362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:24:59.991362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:24:59.991362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:24:59.991362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:24:59.991362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:59.991362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:59.991362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:24:59.991362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:24:59.991362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:24:59.991362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:24:59.991362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:24:59.991362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:24:59.991362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:24:59.991362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:24:59.991362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:24:59.991362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:24:59.991362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:24:59.991362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:24:59.993215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:24:59.993215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:24:59.993215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:24:59.993215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:24:59.993215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:24:59.993215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:25:00.157828 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:25:00.158462 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:25:00.158462 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:25:00.160589 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:25:00.160830 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:25:00.160991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:00.161700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:25:00.161700 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:00.161700 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:25:00.161700 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:25:00.161700 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:00.161700 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:25:00.161700 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:00.161700 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:25:00.163300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:00.163300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:25:00.163300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:00.163300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:25:00.163300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:25:00.163300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:25:00.163300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:25:00.163300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:00.163300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:00.163300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:00.163300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:25:00.163300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:00.163692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:00.163692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:00.163692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:25:00.163692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:00.163692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:25:00.163692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:25:00.163692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:00.163692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:25:00.163692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:00.163692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:25:00.163692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:25:00.163692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:00.163692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:25:00.164048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:00.164048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:25:00.165145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:00.165145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:25:00.165145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:25:00.165145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:25:00.165145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:25:00.165145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:25:00.165145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:00.165145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:25:00.165145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:25:00.165145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:25:00.165145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:25:00.165145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:25:00.165145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:00.165145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:00.165145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:25:00.165145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:25:00.165697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:25:00.165697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:25:00.165697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:00.165697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:00.165697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:00.165697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:25:00.165697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:00.165697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:00.165697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:00.165697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:25:00.165697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:00.165697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:25:00.165697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:25:00.165697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:00.165697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:25:00.165697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:00.165697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:25:00.165697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:25:00.165697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:00.166191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:25:00.166191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:00.166191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:25:00.169281 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:25:00.169661 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:25:00.169823 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:25:00.169994 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:25:00.170638 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:25:00.171001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:00.171866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:25:00.171866 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:00.171866 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:25:00.171866 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:25:00.171866 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:00.171866 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:25:00.171866 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:00.171866 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:25:00.172653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:00.172653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:25:00.172653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:25:00.172653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:25:00.172653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:00.172653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:25:00.172653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:00.172653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:25:00.172653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:00.172653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:25:00.172653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:00.172653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:25:00.172653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:00.172653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:25:00.172653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:25:00.173719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:00.173719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:25:00.173719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:00.173719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:25:00.173719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:25:00.173719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:00.173719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:25:00.173719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:00.173719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:25:00.173719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:00.173719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:25:00.173719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:00.173719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:25:00.173719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:25:00.174640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:25:00.174640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:25:00.174640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:25:00.174640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:25:00.174640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:25:00.174640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:00.174640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:00.174640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:00.174640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:25:00.174640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:00.174640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:00.174640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:00.174640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:25:00.175459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:00.175459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:25:00.175459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:25:00.175459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:00.175459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:25:00.175459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:00.175459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:25:00.175459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:25:00.175459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:00.175459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:25:00.175459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:00.175459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:25:00.176184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:00.176184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:25:00.176184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:25:00.176184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:25:00.176184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:25:00.176184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:25:00.176184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:00.176184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:25:00.176184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:25:00.176184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:25:00.176184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:25:00.176184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:25:00.176184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:00.176184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:00.176184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:25:00.176184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:25:00.177822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:25:00.177822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:25:00.177822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:25:00.177822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:25:00.177822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:25:00.177822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:00.177822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:00.177822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:00.177822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:25:00.177822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:00.177822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:00.177822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:00.177822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:25:00.177822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:00.177822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:25:00.177822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:25:00.177822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:00.177822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:25:00.179504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:00.179504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:25:00.179504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:25:00.179504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:00.179504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:25:00.179504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:00.179504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:25:00.194396 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:25:00.194883 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:25:00.195145 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:25:00.195245 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:25:00.195817 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:25:00.196175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:00.196899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:25:00.196899 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:00.196899 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:25:00.196899 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:25:00.196899 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:00.196899 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:25:00.196899 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:00.196899 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:25:00.198057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:00.198057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:25:00.198057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:25:00.198057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:25:00.198057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:00.198057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:25:00.198057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:00.198057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:25:00.198057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:00.198057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:25:00.198057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:00.198057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:25:00.198057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:00.198057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:25:00.198057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:25:00.198797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:00.198797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:25:00.198797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:00.198797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:25:00.198797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:25:00.198797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:00.198797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:25:00.198797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:00.198797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:25:00.199606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:00.199606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:25:00.199606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:00.199606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:25:00.199606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:25:00.199606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:25:00.199606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:25:00.199606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:25:00.199606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:25:00.199606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:00.199606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:00.200185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:00.200185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:25:00.200185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:00.200185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:00.200185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:00.200185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:25:00.200185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:00.200185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:25:00.200185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:25:00.200185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:00.200185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:25:00.200185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:00.200185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:25:00.200185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:25:00.200855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:00.200855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:25:00.200855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:00.200855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:25:00.201054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:00.201054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:25:00.201054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:25:00.201054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:25:00.201054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:25:00.201054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:25:00.201054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:00.201054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:25:00.201054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:25:00.201054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:25:00.201054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:25:00.201054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:25:00.201054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:00.201054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:00.201054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:25:00.201054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:25:00.201054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:25:00.202010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:25:00.202010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:25:00.202010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:25:00.202010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:00.202010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:00.202010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:00.202010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:25:00.202010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:00.202010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:00.202010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:00.202010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:25:00.202010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:00.202010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:25:00.202010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:25:00.202010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:00.202010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:25:00.202010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:00.202010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:25:00.202010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:25:00.202918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:00.202918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:25:00.202918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:00.202918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:25:00.204043 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:25:00.204369 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:25:00.204547 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:25:00.204758 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:25:00.205210 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:25:00.205578 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:25:00.205799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:00.206756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:25:00.206756 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:00.206756 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:25:00.206756 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:25:00.206756 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:00.206756 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:25:00.206756 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:00.206756 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:25:00.208135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:00.208135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:25:00.208135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:00.208135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:25:00.208135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:25:00.208135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:25:00.208135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:25:00.208135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:00.208135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:00.208135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:00.208135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:25:00.208135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:00.209038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:00.209038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:00.209038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:25:00.209038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:00.209038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:25:00.209038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:25:00.209038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:00.209038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:25:00.209038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:00.209038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:25:00.209038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:25:00.209038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:00.209038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:25:00.209691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:00.209691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:25:00.209691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:00.209691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:25:00.209691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:25:00.209691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:25:00.209691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:25:00.209691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:25:00.209691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:00.209691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:25:00.209691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:25:00.209691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:25:00.209691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:25:00.209691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:25:00.209691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:00.210434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:00.210434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:25:00.210434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:25:00.210434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:25:00.210434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:25:00.210434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:00.210434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:00.210434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:00.210434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:25:00.210434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:00.210434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:00.210434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:00.210434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:25:00.210434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:00.210434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:25:00.210434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:25:00.210434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:00.210434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:25:00.210434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:00.211351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:25:00.211351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:25:00.211351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:00.211351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:25:00.211351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:00.211351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:25:04.945579 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:25:04.946343 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:25:04.946343 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:25:04.947573 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:25:04.981287 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:25:04.981608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:04.982248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:25:04.982248 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:04.982248 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:25:04.982248 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:25:04.982248 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:04.982248 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:25:04.982248 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:04.982248 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:25:04.983197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:04.983197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:25:04.983197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:04.983197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:25:04.983197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:25:04.983197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:25:04.983197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:25:04.983197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:04.983197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:04.983197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:04.983197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:25:04.983197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:04.983786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:04.983786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:04.983786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:25:04.983786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:04.983786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:25:04.983786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:25:04.983786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:04.983786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:25:04.983786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:04.983786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:25:04.983786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:25:04.983786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:04.983786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:25:04.984323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:04.984323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:25:04.984323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:04.984323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:25:04.984323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:25:04.984323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:25:04.984323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:25:04.984323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:25:04.984323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:04.984323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:25:04.984323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:25:04.984323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:25:04.984323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:25:04.984323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:25:04.984323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:04.984931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:04.984931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:25:04.984931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:25:04.984931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:25:04.984931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:25:04.984931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:04.984931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:04.984931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:04.984931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:25:04.984931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:04.984931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:04.984931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:04.984931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:25:04.984931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:04.984931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:25:04.984931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:25:04.984931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:04.984931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:25:04.984931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:04.985755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:25:04.985755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:25:04.985755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:04.985755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:25:04.985755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:04.985755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:25:05.158257 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:25:05.158938 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:25:05.159122 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:25:05.160090 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:25:05.160425 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:25:05.160702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:05.161587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:25:05.161587 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:05.161587 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:25:05.161587 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:25:05.161587 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:05.161587 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:25:05.161587 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:05.161587 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:25:05.162968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:05.162968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:25:05.162968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:05.162968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:25:05.162968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:25:05.162968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:25:05.162968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:25:05.162968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:05.162968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:05.162968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:05.162968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:25:05.162968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:05.163350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:05.163350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:05.163350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:25:05.163350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:05.163350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:25:05.163350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:25:05.163350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:05.163350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:25:05.163350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:05.163350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:25:05.163350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:25:05.163350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:05.163350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:25:05.163697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:05.163697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:25:05.164578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:05.164578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:25:05.164578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:25:05.164578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:25:05.164578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:25:05.164578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:25:05.164578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:05.164578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:25:05.164578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:25:05.164578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:25:05.164578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:25:05.164578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:25:05.164578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:05.164578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:05.164578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:25:05.164578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:25:05.164981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:25:05.164981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:25:05.164981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:05.164981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:05.164981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:05.164981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:25:05.164981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:05.164981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:05.164981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:05.164981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:25:05.164981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:05.164981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:25:05.164981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:25:05.164981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:05.164981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:25:05.164981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:05.164981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:25:05.164981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:25:05.164981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:05.165486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:25:05.165486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:05.165486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:25:05.169646 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:25:05.169903 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:25:05.170011 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:25:05.170125 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:25:05.170621 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:25:05.170898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:05.171602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:25:05.171602 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:05.171602 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:25:05.171602 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:25:05.171602 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:05.171602 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:25:05.171602 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:05.171602 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:25:05.172266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:05.172266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:25:05.172266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:25:05.172266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:25:05.172266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:05.172266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:25:05.172266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:05.172266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:25:05.172266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:05.172266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:25:05.172266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:05.172266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:25:05.172266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:05.172266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:25:05.172266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:25:05.172739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:05.172739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:25:05.172739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:05.172739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:25:05.172739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:25:05.172739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:05.172739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:25:05.172739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:05.172739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:25:05.173114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:05.173114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:25:05.173114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:05.173114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:25:05.173114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:25:05.173114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:25:05.173114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:25:05.173114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:25:05.173114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:25:05.173114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:25:05.173114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:05.173114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:05.173538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:05.173538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:25:05.173538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:05.173538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:05.173538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:05.173538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:25:05.173538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:05.173538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:25:05.173538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:25:05.173538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:05.173538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:25:05.173538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:05.173538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:25:05.173538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:25:05.174184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:05.174184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:25:05.174184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:05.174184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:25:05.174526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:05.174526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:25:05.174526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:25:05.174526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:25:05.174526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:25:05.174526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:25:05.174526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:05.174526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:25:05.174526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:25:05.174526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:25:05.174526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:25:05.174526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:25:05.174526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:05.174526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:05.174526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:25:05.174526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:25:05.175474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:25:05.175474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:25:05.175474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:25:05.175474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:25:05.175474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:25:05.175474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:05.175474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:05.175474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:05.175474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:25:05.175474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:05.175474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:05.175474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:05.175474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:25:05.175474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:05.175474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:25:05.175474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:25:05.175474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:05.175474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:25:05.176232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:05.176232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:25:05.176232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:25:05.176232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:05.176232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:25:05.176232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:05.176232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:25:05.194871 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:25:05.195241 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:25:05.195422 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:25:05.195706 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:25:05.196085 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:25:05.196485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:05.197134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:25:05.197134 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:05.197134 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:25:05.197134 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:25:05.197134 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:05.197134 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:25:05.197134 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:05.197134 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:25:05.198201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:05.198201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:25:05.198201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:25:05.198201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:25:05.198201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:05.198201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:25:05.198201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:05.198201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:25:05.198201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:05.198201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:25:05.198201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:05.198201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:25:05.198201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:05.198201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:25:05.198201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:25:05.199050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:05.199050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:25:05.199050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:05.199050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:25:05.199050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:25:05.199050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:05.199050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:25:05.199050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:05.199050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:25:05.199586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:05.199586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:25:05.199586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:05.199586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:25:05.199586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:25:05.199586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:25:05.199586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:25:05.199586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:25:05.199586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:25:05.199586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:05.199586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:05.200040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:05.200040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:25:05.200040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:05.200040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:05.200040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:05.200040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:25:05.200040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:05.200040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:25:05.200040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:25:05.200040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:05.200040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:25:05.200040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:05.200040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:25:05.200040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:25:05.200643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:05.200643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:25:05.200643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:05.200643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:25:05.201472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:05.201472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:25:05.201472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:25:05.201472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:25:05.201472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:25:05.201472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:25:05.201472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:05.201472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:25:05.201472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:25:05.201472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:25:05.201472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:25:05.201472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:25:05.201472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:05.201472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:05.201472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:25:05.201472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:25:05.201472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:25:05.202157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:25:05.202157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:25:05.202157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:25:05.202157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:05.202157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:05.202157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:05.202157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:25:05.202157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:05.202157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:05.202157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:05.202157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:25:05.202157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:05.202157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:25:05.202157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:25:05.202157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:05.202157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:25:05.202157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:05.202157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:25:05.202157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:25:05.202953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:05.202953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:25:05.202953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:05.202953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:25:05.204540 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:25:05.204795 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:25:05.204888 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:25:05.205061 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:25:05.205487 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:25:05.205841 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:25:05.206064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:05.206743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:25:05.206743 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:05.206743 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:25:05.206743 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:25:05.206743 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:05.206743 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:25:05.206743 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:05.206743 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:25:05.207498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:05.207498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:25:05.207498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:05.207498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:25:05.207498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:25:05.207498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:25:05.207498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:25:05.207498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:05.207498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:05.207498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:05.207498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:25:05.207498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:05.207815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:05.207815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:05.207815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:25:05.207815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:05.207815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:25:05.207815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:25:05.207815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:05.207815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:25:05.207815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:05.207815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:25:05.207815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:25:05.207815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:05.207815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:25:05.208155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:05.208155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:25:05.208387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:05.208387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:25:05.208387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:25:05.208387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:25:05.208387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:25:05.208387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:25:05.208387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:05.208387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:25:05.208387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:25:05.208387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:25:05.208387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:25:05.208387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:25:05.208387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:05.208387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:05.208387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:25:05.208387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:25:05.208914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:25:05.208914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:25:05.208914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:05.208914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:05.208914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:05.208914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:25:05.208914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:05.208914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:05.208914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:05.208914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:25:05.208914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:05.208914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:25:05.208914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:25:05.208914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:05.208914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:25:05.208914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:05.208914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:25:05.208914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:25:05.208914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:05.209417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:25:05.209417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:05.209417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:25:09.946311 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:25:09.947236 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:25:09.947236 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:25:09.948268 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:25:09.976088 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:25:09.976496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:09.977124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:25:09.977124 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:09.977124 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:25:09.977124 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:25:09.977124 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:09.977124 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:25:09.977124 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:09.977124 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:25:09.978536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.030 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:09.978536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:25:09.978536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:09.978536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:25:09.978536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:25:09.978536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:25:09.978536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:25:09.978536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:09.978536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:09.978536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:09.978536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:25:09.978536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:09.979215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:09.979215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:09.979215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:25:09.979215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:09.979215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:25:09.979215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:25:09.979215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:09.979215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:25:09.979215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:09.979215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:25:09.979215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:25:09.979215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:09.979215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:25:09.979945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:09.979945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:25:09.979945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:09.979945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:25:09.979945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:25:09.979945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:25:09.979945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:25:09.979945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:25:09.979945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:09.979945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:25:09.979945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:25:09.979945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:25:09.979945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:25:09.979945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:25:09.979945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:09.980758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:09.980758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:25:09.980758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:25:09.980758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:25:09.980758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:25:09.980758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:09.980758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:09.980758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:09.980758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:25:09.980758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:09.980758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:09.980758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:09.980758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:25:09.980758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:09.980758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:25:09.980758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:25:09.980758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:09.980758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:25:09.980758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:09.981727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:25:09.981727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:25:09.981727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:09.981727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:25:09.981727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:09.981727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:25:10.158946 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:25:10.159240 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:25:10.159394 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:25:10.160199 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:25:10.160463 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:25:10.160620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:10.161374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:25:10.161374 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:10.161374 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:25:10.161374 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:25:10.161374 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:10.161374 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:25:10.161374 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:10.161374 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:25:10.162384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:10.162384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:25:10.162384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:10.162384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:25:10.162384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:25:10.162384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:25:10.162384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:25:10.162384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:10.162384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:10.162384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:10.162384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:25:10.162384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:10.162767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:10.162767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:10.162767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:25:10.162767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:10.162767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:25:10.162767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:25:10.162767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:10.162767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:25:10.162767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:10.162767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:25:10.162767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:25:10.162767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:10.162767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:25:10.163159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:10.163159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:25:10.163702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:10.163702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:25:10.163702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:25:10.163702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:25:10.163702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:25:10.163702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:25:10.163702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:10.163702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:25:10.163702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:25:10.163702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:25:10.163702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:25:10.163702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:25:10.163702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:10.163702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:10.163702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:25:10.163702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:25:10.164270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:25:10.164270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:25:10.164270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:10.164270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:10.164270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:10.164270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:25:10.164270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:10.164270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:10.164270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:10.164270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:25:10.164270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:10.164270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:25:10.164270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:25:10.164270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:10.164270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:25:10.164270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:10.164270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:25:10.164270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:25:10.164270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:10.164820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:25:10.164820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:10.164820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:25:10.169954 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:25:10.170124 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:25:10.170227 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:25:10.170340 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:25:10.170844 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:25:10.171132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:10.171903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:25:10.171903 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:10.171903 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:25:10.171903 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:25:10.171903 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:10.171903 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:25:10.171903 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:10.171903 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:25:10.173052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:10.173052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:25:10.173052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:25:10.173052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:25:10.173052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:10.173052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:25:10.173052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:10.173052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:25:10.173052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:10.173052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:25:10.173052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:10.173052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:25:10.173052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:10.173052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:25:10.173052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:25:10.173555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:10.173555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:25:10.173555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:10.173555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:25:10.173555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:25:10.173555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:10.173555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:25:10.173555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:10.173555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:25:10.174335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:10.174335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:25:10.174335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:10.174335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:25:10.174335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:25:10.174335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:25:10.174335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:25:10.174335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:25:10.174335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:25:10.174335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:25:10.174335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:10.174335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:10.174746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:10.174746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:25:10.174746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:10.174746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:10.174746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:10.174746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:25:10.174746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:10.174746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:25:10.174746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:25:10.174746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:10.174746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:25:10.174746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:10.174746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:25:10.174746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:25:10.175156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:10.175156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:25:10.175156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:10.175156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:25:10.176024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:10.176024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:25:10.176024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:25:10.176024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:25:10.176024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:25:10.176024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:25:10.176024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:10.176024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:25:10.176024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:25:10.176024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:25:10.176024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:25:10.176024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:25:10.176024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:10.176024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:10.176024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:25:10.176024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:25:10.176495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:25:10.176495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:25:10.176495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:25:10.176495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:25:10.176495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:25:10.176495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:10.176495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:10.176495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:10.176495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:25:10.176495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:10.176495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:10.176495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:10.176495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:25:10.176495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:10.176495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:25:10.176495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:25:10.176495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:10.176495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:25:10.177012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:10.177012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:25:10.177012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:25:10.177012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:10.177012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:25:10.177012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:10.177012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:25:10.195294 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:25:10.195627 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:25:10.195762 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:25:10.195946 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:25:10.196443 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:25:10.196770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:10.197544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:25:10.197544 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:10.197544 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:25:10.197544 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:25:10.197544 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:10.197544 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:25:10.197544 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:10.197544 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:25:10.198640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:10.198640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:25:10.198640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:25:10.198640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:25:10.198640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:10.198640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:25:10.198640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:10.198640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:25:10.198640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:10.198640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:25:10.198640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:10.198640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:25:10.198640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:10.198640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:25:10.198640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:25:10.199037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:10.199037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:25:10.199037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:10.199037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:25:10.199037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:25:10.199037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:10.199037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:25:10.199037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:10.199037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:25:10.200076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:10.200076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:25:10.200076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:10.200076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:25:10.200076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:25:10.200076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:25:10.200076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:25:10.200076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:25:10.200076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:25:10.200076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:10.200076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:10.200443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:10.200443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:25:10.200443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:10.200443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:10.200443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:10.200443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:25:10.200443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:10.200443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:25:10.200443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:25:10.200443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:10.200443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:25:10.200443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:10.200443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:25:10.200443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:25:10.200827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:10.200827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:25:10.200827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:10.200827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:25:10.201774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:10.201774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:25:10.201774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:25:10.201774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:25:10.201774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:25:10.201774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:25:10.201774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:10.201774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:25:10.201774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:25:10.201774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:25:10.201774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:25:10.201774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:25:10.201774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:10.201774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:10.201774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:25:10.201774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:25:10.201774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:25:10.202274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:25:10.202274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:25:10.202274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:25:10.202274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:10.202274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:10.202274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:10.202274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:25:10.202274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:10.202274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:10.202274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:10.202274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:25:10.202274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:10.202274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:25:10.202274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:25:10.202274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:10.202274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:25:10.202274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:10.202274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:25:10.202274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:25:10.202891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:10.202891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:25:10.202891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:10.202891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:25:10.204886 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:25:10.205160 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:25:10.205318 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:25:10.205582 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:25:10.206046 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:25:10.206366 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:25:10.206606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:10.207266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:25:10.207266 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:10.207266 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:25:10.207266 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:25:10.207266 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:10.207266 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:25:10.207266 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:10.207266 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:25:10.208071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:10.208071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:25:10.208071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:10.208071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:25:10.208071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:25:10.208071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:25:10.208071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:25:10.208071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:10.208071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:10.208071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:10.208071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:25:10.208071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:10.208474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:10.208474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:10.208474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:25:10.208474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:10.208474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:25:10.208474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:25:10.208474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:10.208474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:25:10.208474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:10.208474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:25:10.208474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:25:10.208474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:10.208474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:25:10.208832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:10.208832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:25:10.209004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:10.209004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:25:10.209004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:25:10.209004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:25:10.209004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:25:10.209004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:25:10.209004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:10.209004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:25:10.209004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:25:10.209004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:25:10.209004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:25:10.209004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:25:10.209004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:10.209004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:10.209004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:25:10.209004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:25:10.209494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:25:10.209494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:25:10.209494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:10.209494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:10.209494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:10.209494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:25:10.209494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:10.209494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:10.209494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:10.209494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:25:10.209494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:10.209494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:25:10.209494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:25:10.209494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:10.209494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:25:10.209494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:10.209494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:25:10.209494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:25:10.209494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:10.210001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:25:10.210001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:10.210001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:25:14.947002 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:25:14.947941 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:25:14.947941 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:25:14.948852 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:25:14.990326 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:25:14.990681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:14.991515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:25:14.991515 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:14.991515 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:25:14.991515 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:25:14.991515 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:14.991515 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:25:14.991515 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:14.991515 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:25:14.992274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:14.992274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:25:14.992274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:14.992274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:25:14.992274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:25:14.992274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:25:14.992274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:25:14.992274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:14.992274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:14.992274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:14.992274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:25:14.992274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:14.992920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:14.992920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:14.992920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:25:14.992920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:14.992920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:25:14.992920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:25:14.992920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:14.992920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:25:14.992920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:14.992920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:25:14.992920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:25:14.992920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:14.992920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:25:14.993605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:14.993605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:25:14.993605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:14.993605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:25:14.993605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:25:14.993605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:25:14.993605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:25:14.993605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:25:14.993605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:14.993605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:25:14.993605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:25:14.993605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:25:14.993605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:25:14.993605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:25:14.993605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:14.994345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:14.994345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:25:14.994345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:25:14.994345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:25:14.994345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:25:14.994345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:14.994345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:14.994345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:14.994345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:25:14.994345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:14.994345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:14.994345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:14.994345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:25:14.994345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:14.994345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:25:14.994345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:25:14.994345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:14.994345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:25:14.994345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:14.995263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:25:14.995263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:25:14.995263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:14.995263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:25:14.995263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:14.995263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:25:15.159528 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:25:15.159975 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:25:15.160170 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:25:15.161118 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:25:15.161372 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:25:15.161616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:15.162177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:25:15.162177 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:15.162177 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:25:15.162177 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:25:15.162177 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:15.162177 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:25:15.162177 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:15.162177 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:25:15.163435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:15.163435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:25:15.163435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:15.163435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:25:15.163435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:25:15.163435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:25:15.163435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:25:15.163435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:15.163435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:15.163435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:15.163435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:25:15.163435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:15.164023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:15.164023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:15.164023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:25:15.164023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:15.164023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:25:15.164023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:25:15.164023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:15.164023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:25:15.164023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:15.164023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:25:15.164023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:25:15.164023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:15.164023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:25:15.164727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:15.164727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:25:15.164727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:15.164727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:25:15.164727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:25:15.164727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:25:15.164727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:25:15.164727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:25:15.164727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:15.164727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:25:15.164727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:25:15.164727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:25:15.164727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:25:15.164727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:25:15.164727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:15.165346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:15.165346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:25:15.165346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:25:15.165346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:25:15.165346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:25:15.165346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:15.165346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:15.165346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:15.165346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:25:15.165346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:15.165346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:15.165346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:15.165346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:25:15.165346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:15.165346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:25:15.165346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:25:15.165346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:15.165346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:25:15.165346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:15.166192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:25:15.166192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:25:15.166192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:15.166192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:25:15.166192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:15.166192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:25:15.170285 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:25:15.170584 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:25:15.170728 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:25:15.170895 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:25:15.171344 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:25:15.171666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:15.172264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:25:15.172264 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:15.172264 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:25:15.172264 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:25:15.172264 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:15.172264 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:25:15.172264 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:15.172264 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:25:15.173119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:15.173119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:25:15.173119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:25:15.173119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:25:15.173119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:15.173119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:25:15.173119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:15.173119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:25:15.173119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:15.173119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:25:15.173119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:15.173119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:25:15.173119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:15.173119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:25:15.173119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:25:15.173590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:15.173590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:25:15.173590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:15.173590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:25:15.173590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:25:15.173590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:15.173590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:25:15.173590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:15.173590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:25:15.174222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:15.174222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:25:15.174222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:15.174222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:25:15.174222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:25:15.174222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:25:15.174222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:25:15.174222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:25:15.174222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:25:15.174222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:25:15.174222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:15.174222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:15.174579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:15.174579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:25:15.174579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:15.174579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:15.174579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:15.174579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:25:15.174579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:15.174579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:25:15.174579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:25:15.174579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:15.174579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:25:15.174579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:15.174579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:25:15.174579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:25:15.174943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:15.174943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:25:15.174943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:15.174943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:25:15.175415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:15.175415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:25:15.175415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:25:15.175415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:25:15.175415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:25:15.175415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:25:15.175415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:15.175415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:25:15.175415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:25:15.175415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:25:15.175415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:25:15.175415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:25:15.175415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:15.175415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:15.175415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:25:15.175415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:25:15.175827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:25:15.175827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:25:15.175827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:25:15.175827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:25:15.175827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:25:15.175827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:15.175827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:15.175827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:15.175827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:25:15.175827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:15.175827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:15.175827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:15.175827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:25:15.175827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:15.175827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:25:15.175827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:25:15.175827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:15.175827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:25:15.176297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:15.176297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:25:15.176297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:25:15.176297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:15.176297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:25:15.176297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:15.176297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:25:15.195720 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:25:15.196136 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:25:15.196326 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:25:15.196585 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:25:15.197147 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:25:15.197476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:15.198340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:25:15.198340 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:15.198340 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:25:15.198340 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:25:15.198340 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:15.198340 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:25:15.198340 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:15.198340 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:25:15.199742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:15.199742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:25:15.199742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:25:15.199742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:25:15.199742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:15.199742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:25:15.199742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:15.199742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:25:15.199742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:15.199742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:25:15.199742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:15.199742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:25:15.199742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:15.199742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:25:15.199742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:25:15.200420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:15.200420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:25:15.200420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:15.200420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:25:15.200420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:25:15.200420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:15.200420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:25:15.200420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:15.200420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:25:15.201237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:15.201237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:25:15.201237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:15.201237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:25:15.201237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:25:15.201237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:25:15.201237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:25:15.201237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:25:15.201237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:25:15.201237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:15.201237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:15.201820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:15.201820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:25:15.201820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:15.201820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:15.201820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:15.201820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:25:15.201820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:15.201820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:25:15.201820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:25:15.201820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:15.201820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:25:15.201820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:15.201820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:25:15.201820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:25:15.202440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:15.202440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:25:15.202440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:15.202440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:25:15.204723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:15.204723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:25:15.204723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:25:15.204723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:25:15.204723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:25:15.204723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:25:15.204723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:15.204723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:25:15.204723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:25:15.204723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:25:15.204723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:25:15.204723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:25:15.204723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:15.204723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:15.204723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:25:15.204723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:25:15.204723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:25:15.205766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:25:15.205766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:25:15.205766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:25:15.205766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:15.205766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:15.205766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:15.205766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:25:15.205766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:15.205766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:15.205766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:15.205766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:25:15.205766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:15.205766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:25:15.205766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:25:15.205766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:15.205766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:25:15.205766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:15.205766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:25:15.205766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:25:15.206623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:15.206623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:25:15.206623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:15.206623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:25:15.206623 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:25:15.206623 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:25:15.206623 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:25:15.206623 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:25:15.207046 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:25:15.207046 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:25:15.207046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:15.207609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:25:15.207609 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:15.207609 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:25:15.207609 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:25:15.207609 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:15.207609 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:25:15.207609 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:15.207609 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:25:15.208475 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:15.208475 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:25:15.208475 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:15.208475 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:25:15.208475 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:25:15.208475 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:25:15.208475 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:25:15.208475 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:15.208475 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:15.208475 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:15.208475 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:25:15.208475 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:15.208958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:15.208958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:15.208958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:25:15.208958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:15.208958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:25:15.208958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:25:15.208958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:15.208958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:25:15.208958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:15.208958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:25:15.208958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:25:15.208958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:15.208958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:25:15.209321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:15.209321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:25:15.209538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:15.209538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:25:15.209538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:25:15.209538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:25:15.209538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:25:15.209538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:25:15.209538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:15.209538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:25:15.209538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:25:15.209538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:25:15.209538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:25:15.209538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:25:15.209538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:15.209538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:15.209538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:25:15.209538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:25:15.209987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:25:15.209987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:25:15.209987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:15.209987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:15.209987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:15.209987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:25:15.209987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:15.209987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:15.209987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:15.209987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:25:15.209987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:15.209987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:25:15.209987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:25:15.209987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:15.209987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:25:15.209987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:15.209987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:25:15.209987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:25:15.209987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:15.210515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:25:15.210515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:15.210515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:25:19.947789 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:25:19.948740 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:25:19.948740 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:25:19.949217 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:25:19.983666 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:25:19.984036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:19.984900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:25:19.984900 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:19.984900 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:25:19.984900 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:25:19.984900 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:19.984900 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:25:19.984900 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:19.984900 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:25:19.987492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:19.987492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:25:19.987492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:19.987492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:25:19.987492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:25:19.987492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:25:19.987492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:25:19.987492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:19.987492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:19.987492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:19.987492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:25:19.987492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:19.988023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:19.988023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:19.988023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:25:19.988023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:19.988023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:25:19.988023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:25:19.988023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:19.988023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:25:19.988023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:19.988023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:25:19.988023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:25:19.988023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:19.988023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:25:19.990552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:19.990552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:25:19.990552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:19.990552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:25:19.990552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:25:19.990552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:25:19.990552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:25:19.990552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:25:19.990552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:19.990552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:25:19.990552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:25:19.990552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:25:19.990552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:25:19.990552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:25:19.990552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:19.991160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:19.991160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:25:19.991160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:25:19.991160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:25:19.991160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:25:19.991160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:19.991160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:19.991160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:19.991160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:25:19.991160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:19.991160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:19.991160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:19.991160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:25:19.991160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:19.991160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:25:19.991160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:25:19.991160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:19.991160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:25:19.991160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:19.991815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:25:19.991815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:25:19.991815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:19.991815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:25:19.991815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:19.991815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:25:20.160027 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:25:20.160670 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:25:20.160784 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:25:20.161572 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:25:20.161810 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:25:20.162030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:20.162637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:25:20.162637 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:20.162637 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:25:20.162637 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:25:20.162637 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:20.162637 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:25:20.162637 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:20.162637 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:25:20.163715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:20.163715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:25:20.163715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:20.163715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:25:20.163715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:25:20.163715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:25:20.163715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:25:20.163715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:20.163715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:20.163715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:20.163715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:25:20.163715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:20.164105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:20.164105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:20.164105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:25:20.164105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:20.164105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:25:20.164105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:25:20.164105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:20.164105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:25:20.164105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:20.164105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:25:20.164105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:25:20.164105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:20.164105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:25:20.164502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:20.164502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:25:20.164789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:20.164789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:25:20.164789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:25:20.164789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:25:20.164789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:25:20.164789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:25:20.164789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:20.164789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:25:20.164789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:25:20.164789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:25:20.164789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:25:20.164789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:25:20.164789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:20.164789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:20.164789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:25:20.164789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:25:20.165242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:25:20.165242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:25:20.165242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:20.165242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:20.165242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:20.165242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:25:20.165242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:20.165242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:20.165242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:20.165242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:25:20.165242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:20.165242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:25:20.165242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:25:20.165242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:20.165242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:25:20.165242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:20.165242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:25:20.165242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:25:20.165242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:20.165784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:25:20.165784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:20.165784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:25:20.170672 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:25:20.170937 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:25:20.171049 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:25:20.171187 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:25:20.171562 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:25:20.171797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:20.172333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:25:20.172333 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:20.172333 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:25:20.172333 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:25:20.172333 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:20.172333 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:25:20.172333 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:20.172333 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:25:20.173100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:20.173100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:25:20.173100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:25:20.173100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:25:20.173100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:20.173100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:25:20.173100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:20.173100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:25:20.173100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:20.173100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:25:20.173100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:20.173100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:25:20.173100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:20.173100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:25:20.173100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:25:20.173529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:20.173529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:25:20.173529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:20.173529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:25:20.173529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:25:20.173529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:20.173529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:25:20.173529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:20.173529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:25:20.173903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:20.173903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:25:20.173903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:20.173903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:25:20.173903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:25:20.173903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:25:20.173903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:25:20.173903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:25:20.173903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:25:20.173903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:25:20.173903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:20.173903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:20.174275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:20.174275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:25:20.174275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:20.174275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:20.174275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:20.174275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:25:20.174275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:20.174275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:25:20.174275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:25:20.174275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:20.174275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:25:20.174275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:20.174275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:25:20.174275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:25:20.174683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:20.174683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:25:20.174683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:20.174683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:25:20.174904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:20.174904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:25:20.174904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:25:20.174904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:25:20.174904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:25:20.174904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:25:20.174904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:20.174904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:25:20.174904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:25:20.174904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:25:20.174904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:25:20.174904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:25:20.174904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:20.174904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:20.174904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:25:20.174904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:25:20.175675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:25:20.175675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:25:20.175675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:25:20.175675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:25:20.175675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:25:20.175675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:20.175675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:20.175675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:20.175675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:25:20.175675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:20.175675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:20.175675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:20.175675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:25:20.175675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:20.175675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:25:20.175675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:25:20.175675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:20.175675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:25:20.176518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:20.176518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:25:20.176518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:25:20.176518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:20.176518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:25:20.176518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:20.176518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:25:20.196180 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:25:20.196498 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:25:20.196597 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:25:20.196732 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:25:20.197317 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:25:20.197741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:20.198383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:25:20.198383 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:20.198383 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:25:20.198383 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:25:20.198383 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:20.198383 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:25:20.198383 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:20.198383 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:25:20.199047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:20.199047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:25:20.199047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:25:20.199047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:25:20.199047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:20.199047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:25:20.199047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:20.199047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:25:20.199047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:20.199047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:25:20.199047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:20.199047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:25:20.199047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:20.199047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:25:20.199047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:25:20.199796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:20.199796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:25:20.199796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:20.199796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:25:20.199796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:25:20.199796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:20.199796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:25:20.199796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:20.199796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:25:20.200213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:20.200213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:25:20.200213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:20.200213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:25:20.200213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:25:20.200213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:25:20.200213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:25:20.200213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:25:20.200213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:25:20.200213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:20.200213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:20.200791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:20.200791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:25:20.200791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:20.200791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:20.200791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:20.200791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:25:20.200791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:20.200791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:25:20.200791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:25:20.200791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:20.200791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:25:20.200791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:20.200791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:25:20.200791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:25:20.201654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:20.201654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:25:20.201654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:20.201654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:25:20.201654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:20.201654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:25:20.201654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:25:20.201654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:25:20.201654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:25:20.201654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:25:20.201654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:20.201654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:25:20.201654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:25:20.201654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:25:20.202662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:25:20.202662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:25:20.202662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:20.202662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:20.202662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:25:20.202662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:25:20.202662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:25:20.202662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:25:20.202662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:25:20.202662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:25:20.202662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:20.202662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:20.202662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:20.202662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:25:20.202662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:20.202662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:20.202662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:20.202662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:25:20.202662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:20.203392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:25:20.203392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:25:20.203392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:20.203392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:25:20.203392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:20.203392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:25:20.203392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:25:20.203392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:20.203392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:25:20.203392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:20.203392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:25:20.205764 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:25:20.206303 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:25:20.206401 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:25:20.206569 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:25:20.206877 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:25:20.207153 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:25:20.207350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:20.207897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:25:20.207897 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:20.207897 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:25:20.207897 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:25:20.207897 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:20.207897 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:25:20.207897 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:20.207897 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:25:20.208686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:20.208686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:25:20.208686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:20.208686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:25:20.208686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:25:20.208686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:25:20.208686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:25:20.208686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:20.208686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:20.208686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:20.208686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:25:20.208686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:20.209129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:20.209129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:20.209129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:25:20.209129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:20.209129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:25:20.209129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:25:20.209129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:20.209129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:25:20.209129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:20.209129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:25:20.209129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:25:20.209129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:20.209129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:25:20.209503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:20.209503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:25:20.209634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:20.209634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:25:20.209634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:25:20.209634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:25:20.209634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:25:20.209634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:25:20.209634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:20.209634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:25:20.209634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:25:20.209634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:25:20.209634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:25:20.209634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:25:20.209634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:20.209634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:20.209634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:25:20.209634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:25:20.210139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:25:20.210139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:25:20.210139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:20.210139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:20.210139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:20.210139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:25:20.210139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:20.210139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:20.210139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:20.210139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:25:20.210139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:20.210139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:25:20.210139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:25:20.210139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:20.210139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:25:20.210139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:20.210139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:25:20.210139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:25:20.210139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:20.210651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:25:20.210651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:20.210651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:25:24.948671 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:25:24.949607 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:25:24.949607 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:25:24.950400 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:25:24.952193 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG dbcounter [-] [105581] Writing DB stats neutron:SELECT=434 {{(pid=105581) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 14:25:24.989895 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:25:24.990141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:24.990822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:25:24.990822 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:24.990822 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:25:24.990822 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:25:24.990822 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:24.990822 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:25:24.990822 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:24.990822 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:25:24.991770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:24.991770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:25:24.991770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:24.991770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:25:24.991770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:25:24.991770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:25:24.991770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:25:24.991770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:24.991770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:24.991770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:24.991770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:25:24.991770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:24.992582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:24.992582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:24.992582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:25:24.992582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:24.992582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:25:24.992582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:25:24.992582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:24.992582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:25:24.992582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:24.992582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:25:24.992582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:25:24.992582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:24.992582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:25:24.993385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:24.993385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:25:24.993385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:24.993385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:25:24.993385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:25:24.993385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:25:24.993385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:25:24.993385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:25:24.993385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:24.993385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:25:24.993385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:25:24.993385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:25:24.993385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:25:24.993385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:25:24.993385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:24.994186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:24.994186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:25:24.994186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:25:24.994186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:25:24.994186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:25:24.994186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:24.994186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:24.994186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:24.994186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:25:24.994186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:24.994186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:24.994186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:24.994186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:25:24.994186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:24.994186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:25:24.994186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:25:24.994186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:24.994186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:25:24.994186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:24.995127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:25:24.995127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:25:24.995127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:24.995127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:25:24.995127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:24.995127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:25:25.160856 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:25:25.161325 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:25:25.161503 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:25:25.162424 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:25:25.162656 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:25:25.162829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:25.163476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:25:25.163476 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:25.163476 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:25:25.163476 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:25:25.163476 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:25.163476 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:25:25.163476 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:25.163476 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:25:25.164513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:25.164513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:25:25.164513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:25.164513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:25:25.164513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:25:25.164513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:25:25.164513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:25:25.164513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:25.164513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:25.164513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:25.164513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:25:25.164513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:25.165294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:25.165294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:25.165294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:25:25.165294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:25.165294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:25:25.165294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:25:25.165294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:25.165294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:25:25.165294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:25.165294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:25:25.165294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:25:25.165294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:25.165294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:25:25.166074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:25.166074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:25:25.166074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:25.166074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:25:25.166074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:25:25.166074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:25:25.166074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:25:25.166074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:25:25.166074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:25.166074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:25:25.166074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:25:25.166074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:25:25.166074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:25:25.166074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:25:25.166074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:25.166981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:25.166981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:25:25.166981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:25:25.166981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:25:25.166981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:25:25.166981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:25.166981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:25.166981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:25.166981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:25:25.166981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:25.166981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:25.166981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:25.166981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:25:25.166981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:25.166981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:25:25.166981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:25:25.166981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:25.166981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:25:25.166981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:25.168138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:25:25.168138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:25:25.168138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:25.168138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:25:25.168138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:25.168138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:25:25.171017 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:25:25.171486 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:25:25.171579 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:25:25.171704 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:25:25.172202 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:25:25.172639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:25.173478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:25:25.173478 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:25.173478 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:25:25.173478 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:25:25.173478 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:25.173478 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:25:25.173478 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:25.173478 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:25:25.174211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:25.174211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:25:25.174211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:25:25.174211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:25:25.174211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:25.174211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:25:25.174211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:25.174211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:25:25.174211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:25.174211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:25:25.174211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:25.174211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:25:25.174211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:25.174211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:25:25.174211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:25:25.175191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:25.175191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:25:25.175191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:25.175191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:25:25.175191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:25:25.175191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:25.175191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:25:25.175191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:25.175191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:25:25.175191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:25.175191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:25:25.175191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:25.175191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:25:25.175191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:25:25.175955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:25:25.175955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:25:25.175955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:25:25.175955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:25:25.175955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:25:25.175955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:25.175955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:25.175955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:25.175955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:25:25.175955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:25.175955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:25.175955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:25.175955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:25:25.176482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:25.176482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:25:25.176482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:25:25.176482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:25.176482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:25:25.176482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:25.176482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:25:25.176482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:25:25.176482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:25.176482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:25:25.176482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:25.176482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:25:25.176961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:25.176961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:25:25.176961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:25:25.176961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:25:25.176961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:25:25.176961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:25:25.176961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:25.176961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:25:25.176961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:25:25.176961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:25:25.176961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:25:25.176961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:25:25.176961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:25.176961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:25.176961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:25:25.176961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:25:25.177565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:25:25.177565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:25:25.177565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:25:25.177565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:25:25.177565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:25:25.177565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:25.177565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:25.177565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:25.177565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:25:25.177565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:25.177565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:25.177565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:25.177565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:25:25.177565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:25.177565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:25:25.177565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:25:25.177565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:25.177565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:25:25.178231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:25.178231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:25:25.178231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:25:25.178231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:25.178231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:25:25.178231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:25.178231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:25:25.196677 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:25:25.196971 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:25:25.197078 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:25:25.197172 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:25:25.198888 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:25:25.198888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:25.199067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:25:25.199067 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:25.199067 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:25:25.199067 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:25:25.199067 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:25.199067 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:25:25.199067 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:25.199067 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:25:25.199741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:25.199741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:25:25.199741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:25:25.199741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:25:25.199741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:25.199741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:25:25.199741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:25.199741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:25:25.199741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:25.199741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:25:25.199741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:25.199741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:25:25.199741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:25.199741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:25:25.199741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:25:25.200763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:25.200763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:25:25.200763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:25.200763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:25:25.200763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:25:25.200763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:25.200763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:25:25.200763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:25.200763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:25:25.200763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:25.200763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:25:25.200763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:25.200763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:25:25.200763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:25:25.201492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:25:25.201492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:25:25.201492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:25:25.201492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:25:25.201492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:25.201492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:25.201492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:25.201492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:25:25.201492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:25.201492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:25.201492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:25.201492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:25:25.201492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:25.202001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:25:25.202001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:25:25.202001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:25.202001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:25:25.202001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:25.202001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:25:25.202001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:25:25.202001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:25.202001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:25:25.202001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:25.202001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:25:25.202001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:25.202508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:25:25.202508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:25:25.202508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:25:25.202508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:25:25.202508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:25:25.202508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:25.202508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:25:25.202508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:25:25.202508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:25:25.202508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:25:25.202508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:25:25.202508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:25.202508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:25.202508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:25:25.202508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:25:25.202508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:25:25.202508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:25:25.202508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:25:25.202508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:25:25.203167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:25.203167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:25.203167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:25.203167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:25:25.203167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:25.203167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:25.203167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:25.203167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:25:25.203167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:25.203167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:25:25.203167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:25:25.203167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:25.203167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:25:25.203167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:25.203167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:25:25.203167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:25:25.203167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:25.203167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:25:25.203167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:25.203167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:25:25.206447 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:25:25.206736 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:25:25.206839 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:25:25.206997 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:25:25.207360 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:25:25.207748 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:25:25.208035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:25.208840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:25:25.208840 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:25.208840 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:25:25.208840 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:25:25.208840 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:25.208840 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:25:25.208840 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:25.208840 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:25:25.210152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:25.210152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:25:25.210152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:25.210152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:25:25.210152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:25:25.210152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:25:25.210152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:25:25.210152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:25.210152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:25.210152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:25.210152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:25:25.210152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:25.210751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:25.210751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:25.210751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:25:25.210751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:25.210751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:25:25.210751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:25:25.210751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:25.210751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:25:25.210751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:25.210751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:25:25.210751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:25:25.210751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:25.210751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:25:25.211245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:25.211245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:25:25.211780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:25.211780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:25:25.211780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:25:25.211780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:25:25.211780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:25:25.211780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:25:25.211780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:25.211780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:25:25.211780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:25:25.211780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:25:25.211780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:25:25.211780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:25:25.211780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:25.211780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:25.211780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:25:25.211780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:25:25.212427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:25:25.212427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:25:25.212427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:25.212427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:25.212427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:25.212427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:25:25.212427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:25.212427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:25.212427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:25.212427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:25:25.212427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:25.212427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:25:25.212427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:25:25.212427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:25.212427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:25:25.212427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:25.212427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:25:25.212427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:25:25.212427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:25.213177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:25:25.213177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:25.213177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:25:29.949294 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:25:29.949995 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:25:29.950242 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:25:29.951935 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:25:29.989765 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:25:29.989765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:29.989765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:25:29.989765 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:29.989765 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:25:29.989765 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:25:29.989765 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:29.989765 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:25:29.989765 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:29.989765 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:25:29.990516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:29.990516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:25:29.990516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:29.990516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:25:29.990516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:25:29.990516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:25:29.990516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:25:29.990516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:29.990516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:29.990516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:29.990516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:25:29.990516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:29.991075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:29.991075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:29.991075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:25:29.991075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:29.991075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:25:29.991075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:25:29.991075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:29.991075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:25:29.991075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:29.991075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:25:29.991075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:25:29.991075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:29.991075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:25:29.991489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:29.991489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:25:29.991489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:29.991489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:25:29.991489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:25:29.991489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:25:29.991489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:25:29.991489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:25:29.991489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:29.991489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:25:29.991489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:25:29.991489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:25:29.991489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:25:29.991489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:25:29.991489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:29.991891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:29.991891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:25:29.991891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:25:29.991891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:25:29.991891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:25:29.991891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:29.991891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:29.991891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:29.991891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:25:29.991891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:29.991891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:29.991891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:29.991891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:25:29.991891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:29.991891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:25:29.991891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:25:29.991891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:29.991891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:25:29.991891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:29.992390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:25:29.992390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:25:29.992390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:29.992390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:25:29.992390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:29.992390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:25:30.162037 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:25:30.162202 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:25:30.162202 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:25:30.164046 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:25:30.164046 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:25:30.164046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:30.164298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:25:30.164298 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:30.164298 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:25:30.164298 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:25:30.164298 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:30.164298 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:25:30.164298 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:30.164298 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:25:30.165751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:30.165751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:25:30.165751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:30.165751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:25:30.165751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:25:30.165751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:25:30.165751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:25:30.165751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:30.165751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:30.165751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:30.165751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:25:30.165751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:30.166069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:30.166069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:30.166069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:25:30.166069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:30.166069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:25:30.166069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:25:30.166069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:30.166069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:25:30.166069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:30.166069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:25:30.166069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:25:30.166069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:30.166069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:25:30.166401 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:30.166401 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:25:30.167396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:30.167396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:25:30.167396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:25:30.167396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:25:30.167396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:25:30.167396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:25:30.167396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:30.167396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:25:30.167396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:25:30.167396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:25:30.167396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:25:30.167396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:25:30.167396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:30.167396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:30.167396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:25:30.167396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:25:30.167826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:25:30.167826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:25:30.167826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:30.167826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:30.167826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:30.167826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:25:30.167826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:30.167826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:30.167826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:30.167826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:25:30.167826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:30.167826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:25:30.167826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:25:30.167826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:30.167826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:25:30.167826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:30.167826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:25:30.167826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:25:30.167826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:30.168310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:25:30.168310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:30.168310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:25:30.171648 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:25:30.172086 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:25:30.172301 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:25:30.172543 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:25:30.173476 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:25:30.173476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:30.173662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:25:30.173662 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:30.173662 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:25:30.173662 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:25:30.173662 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:30.173662 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:25:30.173662 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:30.173662 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:25:30.174381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:30.174381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:25:30.174381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:25:30.174381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:25:30.174381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:30.174381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:25:30.174381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:30.174381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:25:30.174381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:30.174381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:25:30.174381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:30.174381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:25:30.174381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:30.174381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:25:30.174381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:25:30.174805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:30.174805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:25:30.174805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:30.174805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:25:30.174805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:25:30.174805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:30.174805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:25:30.174805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:30.174805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:25:30.175198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:30.175198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:25:30.175198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:30.175198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:25:30.175198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:25:30.175198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:25:30.175198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:25:30.175198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:25:30.175198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:25:30.175198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:25:30.175198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:30.175198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:30.175569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:30.175569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:25:30.175569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:30.175569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:30.175569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:30.175569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:25:30.175569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:30.175569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:25:30.175569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:25:30.175569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:30.175569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:25:30.175569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:30.175569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:25:30.175569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:25:30.175947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:30.175947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:25:30.175947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:30.175947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:25:30.176197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:30.176197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:25:30.176197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:25:30.176197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:25:30.176197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:25:30.176197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:25:30.176197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:30.176197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:25:30.176197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:25:30.176197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:25:30.176197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:25:30.176197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:25:30.176197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:30.176197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:30.176197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:25:30.176197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:25:30.176697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:25:30.176697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:25:30.176697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:25:30.176697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:25:30.176697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:25:30.176697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:30.176697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:30.176697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:30.176697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:25:30.176697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:30.176697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:30.176697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:30.176697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:25:30.176697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:30.176697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:25:30.176697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:25:30.176697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:30.176697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:25:30.177285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:30.177285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:25:30.177285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:25:30.177285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:30.177285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:25:30.177285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:30.177285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:25:30.197135 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:25:30.197565 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:25:30.197735 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:25:30.197956 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:25:30.198553 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:25:30.199058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:30.200134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:25:30.200134 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:30.200134 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:25:30.200134 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:25:30.200134 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:30.200134 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:25:30.200134 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:30.200134 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:25:30.201350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:30.201350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:25:30.201350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:25:30.201350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:25:30.201350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:30.201350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:25:30.201350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:30.201350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:25:30.201350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:30.201350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:25:30.201350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:30.201350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:25:30.201350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:30.201350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:25:30.201350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:25:30.202037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:30.202037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:25:30.202037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:30.202037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:25:30.202037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:25:30.202037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:30.202037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:25:30.202037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:30.202037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:25:30.202798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:30.202798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:25:30.202798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:30.202798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:25:30.202798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:25:30.202798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:25:30.202798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:25:30.202798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:25:30.202798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:25:30.202798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:30.202798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:30.203356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:30.203356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:25:30.203356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:30.203356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:30.203356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:30.203356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:25:30.203356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:30.203356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:25:30.203356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:25:30.203356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:30.203356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:25:30.203356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:30.203356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:25:30.203356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:25:30.204164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:30.204164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:25:30.204164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:30.204164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:25:30.204587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:30.204587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:25:30.204587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:25:30.204587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:25:30.204587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:25:30.204587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:25:30.204587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:30.204587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:25:30.204587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:25:30.204587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:25:30.204587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:25:30.204587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:25:30.204587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:30.204587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:30.204587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:25:30.204587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:25:30.204587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:25:30.205365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:25:30.205365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:25:30.205365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:25:30.205365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:30.205365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:30.205365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:30.205365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:25:30.205365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:30.205365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:30.205365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:30.205365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:25:30.205365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:30.205365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:25:30.205365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:25:30.205365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:30.205365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:25:30.205365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:30.205365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:25:30.205365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:25:30.206486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:30.206486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:25:30.206486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:30.206486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:25:30.206886 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:25:30.207227 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:25:30.207511 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:25:30.207742 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:25:30.208505 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:25:30.208600 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:25:30.208766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:30.209283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:25:30.209283 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:30.209283 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:25:30.209283 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:25:30.209283 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:30.209283 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:25:30.209283 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:30.209283 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:25:30.210074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:30.210074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:25:30.210074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:30.210074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:25:30.210074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:25:30.210074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:25:30.210074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:25:30.210074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:30.210074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:30.210074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:30.210074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:25:30.210074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:30.210424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:30.210424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:30.210424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:25:30.210424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:30.210424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:25:30.210424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:25:30.210424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:30.210424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:25:30.210424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:30.210424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:25:30.210424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:25:30.210424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:30.210424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:25:30.210783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:30.210783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:25:30.210996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:30.210996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:25:30.210996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:25:30.210996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:25:30.210996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:25:30.210996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:25:30.210996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:30.210996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:25:30.210996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:25:30.210996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:25:30.210996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:25:30.210996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:25:30.210996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:30.210996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:30.210996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:25:30.210996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:25:30.211452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:25:30.211452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:25:30.211452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:30.211452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:30.211452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:30.211452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:25:30.211452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:30.211452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:30.211452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:30.211452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:25:30.211452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:30.211452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:25:30.211452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:25:30.211452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:30.211452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:25:30.211452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:30.211452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:25:30.211452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:25:30.211452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:30.211932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:25:30.211932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:30.211932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:25:34.950316 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:25:34.951356 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:25:34.951356 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:25:34.952609 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:25:34.989717 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:25:34.989902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:34.990543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:25:34.990543 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:34.990543 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:25:34.990543 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:25:34.990543 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:34.990543 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:25:34.990543 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:34.990543 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:25:34.991205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:34.991205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:25:34.991205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:34.991205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:25:34.991205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:25:34.991205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:25:34.991205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:25:34.991205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:34.991205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:34.991205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:34.991205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:25:34.991205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:34.991541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:34.991541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:34.991541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:25:34.991541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:34.991541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:25:34.991541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:25:34.991541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:34.991541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:25:34.991541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:34.991541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:25:34.991541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:25:34.991541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:34.991541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:25:34.991813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:34.991813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:25:34.992064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:34.992064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:25:34.992064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:25:34.992064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:25:34.992064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:25:34.992064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:25:34.992064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:34.992064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:25:34.992064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:25:34.992064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:25:34.992064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:25:34.992064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:25:34.992064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:34.992064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:34.992064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:25:34.992064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:25:34.992856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:25:34.992856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:25:34.992856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:34.992856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:34.992856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:34.992856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:25:34.992856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:34.992856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:34.992856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:34.992856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:25:34.992856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:34.992856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:25:34.992856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:25:34.992856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:34.992856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:25:34.992856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:34.992856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:25:34.992856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:25:34.992856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:34.993574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:25:34.993574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:34.993574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:25:35.163279 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:25:35.164304 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:25:35.164527 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:25:35.165683 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:25:35.166105 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:25:35.166356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:35.167116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:25:35.167116 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:35.167116 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:25:35.167116 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:25:35.167116 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:35.167116 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:25:35.167116 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:35.167116 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:25:35.168612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:35.168612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:25:35.168612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:35.168612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:25:35.168612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:25:35.168612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:25:35.168612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:25:35.168612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:35.168612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:35.168612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:35.168612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:25:35.168612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:35.168873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:35.168873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:35.168873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:25:35.168873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:35.168873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:25:35.168873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:25:35.168873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:35.168873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:25:35.168873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:35.168873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:25:35.168873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:25:35.168873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:35.168873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:25:35.170526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:35.170526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:25:35.170526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:35.170526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:25:35.170526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:25:35.170526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:25:35.170526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:25:35.170526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:25:35.170526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:35.170526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:25:35.170526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:25:35.170526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:25:35.170526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:25:35.170526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:25:35.170526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:35.170833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:35.170833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:25:35.170833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:25:35.170833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:25:35.170833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:25:35.170833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:35.170833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:35.170833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:35.170833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:25:35.170833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:35.170833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:35.170833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:35.170833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:25:35.170833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:35.170833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:25:35.170833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:25:35.170833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:35.170833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:25:35.170833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:35.171177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:25:35.171177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:25:35.171177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:35.171177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:25:35.171177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:35.171177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:25:35.172362 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:25:35.172516 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:25:35.172588 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:25:35.172715 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:25:35.173046 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:25:35.173274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:35.173798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:25:35.173798 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:35.173798 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:25:35.173798 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:25:35.173798 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:35.173798 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:25:35.173798 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:35.173798 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:25:35.174597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:35.174597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:25:35.174597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:25:35.174597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:25:35.174597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:35.174597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:25:35.174597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:35.174597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:25:35.174597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:35.174597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:25:35.174597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:35.174597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:25:35.174597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:35.174597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:25:35.174597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:25:35.175150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:35.175150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:25:35.175150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:35.175150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:25:35.175150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:25:35.175150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:35.175150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:25:35.175150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:35.175150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:25:35.175460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:35.175460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:25:35.175460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:35.175460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:25:35.175460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:25:35.175460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:25:35.175460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:25:35.175460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:25:35.175460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:25:35.175460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:25:35.175460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:35.175460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:35.176016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:35.176016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:25:35.176016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:35.176016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:35.176016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:35.176016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:25:35.176016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:35.176016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:25:35.176016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:25:35.176016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:35.176016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:25:35.176016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:35.176016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:25:35.176016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:25:35.176374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:35.176374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:25:35.176374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:35.176374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:25:35.176467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:35.176467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:25:35.176467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:25:35.176467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:25:35.176467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:25:35.176467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:25:35.176467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:35.176467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:25:35.176467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:25:35.176467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:25:35.176467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:25:35.176467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:25:35.176467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:35.176467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:35.176467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:25:35.176467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:25:35.176759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:25:35.176759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:25:35.176759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:25:35.176759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:25:35.176759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:25:35.176759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:35.176759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:35.176759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:35.176759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:25:35.176759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:35.176759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:35.176759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:35.176759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:25:35.176759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:35.176759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:25:35.176759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:25:35.176759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:35.176759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:25:35.177081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:35.177081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:25:35.177081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:25:35.177081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:35.177081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:25:35.177081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:35.177081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:25:35.197675 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:25:35.198174 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:25:35.198302 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:25:35.198519 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:25:35.198997 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:25:35.199476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:35.200190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:25:35.200190 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:35.200190 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:25:35.200190 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:25:35.200190 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:35.200190 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:25:35.200190 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:35.200190 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:25:35.201342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:35.201342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:25:35.201342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:25:35.201342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:25:35.201342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:35.201342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:25:35.201342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:35.201342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:25:35.201342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:35.201342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:25:35.201342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:35.201342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:25:35.201342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:35.201342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:25:35.201342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:25:35.201671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:35.201671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:25:35.201671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:35.201671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:25:35.201671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:25:35.201671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:35.201671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:25:35.201671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:35.201671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:25:35.202852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:35.202852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:25:35.202852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:35.202852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:25:35.202852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:25:35.202852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:25:35.202852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:25:35.202852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:25:35.202852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:25:35.202852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:35.202852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:35.203456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:35.203456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:25:35.203456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:35.203456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:35.203456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:35.203456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:25:35.203456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:35.203456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:25:35.203456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:25:35.203456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:35.203456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:25:35.203456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:35.203456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:25:35.203456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:25:35.203780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:35.203780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:25:35.203780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:35.203780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:25:35.204500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:35.204500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:25:35.204500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:25:35.204500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:25:35.204500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:25:35.204500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:25:35.204500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:35.204500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:25:35.204500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:25:35.204500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:25:35.204500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:25:35.204500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:25:35.204500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:35.204500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:35.204500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:25:35.204500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:25:35.204500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:25:35.205059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:25:35.205059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:25:35.205059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:25:35.205059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:35.205059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:35.205059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:35.205059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:25:35.205059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:35.205059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:35.205059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:35.205059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:25:35.205059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:35.205059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:25:35.205059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:25:35.205059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:35.205059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:25:35.205059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:35.205059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:25:35.205059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:25:35.205443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:35.205443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:25:35.205443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:35.205443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:25:35.207385 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:25:35.207729 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:25:35.207834 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:25:35.207990 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:25:35.208487 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:25:35.208881 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:25:35.209131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:35.209864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:25:35.209864 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:35.209864 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:25:35.209864 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:25:35.209864 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:35.209864 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:25:35.209864 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:35.209864 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:25:35.210747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:35.210747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:25:35.210747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:35.210747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:25:35.210747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:25:35.210747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:25:35.210747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:25:35.210747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:35.210747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:35.210747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:35.210747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:25:35.210747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:35.211027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:35.211027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:35.211027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:25:35.211027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:35.211027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:25:35.211027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:25:35.211027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:35.211027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:25:35.211027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:35.211027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:25:35.211027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:25:35.211027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:35.211027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:25:35.211280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:35.211280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:25:35.211757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:35.211757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:25:35.211757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:25:35.211757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:25:35.211757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:25:35.211757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:25:35.211757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:35.211757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:25:35.211757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:25:35.211757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:25:35.211757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:25:35.211757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:25:35.211757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:35.211757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:35.211757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:25:35.211757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:25:35.212085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:25:35.212085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:25:35.212085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:35.212085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:35.212085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:35.212085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:25:35.212085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:35.212085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:35.212085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:35.212085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:25:35.212085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:35.212085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:25:35.212085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:25:35.212085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:35.212085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:25:35.212085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:35.212085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:25:35.212085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:25:35.212085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:35.212449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:25:35.212449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:35.212449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:25:39.951744 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:25:39.952571 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:25:39.952571 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:25:39.953560 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:25:39.985212 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:25:39.985588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:39.986272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:25:39.986272 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:39.986272 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:25:39.986272 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:25:39.986272 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:39.986272 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:25:39.986272 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:39.986272 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:25:39.986974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:39.986974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:25:39.986974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:39.986974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:25:39.986974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:25:39.986974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:25:39.986974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:25:39.986974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:39.986974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:39.986974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:39.986974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:25:39.986974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:39.987346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:39.987346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:39.987346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:25:39.987346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:39.987346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:25:39.987346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:25:39.987346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:39.987346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:25:39.987346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:39.987346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:25:39.987346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:25:39.987346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:39.987346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:25:39.987731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:39.987731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:25:39.987731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:39.987731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:25:39.987731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:25:39.987731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:25:39.987731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:25:39.987731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:25:39.987731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:39.987731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:25:39.987731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:25:39.987731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:25:39.987731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:25:39.987731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:25:39.987731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:39.988153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:39.988153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:25:39.988153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:25:39.988153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:25:39.988153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:25:39.988153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:39.988153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:39.988153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:39.988153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:25:39.988153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:39.988153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:39.988153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:39.988153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:25:39.988153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:39.988153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:25:39.988153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:25:39.988153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:39.988153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:25:39.988153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:39.988803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:25:39.988803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:25:39.988803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:39.988803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:25:39.988803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:39.988803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:25:40.164834 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:25:40.165612 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:25:40.165992 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:25:40.167012 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:25:40.167219 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:25:40.167336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:40.167887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:25:40.167887 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:40.167887 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:25:40.167887 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:25:40.167887 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:40.167887 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:25:40.167887 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:40.167887 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:25:40.168955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:40.168955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:25:40.168955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:40.168955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:25:40.168955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:25:40.168955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:25:40.168955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:25:40.168955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:40.168955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:40.168955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:40.168955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:25:40.168955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:40.169681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:40.169681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:40.169681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:25:40.169681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:40.169681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:25:40.169681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:25:40.169681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:40.169681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:25:40.169681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:40.169681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:25:40.169681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:25:40.169681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:40.169681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:25:40.170312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:40.170312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:25:40.170312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:40.170312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:25:40.170312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:25:40.170312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:25:40.170312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:25:40.170312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:25:40.170312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:40.170312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:25:40.170312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:25:40.170312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:25:40.170312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:25:40.170312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:25:40.170312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:40.171132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:40.171132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:25:40.171132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:25:40.171132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:25:40.171132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:25:40.171132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:40.171132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:40.171132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:40.171132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:25:40.171132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:40.171132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:40.171132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:40.171132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:25:40.171132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:40.171132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:25:40.171132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:25:40.171132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:40.171132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:25:40.171132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:40.172309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:25:40.172309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:25:40.172309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:40.172309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:25:40.172309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:40.172309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:25:40.172736 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:25:40.172871 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:25:40.172948 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:25:40.173104 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:25:40.173391 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:25:40.173561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:40.173973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:25:40.173973 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:40.173973 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:25:40.173973 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:25:40.173973 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:40.173973 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:25:40.173973 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:40.173973 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:25:40.174700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:40.174700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:25:40.174700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:25:40.174700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:25:40.174700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:40.174700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:25:40.174700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:40.174700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:25:40.174700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:40.174700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:25:40.174700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:40.174700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:25:40.174700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:40.174700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:25:40.174700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:25:40.175570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:40.175570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:25:40.175570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:40.175570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:25:40.175570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:25:40.175570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:40.175570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:25:40.175570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:40.175570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:25:40.175570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:40.175570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:25:40.175570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:40.175570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:25:40.175570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:25:40.176291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:25:40.176291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:25:40.176291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:25:40.176291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:25:40.176291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:25:40.176291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:40.176291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:40.176291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:40.176291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:25:40.176291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:40.176291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:40.176291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:40.176291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:25:40.177340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:40.177340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:25:40.177340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:25:40.177340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:40.177340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:25:40.177340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:40.177340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:25:40.177340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:25:40.177340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:40.177340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:25:40.177340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:40.177340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:25:40.177618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:40.177618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:25:40.177618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:25:40.177618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:25:40.177618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:25:40.177618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:25:40.177618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:40.177618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:25:40.177618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:25:40.177618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:25:40.177618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:25:40.177618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:25:40.177618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:40.177618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:40.177618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:25:40.177618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:25:40.177960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:25:40.177960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:25:40.177960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:25:40.177960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:25:40.177960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:25:40.177960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:40.177960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:40.177960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:40.177960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:25:40.177960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:40.177960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:40.177960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:40.177960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:25:40.177960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:40.177960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:25:40.177960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:25:40.177960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:40.177960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:25:40.178335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:40.178335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:25:40.178335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:25:40.178335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:40.178335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:25:40.178335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:40.178335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:25:40.198812 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:25:40.199277 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:25:40.199439 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:25:40.199697 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:25:40.200000 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:25:40.200181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:40.200756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:25:40.200756 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:40.200756 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:25:40.200756 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:25:40.200756 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:40.200756 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:25:40.200756 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:40.200756 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:25:40.202161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:40.202161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:25:40.202161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:25:40.202161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:25:40.202161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:40.202161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:25:40.202161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:40.202161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:25:40.202161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:40.202161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:25:40.202161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:40.202161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:25:40.202161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:40.202161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:25:40.202161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:25:40.202986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:40.202986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:25:40.202986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:40.202986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:25:40.202986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:25:40.202986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:40.202986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:25:40.202986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:40.202986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:25:40.203695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:40.203695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:25:40.203695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:40.203695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:25:40.203695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:25:40.203695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:25:40.203695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:25:40.203695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:25:40.203695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:25:40.203695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:40.203695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:40.204173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:40.204173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:25:40.204173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:40.204173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:40.204173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:40.204173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:25:40.204173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:40.204173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:25:40.204173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:25:40.204173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:40.204173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:25:40.204173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:40.204173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:25:40.204173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:25:40.204601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:40.204601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:25:40.204601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:40.204601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:25:40.205350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:40.205350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:25:40.205350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:25:40.205350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:25:40.205350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:25:40.205350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:25:40.205350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:40.205350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:25:40.205350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:25:40.205350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:25:40.205350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:25:40.205350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:25:40.205350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:40.205350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:40.205350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:25:40.205350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:25:40.205350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:25:40.205908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:25:40.205908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:25:40.205908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:25:40.205908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:40.205908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:40.205908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:40.205908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:25:40.205908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:40.205908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:40.205908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:40.205908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:25:40.205908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:40.205908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:25:40.205908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:25:40.205908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:40.205908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:25:40.205908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:40.205908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:25:40.205908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:25:40.206517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:40.206517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:25:40.206517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:40.206517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:25:40.208550 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:25:40.208997 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:25:40.209176 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:25:40.209422 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:25:40.209705 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:25:40.209956 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:25:40.209956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:40.210367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:25:40.210367 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:40.210367 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:25:40.210367 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:25:40.210367 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:40.210367 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:25:40.210367 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:40.210367 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:25:40.211198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:40.211198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:25:40.211198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:40.211198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:25:40.211198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:25:40.211198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:25:40.211198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:25:40.211198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:40.211198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:40.211198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:40.211198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:25:40.211198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:40.211660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:40.211660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:40.211660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:25:40.211660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:40.211660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:25:40.211660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:25:40.211660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:40.211660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:25:40.211660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:40.211660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:25:40.211660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:25:40.211660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:40.211660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:25:40.212046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:40.212046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:25:40.212197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:40.212197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:25:40.212197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:25:40.212197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:25:40.212197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:25:40.212197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:25:40.212197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:40.212197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:25:40.212197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:25:40.212197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:25:40.212197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:25:40.212197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:25:40.212197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:40.212197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:40.212197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:25:40.212197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:25:40.212745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:25:40.212745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:25:40.212745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:40.212745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:40.212745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:40.212745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:25:40.212745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:40.212745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:40.212745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:40.212745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:25:40.212745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:40.212745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:25:40.212745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:25:40.212745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:40.212745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:25:40.212745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:40.212745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:25:40.212745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:25:40.212745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:40.213334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:25:40.213334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:40.213334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:25:44.952004 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:25:44.952844 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:25:44.952844 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:25:44.954110 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:25:44.988100 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:25:44.988360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:44.989238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:25:44.989238 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:44.989238 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:25:44.989238 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:25:44.989238 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:44.989238 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:25:44.989238 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:44.989238 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:25:44.990067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:44.990067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:25:44.990067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:44.990067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:25:44.990067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:25:44.990067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:25:44.990067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:25:44.990067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:44.990067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:44.990067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:44.990067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:25:44.990067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:44.991443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:44.991443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:44.991443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:25:44.991443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:44.991443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:25:44.991443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:25:44.991443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:44.991443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:25:44.991443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:44.991443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:25:44.991443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:25:44.991443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:44.991443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:25:44.992091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:44.992091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:25:44.992091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:44.992091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:25:44.992091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:25:44.992091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:25:44.992091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:25:44.992091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:25:44.992091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:44.992091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:25:44.992091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:25:44.992091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:25:44.992091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:25:44.992091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:25:44.992091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:44.992846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:44.992846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:25:44.992846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:25:44.992846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:25:44.992846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:25:44.992846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:44.992846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:44.992846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:44.992846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:25:44.992846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:44.992846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:44.992846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:44.992846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:25:44.992846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:44.992846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:25:44.992846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:25:44.992846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:44.992846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:25:44.992846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:44.993991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:25:44.993991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:25:44.993991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:44.993991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:25:44.993991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:44.993991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:25:45.166235 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:25:45.167203 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:25:45.167513 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:25:45.168882 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:25:45.169253 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:25:45.169541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:45.170440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:25:45.170440 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:45.170440 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:25:45.170440 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:25:45.170440 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:45.170440 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:25:45.170440 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:45.170440 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:25:45.171950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:45.171950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:25:45.171950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:45.171950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:25:45.171950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:25:45.171950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:25:45.171950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:25:45.171950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:45.171950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:45.171950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:45.171950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:25:45.171950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:45.173502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:45.173502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:45.173502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:25:45.173502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:45.173502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:25:45.173502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:25:45.173502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:45.173502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:25:45.173502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:45.173502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:25:45.173502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:25:45.173502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:45.173502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:25:45.174197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:45.174197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:25:45.174197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:45.174197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:25:45.174197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:25:45.174197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:25:45.174197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:25:45.174197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:25:45.174197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:45.174197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:25:45.174197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:25:45.174197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:25:45.174197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:25:45.174197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:25:45.174197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:45.174851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:45.174851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:25:45.174851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:25:45.174851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:25:45.174851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:25:45.174851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:45.174851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:45.174851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:45.174851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:25:45.174851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:45.174851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:45.174851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:45.174851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:25:45.174851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:45.174851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:25:45.174851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:25:45.174851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:45.174851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:25:45.174851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:45.175869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:25:45.175869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:25:45.175869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:45.175869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:25:45.175869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:45.175869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:25:45.175869 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:25:45.175869 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:25:45.175869 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:25:45.175869 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:25:45.176900 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:25:45.176900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:45.176900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:25:45.176900 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:45.176900 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:25:45.176900 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:25:45.176900 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:45.176900 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:25:45.176900 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:45.176900 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:25:45.177625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:45.177625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:25:45.177625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:25:45.177625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:25:45.177625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:45.177625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:25:45.177625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:45.177625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:25:45.177625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:45.177625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:25:45.177625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:45.177625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:25:45.177625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:45.177625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:25:45.177625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:25:45.178664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:45.178664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:25:45.178664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:45.178664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:25:45.178664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:25:45.178664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:45.178664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:25:45.178664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:45.178664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:25:45.179325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:45.179325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:25:45.179325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:45.179325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:25:45.179325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:25:45.179325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:25:45.179325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:25:45.179325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:25:45.179325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:25:45.179325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:25:45.179325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:45.179325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:45.179786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:45.179786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:25:45.179786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:45.179786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:45.179786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:45.179786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:25:45.179786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:45.179786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:25:45.179786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:25:45.179786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:45.179786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:25:45.179786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:45.179786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:25:45.179786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:25:45.180187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:45.180187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:25:45.180187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:45.180187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:25:45.180789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:45.180789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:25:45.180789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:25:45.180789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:25:45.180789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:25:45.180789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:25:45.180789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:45.180789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:25:45.180789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:25:45.180789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:25:45.180789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:25:45.180789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:25:45.180789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:45.180789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:45.180789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:25:45.180789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:25:45.181630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:25:45.181630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:25:45.181630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:25:45.181630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:25:45.181630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:25:45.181630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:45.181630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:45.181630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:45.181630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:25:45.181630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:45.181630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:45.181630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:45.181630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:25:45.181630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:45.181630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:25:45.181630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:25:45.181630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:45.181630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:25:45.182662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:45.182662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:25:45.182662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:25:45.182662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:45.182662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:25:45.182662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:45.182662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:25:45.199415 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:25:45.199985 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:25:45.200201 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:25:45.200379 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:25:45.200938 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:25:45.201589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:45.202175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:25:45.202175 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:45.202175 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:25:45.202175 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:25:45.202175 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:45.202175 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:25:45.202175 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:45.202175 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:25:45.203484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:45.203484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:25:45.203484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:25:45.203484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:25:45.203484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:45.203484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:25:45.203484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:45.203484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:25:45.203484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:45.203484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:25:45.203484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:45.203484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:25:45.203484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:45.203484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:25:45.203484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:25:45.204002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:45.204002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:25:45.204002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:45.204002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:25:45.204002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:25:45.204002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:45.204002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:25:45.204002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:45.204002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:25:45.205225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:45.205225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:25:45.205225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:45.205225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:25:45.205225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:25:45.205225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:25:45.205225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:25:45.205225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:25:45.205225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:25:45.205225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:45.205225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:45.205900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:45.205900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:25:45.205900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:45.205900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:45.205900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:45.205900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:25:45.205900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:45.205900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:25:45.205900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:25:45.205900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:45.205900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:25:45.205900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:45.205900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:25:45.205900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:25:45.206573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:45.206573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:25:45.206573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:45.206573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:25:45.207137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:45.207137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:25:45.207137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:25:45.207137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:25:45.207137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:25:45.207137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:25:45.207137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:45.207137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:25:45.207137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:25:45.207137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:25:45.207137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:25:45.207137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:25:45.207137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:45.207137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:45.207137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:25:45.207137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:25:45.207137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:25:45.208040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:25:45.208040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:25:45.208040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:25:45.208040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:45.208040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:45.208040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:45.208040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:25:45.208040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:45.208040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:45.208040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:45.208040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:25:45.208040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:45.208040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:25:45.208040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:25:45.208040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:45.208040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:25:45.208040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:45.208040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:25:45.208040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:25:45.208919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:45.208919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:25:45.208919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:45.208919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:25:45.209137 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:25:45.209428 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:25:45.209597 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:25:45.209826 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:25:45.210250 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:25:45.210633 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:25:45.210857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:45.211566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:25:45.211566 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:45.211566 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:25:45.211566 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:25:45.211566 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:45.211566 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:25:45.211566 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:45.211566 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:25:45.212379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:45.212379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:25:45.212379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:45.212379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:25:45.212379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:25:45.212379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:25:45.212379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:25:45.212379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:45.212379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:45.212379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:45.212379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:25:45.212379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:45.212760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:45.212760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:45.212760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:25:45.212760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:45.212760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:25:45.212760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:25:45.212760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:45.212760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:25:45.212760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:45.212760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:25:45.212760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:25:45.212760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:45.212760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:25:45.213121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:45.213121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:25:45.213481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:45.213481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:25:45.213481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:25:45.213481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:25:45.213481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:25:45.213481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:25:45.213481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:45.213481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:25:45.213481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:25:45.213481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:25:45.213481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:25:45.213481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:25:45.213481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:45.213481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:45.213481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:25:45.213481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:25:45.213935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:25:45.213935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:25:45.213935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:45.213935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:45.213935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:45.213935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:25:45.213935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:45.213935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:45.213935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:45.213935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:25:45.213935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:45.213935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:25:45.213935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:25:45.213935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:45.213935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:25:45.213935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:45.213935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:25:45.213935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:25:45.213935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:45.214463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:25:45.214463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:45.214463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:25:49.952652 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:25:49.953364 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:25:49.953469 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:25:49.954804 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:25:49.989773 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:25:49.990104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:49.990825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:25:49.990825 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:49.990825 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:25:49.990825 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:25:49.990825 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:49.990825 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:25:49.990825 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:49.990825 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:25:49.991789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:49.991789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:25:49.991789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:49.991789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:25:49.991789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:25:49.991789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:25:49.991789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:25:49.991789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:49.991789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:49.991789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:49.991789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:25:49.991789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:49.992328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:49.992328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:49.992328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:25:49.992328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:49.992328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:25:49.992328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:25:49.992328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:49.992328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:25:49.992328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:49.992328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:25:49.992328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:25:49.992328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:49.992328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:25:49.992879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:49.992879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:25:49.992879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:49.992879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:25:49.992879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:25:49.992879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:25:49.992879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:25:49.992879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:25:49.992879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:49.992879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:25:49.992879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:25:49.992879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:25:49.992879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:25:49.992879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:25:49.992879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:49.993604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:49.993604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:25:49.993604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:25:49.993604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:25:49.993604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:25:49.993604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:49.993604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:49.993604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:49.993604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:25:49.993604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:49.993604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:49.993604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:49.993604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:25:49.993604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:49.993604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:25:49.993604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:25:49.993604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:49.993604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:25:49.993604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:49.994458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:25:49.994458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:25:49.994458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:49.994458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:25:49.994458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:49.994458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:25:50.167125 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:25:50.167612 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:25:50.167699 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:25:50.168599 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:25:50.168951 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:25:50.169028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:50.169600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:25:50.169600 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:50.169600 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:25:50.169600 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:25:50.169600 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:50.169600 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:25:50.169600 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:50.169600 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:25:50.173447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:50.173447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:25:50.173447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:50.173447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:25:50.173447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:25:50.173447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:25:50.173447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:25:50.173447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:50.173447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:50.173447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:50.173447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:25:50.173447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:50.174151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:50.174151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:50.174151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:25:50.174151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:50.174151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:25:50.174151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:25:50.174151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:50.174151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:25:50.174151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:50.174151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:25:50.174151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:25:50.174151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:50.174151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:25:50.174776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:50.174776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:25:50.174776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:50.174776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:25:50.174776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:25:50.174776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:25:50.174776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:25:50.174776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:25:50.174776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:50.174776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:25:50.174776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:25:50.174776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:25:50.174776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:25:50.174776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:25:50.174776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:50.175529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:50.175529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:25:50.175529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:25:50.175529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:25:50.175529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:25:50.175529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:50.175529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:50.175529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:50.175529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:25:50.175529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:50.175529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:50.175529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:50.175529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:25:50.175529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:50.175529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:25:50.175529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:25:50.175529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:50.175529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:25:50.175529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:50.176658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:25:50.176658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:25:50.176658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:50.176658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:25:50.176658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:50.176658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:25:50.176658 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:25:50.176658 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:25:50.176658 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:25:50.176658 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:25:50.177164 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:25:50.177164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:50.177164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:25:50.177164 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:50.177164 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:25:50.177164 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:25:50.177164 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:50.177164 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:25:50.177164 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:50.177164 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:25:50.177164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:50.177164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:25:50.177164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:25:50.177164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:25:50.177872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:50.177872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:25:50.177872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:50.177872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:25:50.177872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:50.177872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:25:50.177872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:50.177872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:25:50.177872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:50.177872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:25:50.177872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:25:50.177872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:50.177872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:25:50.177872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:50.177872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:25:50.177872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:25:50.177872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:50.177872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:25:50.178433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:50.178433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:25:50.178433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:50.178433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:25:50.178433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:50.178433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:25:50.178433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:25:50.178433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:25:50.178433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:25:50.178433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:25:50.178433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:25:50.178433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:25:50.178694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:50.178694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:50.178694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:50.178694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:25:50.178694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:50.178694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:50.178694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:50.178694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:25:50.178694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:50.178694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:25:50.178694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:25:50.178694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:50.178694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:25:50.178694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:50.178944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:25:50.178944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:25:50.178944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:50.178944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:25:50.178944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:50.178944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:25:50.178944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:50.178944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:25:50.178944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:25:50.178944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:25:50.178944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:25:50.178944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:25:50.178944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:50.178944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:25:50.179219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:25:50.179219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:25:50.179219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:25:50.179219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:25:50.179219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:50.179219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:50.179219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:25:50.179219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:25:50.179219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:25:50.179219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:25:50.179219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:25:50.179219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:25:50.179219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:25:50.179219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:50.179219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:50.179219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:50.179219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:25:50.179219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:50.179219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:50.179898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:50.179898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:25:50.179898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:50.179898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:25:50.179898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:25:50.179898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:50.179898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:25:50.179898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:50.179898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:25:50.179898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:25:50.179898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:50.179898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:25:50.179898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:50.179898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:25:50.199793 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:25:50.200177 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:25:50.200471 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:25:50.200745 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:25:50.201191 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:25:50.201453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:50.202125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:25:50.202125 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:50.202125 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:25:50.202125 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:25:50.202125 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:50.202125 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:25:50.202125 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:50.202125 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:25:50.203483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:50.203483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:25:50.203483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:25:50.203483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:25:50.203483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:50.203483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:25:50.203483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:50.203483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:25:50.203483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:50.203483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:25:50.203483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:50.203483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:25:50.203483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:50.203483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:25:50.203483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:25:50.204449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:50.204449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:25:50.204449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:50.204449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:25:50.204449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:25:50.204449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:50.204449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:25:50.204449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:50.204449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:25:50.204879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:50.204879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:25:50.204879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:50.204879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:25:50.204879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:25:50.204879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:25:50.204879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:25:50.204879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:25:50.204879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:25:50.204879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:50.204879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:50.205373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:50.205373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:25:50.205373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:50.205373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:50.205373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:50.205373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:25:50.205373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:50.205373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:25:50.205373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:25:50.205373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:50.205373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:25:50.205373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:50.205373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:25:50.205373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:25:50.206040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:50.206040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:25:50.206040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:50.206040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:25:50.206846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:50.206846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:25:50.206846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:25:50.206846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:25:50.206846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:25:50.206846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:25:50.206846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:50.206846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:25:50.206846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:25:50.206846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:25:50.206846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:25:50.206846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:25:50.206846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:50.206846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:50.206846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:25:50.206846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:25:50.206846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:25:50.207809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:25:50.207809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:25:50.207809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:25:50.207809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:50.207809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:50.207809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:50.207809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:25:50.207809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:50.207809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:50.207809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:50.207809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:25:50.207809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:50.207809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:25:50.207809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:25:50.207809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:50.207809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:25:50.207809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:50.207809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:25:50.207809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:25:50.208429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:50.208429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:25:50.208429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:50.208429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:25:50.209415 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:25:50.209840 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:25:50.210052 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:25:50.210204 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:25:50.210544 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:25:50.210782 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:25:50.210929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:50.211418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:25:50.211418 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:50.211418 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:25:50.211418 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:25:50.211418 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:50.211418 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:25:50.211418 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:50.211418 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:25:50.212197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:50.212197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:25:50.212197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:50.212197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:25:50.212197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:25:50.212197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:25:50.212197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:25:50.212197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:50.212197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:50.212197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:50.212197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:25:50.212197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:50.212607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:50.212607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:50.212607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:25:50.212607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:50.212607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:25:50.212607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:25:50.212607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:50.212607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:25:50.212607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:50.212607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:25:50.212607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:25:50.212607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:50.212607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:25:50.212978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:50.212978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:25:50.213172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:50.213172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:25:50.213172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:25:50.213172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:25:50.213172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:25:50.213172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:25:50.213172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:50.213172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:25:50.213172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:25:50.213172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:25:50.213172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:25:50.213172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:25:50.213172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:50.213172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:50.213172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:25:50.213172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:25:50.213863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:25:50.213863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:25:50.213863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:50.213863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:50.213863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:50.213863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:25:50.213863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:50.213863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:50.213863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:50.213863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:25:50.213863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:50.213863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:25:50.213863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:25:50.213863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:50.213863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:25:50.213863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:50.213863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:25:50.213863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:25:50.213863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:50.214595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:25:50.214595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:50.214595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:25:54.953335 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:25:54.954150 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:25:54.954345 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:25:54.955545 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:25:54.991940 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:25:54.992932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:54.993201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:25:54.993201 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:54.993201 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:25:54.993201 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:25:54.993201 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:54.993201 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:25:54.993201 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:54.993201 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:25:54.994224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:54.994224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:25:54.994224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:54.994224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:25:54.994224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:25:54.994224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:25:54.994224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:25:54.994224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:54.994224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:54.994224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:54.994224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:25:54.994224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:54.994894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:54.994894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:54.994894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:25:54.994894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:54.994894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:25:54.994894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:25:54.994894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:54.994894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:25:54.994894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:54.994894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:25:54.994894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:25:54.994894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:54.994894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:25:54.995572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:54.995572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:25:54.995572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:54.995572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:25:54.995572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:25:54.995572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:25:54.995572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:25:54.995572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:25:54.995572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:54.995572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:25:54.995572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:25:54.995572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:25:54.995572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:25:54.995572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:25:54.995572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:54.996341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:54.996341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:25:54.996341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:25:54.996341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:25:54.996341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:25:54.996341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:54.996341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:54.996341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:54.996341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:25:54.996341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:54.996341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:54.996341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:54.996341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:25:54.996341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:54.996341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:25:54.996341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:25:54.996341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:54.996341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:25:54.996341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:54.997291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:25:54.997291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:25:54.997291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:54.997291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:25:54.997291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:54.997291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:25:55.167902 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:25:55.168450 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:25:55.168966 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:25:55.169902 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:25:55.170241 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:25:55.170508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:55.171353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:25:55.171353 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:55.171353 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:25:55.171353 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:25:55.171353 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:55.171353 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:25:55.171353 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:55.171353 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:25:55.173342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:55.173342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:25:55.173342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:55.173342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:25:55.173342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:25:55.173342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:25:55.173342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:25:55.173342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:55.173342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:55.173342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:55.173342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:25:55.173342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:55.173988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:55.173988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:55.173988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:25:55.173988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:55.173988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:25:55.173988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:25:55.173988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:55.173988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:25:55.173988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:55.173988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:25:55.173988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:25:55.173988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:55.173988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:25:55.174618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:55.174618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:25:55.175004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:55.175004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:25:55.175004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:25:55.175004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:25:55.175004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:25:55.175004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:25:55.175004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:55.175004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:25:55.175004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:25:55.175004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:25:55.175004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:25:55.175004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:25:55.175004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:55.175004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:55.175004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:25:55.175004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:25:55.175819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:25:55.175819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:25:55.175819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:55.175819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:55.175819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:55.175819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:25:55.175819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:55.175819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:55.175819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:55.175819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:25:55.175819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:55.175819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:25:55.175819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:25:55.175819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:55.175819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:25:55.175819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:55.175819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:25:55.175819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:25:55.175819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:55.176865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:25:55.176865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:55.176865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:25:55.176865 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:25:55.176865 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:25:55.176865 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:25:55.176865 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:25:55.176865 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:25:55.177304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:55.177644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:25:55.177644 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:55.177644 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:25:55.177644 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:25:55.177644 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:55.177644 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:25:55.177644 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:55.177644 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:25:55.178535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:55.178535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:25:55.178535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:25:55.178535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:25:55.178535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:55.178535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:25:55.178535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:55.178535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:25:55.178535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:55.178535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:25:55.178535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:55.178535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:25:55.178535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:55.178535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:25:55.178535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:25:55.179295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:55.179295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:25:55.179295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:55.179295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:25:55.179295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:25:55.179295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:55.179295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:25:55.179295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:55.179295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:25:55.179295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:55.179295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:25:55.179295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:55.179295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:25:55.179295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:25:55.180041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:25:55.180041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:25:55.180041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:25:55.180041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:25:55.180041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:25:55.180041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:55.180041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:55.180041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:55.180041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:25:55.180041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:55.180041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:55.180041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:55.180041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:25:55.180754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:55.180754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:25:55.180754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:25:55.180754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:55.180754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:25:55.180754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:55.180754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:25:55.180754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:25:55.180754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:55.180754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:25:55.180754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:55.180754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:25:55.181239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:55.181239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:25:55.181239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:25:55.181239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:25:55.181239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:25:55.181239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:25:55.181239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:55.181239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:25:55.181239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:25:55.181239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:25:55.181239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:25:55.181239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:25:55.181239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:55.181239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:55.181239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:25:55.181239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:25:55.182076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:25:55.182076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:25:55.182076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:25:55.182076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:25:55.182076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:25:55.182076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:55.182076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:55.182076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:55.182076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:25:55.182076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:55.182076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:55.182076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:55.182076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:25:55.182076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:55.182076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:25:55.182076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:25:55.182076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:55.182076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:25:55.183488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:55.183488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:25:55.183488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:25:55.183488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:55.183488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:25:55.183488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:55.183488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:25:55.200279 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:25:55.200966 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:25:55.201370 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:25:55.202233 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:25:55.202791 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:25:55.203134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:55.204070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:25:55.204070 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:55.204070 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:25:55.204070 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:25:55.204070 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:55.204070 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:25:55.204070 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:55.204070 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:25:55.205379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:55.205379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:25:55.205379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:25:55.205379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:25:55.205379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:55.205379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:25:55.205379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:55.205379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:25:55.205379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:55.205379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:25:55.205379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:55.205379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:25:55.205379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:55.205379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:25:55.205379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:25:55.206070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:55.206070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:25:55.206070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:55.206070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:25:55.206070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:25:55.206070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:55.206070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:25:55.206070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:55.206070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:25:55.206958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:55.206958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:25:55.206958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:55.206958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:25:55.206958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:25:55.206958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:25:55.206958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:25:55.206958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:25:55.206958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:25:55.206958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:55.206958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:55.207504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:55.207504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:25:55.207504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:55.207504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:55.207504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:55.207504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:25:55.207504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:55.207504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:25:55.207504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:25:55.207504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:55.207504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:25:55.207504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:55.207504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:25:55.207504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:25:55.208047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:55.208047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:25:55.208047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:55.208047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:25:55.209003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:55.209003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:25:55.209003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:25:55.209003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:25:55.209003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:25:55.209003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:25:55.209003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:55.209003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:25:55.209003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:25:55.209003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:25:55.209003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:25:55.209003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:25:55.209003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:55.209003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:55.209003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:25:55.209003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:25:55.209003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:25:55.209582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:25:55.209582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:25:55.209582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:25:55.209582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:55.209582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:55.209582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:55.209582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:25:55.209582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:55.209582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:55.209582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:55.209582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:25:55.209582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:55.209582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:25:55.209582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:25:55.209582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:55.209582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:25:55.209582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:55.209582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:25:55.209582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:25:55.210146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:55.210146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:25:55.210146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:55.210146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:25:55.210146 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:25:55.210434 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:25:55.210678 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:25:55.210925 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:25:55.211458 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:25:55.211820 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:25:55.212053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:55.212887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:25:55.212887 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:55.212887 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:25:55.212887 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:25:55.212887 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:55.212887 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:25:55.212887 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:55.212887 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:25:55.213798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:55.213798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:25:55.213798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:55.213798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:25:55.213798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:25:55.213798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:25:55.213798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:25:55.213798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:55.213798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:55.213798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:55.213798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:25:55.213798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:55.214148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:55.214148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:55.214148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:25:55.214148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:55.214148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:25:55.214148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:25:55.214148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:55.214148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:25:55.214148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:55.214148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:25:55.214148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:25:55.214148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:55.214148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:25:55.214515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:55.214515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:25:55.214787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:55.214787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:25:55.214787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:25:55.214787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:25:55.214787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:25:55.214787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:25:55.214787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:55.214787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:25:55.214787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:25:55.214787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:25:55.214787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:25:55.214787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:25:55.214787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:25:55.214787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:55.214787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:25:55.214787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:25:55.215225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:25:55.215225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:25:55.215225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:55.215225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:55.215225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:55.215225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:25:55.215225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:55.215225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:25:55.215225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:25:55.215225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:25:55.215225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:25:55.215225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:25:55.215225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:25:55.215225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:25:55.215225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:25:55.215225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:55.215225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:25:55.215225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:25:55.215225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:55.215758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:25:55.215758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:55.215758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:25:59.954201 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:25:59.956608 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:25:59.956608 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:25:59.956608 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:25:59.997609 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:25:59.997923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:59.998555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:25:59.998555 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:25:59.998555 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:25:59.998555 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:25:59.998555 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:25:59.998555 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:25:59.998555 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:59.998555 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:25:59.999638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:25:59.999638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:25:59.999638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:25:59.999638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:25:59.999638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:25:59.999638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:25:59.999638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:25:59.999638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:25:59.999638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:25:59.999638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:25:59.999638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:25:59.999638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:00.000018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:00.000018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:00.000018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:26:00.000018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:00.000018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:26:00.000018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:26:00.000018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:00.000018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:26:00.000018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:00.000018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:26:00.000018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:26:00.000018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:00.000018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:26:00.000359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:00.000359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:26:00.000714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:00.000714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:26:00.000714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:26:00.000714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:26:00.000714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:26:00.000714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:26:00.000714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:00.000714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:26:00.000714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:26:00.000714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:26:00.000714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:26:00.000714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:26:00.000714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:00.000714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:00.000714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:26:00.000714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:26:00.001156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:26:00.001156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:26:00.001156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:00.001156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:00.001156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:00.001156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:26:00.001156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:00.001156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:00.001156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:00.001156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:26:00.001156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:00.001156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:26:00.001156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:26:00.001156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:00.001156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:26:00.001156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:00.001156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:26:00.001156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:26:00.001156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:00.001724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:26:00.001724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:00.001724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:26:00.168481 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:26:00.168759 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:26:00.168886 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:26:00.169728 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:26:00.169983 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:26:00.170153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:00.170783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:26:00.170783 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:00.170783 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:26:00.170783 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:26:00.170783 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:00.170783 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:26:00.170783 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:00.170783 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:26:00.171678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:00.171678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:26:00.171678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:00.171678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:26:00.171678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:26:00.171678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:26:00.171678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:26:00.171678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:00.171678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:00.171678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:00.171678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:26:00.171678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:00.171977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:00.171977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:00.171977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:26:00.171977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:00.171977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:26:00.171977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:26:00.171977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:00.171977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:26:00.171977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:00.171977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:26:00.171977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:26:00.171977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:00.171977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:26:00.172258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:00.172258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:26:00.172563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:00.172563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:26:00.172563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:26:00.172563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:26:00.172563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:26:00.172563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:26:00.172563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:00.172563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:26:00.172563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:26:00.172563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:26:00.172563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:26:00.172563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:26:00.172563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:00.172563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:00.172563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:26:00.172563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:26:00.172930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:26:00.172930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:26:00.172930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:00.172930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:00.172930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:00.172930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:26:00.172930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:00.172930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:00.172930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:00.172930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:26:00.172930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:00.172930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:26:00.172930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:26:00.172930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:00.172930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:26:00.172930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:00.172930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:26:00.172930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:26:00.172930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:00.173347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:26:00.173347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:00.173347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:26:00.175590 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:26:00.175857 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:26:00.175976 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:26:00.176101 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:26:00.176498 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:26:00.176709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:00.177219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:26:00.177219 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:00.177219 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:26:00.177219 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:26:00.177219 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:00.177219 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:26:00.177219 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:00.177219 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:26:00.178075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:00.178075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:26:00.178075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:26:00.178075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:26:00.178075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:00.178075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:26:00.178075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:00.178075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:26:00.178075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:00.178075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:26:00.178075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:00.178075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:26:00.178075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:00.178075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:26:00.178075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:26:00.178444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:00.178444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:26:00.178444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:00.178444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:26:00.178444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:26:00.178444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:00.178444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:26:00.178444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:00.178444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:26:00.179434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:00.179434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:26:00.179434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:00.179434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:26:00.179434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:26:00.179434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:26:00.179434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:26:00.179434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:26:00.179434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:26:00.179434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:26:00.179434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:00.179434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:00.179696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:00.179696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:26:00.179696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:00.179696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:00.179696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:00.179696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:26:00.179696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:00.179696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:26:00.179696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:26:00.179696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:00.179696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:26:00.179696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:00.179696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:26:00.179696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:26:00.179949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:00.179949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:26:00.179949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:00.179949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:26:00.179949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:00.179949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:26:00.179949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:26:00.179949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:26:00.179949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:26:00.179949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:26:00.179949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:00.179949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:26:00.179949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:26:00.179949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:26:00.180231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:26:00.180231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:26:00.180231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:00.180231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:00.180231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:26:00.180231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:26:00.180231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:26:00.180231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:26:00.180231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:26:00.180231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:26:00.180231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:26:00.180231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:00.180231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:00.180231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:00.180231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:26:00.180231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:00.180231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:00.180231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:00.180231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:26:00.180626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:00.180626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:26:00.180626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:26:00.180626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:00.180626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:26:00.180626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:00.180626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:26:00.180626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:26:00.180626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:00.180626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:26:00.180626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:00.180626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:26:00.201084 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:26:00.201551 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:26:00.201681 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:26:00.201818 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:26:00.203511 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:26:00.203721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:00.204316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:26:00.204316 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:00.204316 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:26:00.204316 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:26:00.204316 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:00.204316 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:26:00.204316 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:00.204316 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:26:00.205771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:00.205771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:26:00.205771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:26:00.205771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:26:00.205771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:00.205771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:26:00.205771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:00.205771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:26:00.205771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:00.205771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:26:00.205771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:00.205771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:26:00.205771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:00.205771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:26:00.205771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:26:00.206087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:00.206087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:26:00.206087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:00.206087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:26:00.206087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:26:00.206087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:00.206087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:26:00.206087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:00.206087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:26:00.207038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:00.207038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:26:00.207038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:00.207038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:26:00.207038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:26:00.207038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:26:00.207038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:26:00.207038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:26:00.207038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:26:00.207038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:00.207038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:00.207305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:00.207305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:26:00.207305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:00.207305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:00.207305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:00.207305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:26:00.207305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:00.207305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:26:00.207305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:26:00.207305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:00.207305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:26:00.207305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:00.207305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:26:00.207305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:26:00.207570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:00.207570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:26:00.207570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:00.207570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:26:00.209761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:00.209761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:26:00.209761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:26:00.209761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:26:00.209761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:26:00.209761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:26:00.209761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:00.209761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:26:00.209761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:26:00.209761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:26:00.209761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:26:00.209761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:26:00.209761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:00.209761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:00.209761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:26:00.209761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:26:00.209761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:26:00.210142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:26:00.210142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:26:00.210142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:26:00.210142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:00.210142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:00.210142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:00.210142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:26:00.210142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:00.210142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:00.210142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:00.210142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:26:00.210142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:00.210142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:26:00.210142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:26:00.210142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:00.210142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:26:00.210142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:00.210142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:26:00.210142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:26:00.210521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:00.210521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:26:00.210521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:00.210521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:26:00.210521 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:26:00.210975 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:26:00.210975 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:26:00.211187 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:26:00.211747 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:26:00.212016 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:26:00.212176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:00.212782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:26:00.212782 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:00.212782 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:26:00.212782 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:26:00.212782 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:00.212782 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:26:00.212782 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:00.212782 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:26:00.213863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:00.213863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:26:00.213863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:00.213863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:26:00.213863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:26:00.213863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:26:00.213863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:26:00.213863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:00.213863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:00.213863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:00.213863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:26:00.213863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:00.214131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:00.214131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:00.214131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:26:00.214131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:00.214131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:26:00.214131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:26:00.214131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:00.214131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:26:00.214131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:00.214131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:26:00.214131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:26:00.214131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:00.214131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:26:00.214358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:00.214358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:26:00.214358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:00.214358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:26:00.214358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:26:00.214358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:26:00.214358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:26:00.214358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:26:00.214358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:00.214358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:26:00.214358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:26:00.214358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:26:00.214358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:26:00.214358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:26:00.214358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:00.214655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:00.214655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:26:00.214655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:26:00.214655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:26:00.214655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:26:00.214655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:00.214655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:00.214655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:00.214655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:26:00.214655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:00.214655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:00.214655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:00.214655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:26:00.214655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:00.214655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:26:00.214655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:26:00.214655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:00.214655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:26:00.214655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:00.214992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:26:00.214992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:26:00.214992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:00.214992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:26:00.214992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:00.214992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:26:04.955468 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:26:04.956100 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:26:04.956100 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:26:04.957069 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:26:05.038701 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:26:05.038969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:05.039454 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:26:05.039454 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:05.039454 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:26:05.039454 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:26:05.039454 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:05.039454 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:26:05.039454 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:05.039454 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:26:05.040277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.084 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:05.040277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:26:05.040277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:05.040277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:26:05.040277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:26:05.040277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:26:05.040277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:26:05.040277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:05.040277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:05.040277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:05.040277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:26:05.040277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:05.040656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:05.040656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:05.040656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:26:05.040656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:05.040656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:26:05.040656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:26:05.040656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:05.040656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:26:05.040656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:05.040656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:26:05.040656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:26:05.040656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:05.040656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:26:05.041001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:05.041001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:26:05.041419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:05.041419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:26:05.041419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:26:05.041419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:26:05.041419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:26:05.041419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:26:05.041419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:05.041419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:26:05.041419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:26:05.041419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:26:05.041419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:26:05.041419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:26:05.041419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:05.041419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:05.041419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:26:05.041419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:26:05.042673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:26:05.042673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:26:05.042673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:05.042673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:05.042673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:05.042673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:26:05.042673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:05.042673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:05.042673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:05.042673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:26:05.042673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:05.042673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:26:05.042673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:26:05.042673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:05.042673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:26:05.042673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:05.042673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:26:05.042673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:26:05.042673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:05.043177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:26:05.043177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:05.043177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:26:05.169434 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:26:05.169434 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:26:05.169624 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:26:05.170552 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:26:05.170733 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:26:05.170838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:05.171392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:26:05.171392 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:05.171392 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:26:05.171392 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:26:05.171392 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:05.171392 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:26:05.171392 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:05.171392 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:26:05.172600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:05.172600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:26:05.172600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:05.172600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:26:05.172600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:26:05.172600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:26:05.172600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:26:05.172600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:05.172600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:05.172600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:05.172600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:26:05.172600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:05.173341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:05.173341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:05.173341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:26:05.173341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:05.173341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:26:05.173341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:26:05.173341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:05.173341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:26:05.173341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:05.173341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:26:05.173341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:26:05.173341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:05.173341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:26:05.173941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:05.173941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:26:05.173941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:05.173941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:26:05.173941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:26:05.173941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:26:05.173941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:26:05.173941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:26:05.173941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:05.173941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:26:05.173941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:26:05.173941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:26:05.173941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:26:05.173941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:26:05.173941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:05.174319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:05.174319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:26:05.174319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:26:05.174319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:26:05.174319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:26:05.174319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:05.174319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:05.174319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:05.174319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:26:05.174319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:05.174319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:05.174319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:05.174319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:26:05.174319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:05.174319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:26:05.174319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:26:05.174319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:05.174319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:26:05.174319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:05.174828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:26:05.174828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:26:05.174828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:05.174828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:26:05.174828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:05.174828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:26:05.175938 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:26:05.176301 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:26:05.176475 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:26:05.176673 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:26:05.176979 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:26:05.177129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:05.177625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:26:05.177625 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:05.177625 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:26:05.177625 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:26:05.177625 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:05.177625 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:26:05.177625 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:05.177625 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:26:05.178748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:05.178748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:26:05.178748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:26:05.178748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:26:05.178748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:05.178748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:26:05.178748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:05.178748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:26:05.178748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:05.178748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:26:05.178748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:05.178748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:26:05.178748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:05.178748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:26:05.178748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:26:05.179047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:05.179047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:26:05.179047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:05.179047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:26:05.179047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:26:05.179047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:05.179047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:26:05.179047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:05.179047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:26:05.180237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:05.180237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:26:05.180237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:05.180237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:26:05.180237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:26:05.180237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:26:05.180237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:26:05.180237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:26:05.180237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:26:05.180237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:26:05.180237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:05.180237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:05.180519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:05.180519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:26:05.180519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:05.180519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:05.180519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:05.180519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:26:05.180519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:05.180519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:26:05.180519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:26:05.180519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:05.180519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:26:05.180519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:05.180519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:26:05.180519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:26:05.180765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:05.180765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:26:05.180765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:05.180765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:26:05.181999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:05.181999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:26:05.181999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:26:05.181999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:26:05.181999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:26:05.181999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:26:05.181999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:05.181999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:26:05.181999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:26:05.181999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:26:05.181999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:26:05.181999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:26:05.181999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:05.181999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:05.181999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:26:05.181999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:26:05.182306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:26:05.182306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:26:05.182306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:26:05.182306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:26:05.182306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:26:05.182306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:05.182306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:05.182306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:05.182306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:26:05.182306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:05.182306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:05.182306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:05.182306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:26:05.182306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:05.182306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:26:05.182306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:26:05.182306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:05.182306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:26:05.182680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:05.182680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:26:05.182680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:26:05.182680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:05.182680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:26:05.182680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:05.182680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:26:05.201931 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:26:05.202212 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:26:05.202379 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:26:05.202845 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:26:05.203175 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:26:05.203248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:05.204572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:26:05.204572 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:05.204572 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:26:05.204572 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:26:05.204572 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:05.204572 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:26:05.204572 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:05.204572 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:26:05.205127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:05.205127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:26:05.205127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:26:05.205127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:26:05.205127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:05.205127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:26:05.205127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:05.205127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:26:05.205127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:05.205127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:26:05.205127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:05.205127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:26:05.205127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:05.205127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:26:05.205127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:26:05.205992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:05.205992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:26:05.205992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:05.205992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:26:05.205992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:26:05.205992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:05.205992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:26:05.205992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:05.205992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:26:05.205992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:05.205992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:26:05.205992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:05.205992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:26:05.205992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:26:05.207274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:26:05.207274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:26:05.207274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:26:05.207274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:26:05.207274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:05.207274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:05.207274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:05.207274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:26:05.207274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:05.207274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:05.207274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:05.207274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:26:05.207274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:05.208217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:26:05.208217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:26:05.208217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:05.208217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:26:05.208217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:05.208217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:26:05.208217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:26:05.208217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:05.208217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:26:05.208217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:05.208217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:26:05.208217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:05.208917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:26:05.208917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:26:05.208917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:26:05.208917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:26:05.208917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:26:05.208917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:05.208917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:26:05.208917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:26:05.208917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:26:05.208917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:26:05.208917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:26:05.208917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:05.208917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:05.208917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:26:05.208917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:26:05.208917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:26:05.208917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:26:05.208917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:26:05.208917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:26:05.210018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:05.210018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:05.210018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:05.210018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:26:05.210018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:05.210018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:05.210018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:05.210018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:26:05.210018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:05.210018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:26:05.210018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:26:05.210018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:05.210018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:26:05.210018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:05.210018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:26:05.210018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:26:05.210018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:05.210018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:26:05.210018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:05.210018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:26:05.211021 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:26:05.211276 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:26:05.211374 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:26:05.211554 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:26:05.211894 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:26:05.212129 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:26:05.212266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:05.212785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:26:05.212785 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:05.212785 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:26:05.212785 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:26:05.212785 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:05.212785 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:26:05.212785 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:05.212785 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:26:05.213788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:05.213788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:26:05.213788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:05.213788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:26:05.213788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:26:05.213788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:26:05.213788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:26:05.213788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:05.213788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:05.213788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:05.213788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:26:05.213788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:05.214150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:05.214150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:05.214150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:26:05.214150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:05.214150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:26:05.214150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:26:05.214150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:05.214150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:26:05.214150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:05.214150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:26:05.214150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:26:05.214150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:05.214150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:26:05.214633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:05.214633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:26:05.214787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:05.214787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:26:05.214787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:26:05.214787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:26:05.214787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:26:05.214787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:26:05.214787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:05.214787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:26:05.214787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:26:05.214787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:26:05.214787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:26:05.214787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:26:05.214787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:05.214787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:05.214787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:26:05.214787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:26:05.215338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:26:05.215338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:26:05.215338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:05.215338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:05.215338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:05.215338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:26:05.215338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:05.215338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:05.215338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:05.215338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:26:05.215338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:05.215338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:26:05.215338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:26:05.215338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:05.215338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:26:05.215338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:05.215338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:26:05.215338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:26:05.215338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:05.216000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:26:05.216000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:05.216000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:26:09.956251 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:26:09.956858 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:26:09.956858 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:26:09.957829 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:26:09.997668 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:26:09.997925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:09.998654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:26:09.998654 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:09.998654 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:26:09.998654 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:26:09.998654 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:09.998654 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:26:09.998654 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:09.998654 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:26:09.999553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:09.999553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:26:09.999553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:09.999553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:26:09.999553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:26:09.999553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:26:09.999553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:26:09.999553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:09.999553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:09.999553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:09.999553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:26:09.999553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:10.000172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:10.000172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:10.000172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:26:10.000172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:10.000172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:26:10.000172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:26:10.000172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:10.000172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:26:10.000172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:10.000172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:26:10.000172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:26:10.000172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:10.000172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:26:10.000926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:10.000926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:26:10.000926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:10.000926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:26:10.000926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:26:10.000926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:26:10.000926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:26:10.000926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:26:10.000926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:10.000926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:26:10.000926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:26:10.000926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:26:10.000926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:26:10.000926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:26:10.000926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:10.002558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:10.002558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:26:10.002558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:26:10.002558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:26:10.002558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:26:10.002558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:10.002558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:10.002558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:10.002558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:26:10.002558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:10.002558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:10.002558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:10.002558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:26:10.002558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:10.002558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:26:10.002558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:26:10.002558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:10.002558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:26:10.002558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:10.003667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:26:10.003667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:26:10.003667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:10.003667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:26:10.003667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:10.003667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:26:10.169849 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:26:10.170820 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:26:10.170820 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:26:10.171177 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:26:10.171427 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:26:10.171571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:10.172128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:26:10.172128 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:10.172128 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:26:10.172128 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:26:10.172128 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:10.172128 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:26:10.172128 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:10.172128 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:26:10.173053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:10.173053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:26:10.173053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:10.173053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:26:10.173053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:26:10.173053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:26:10.173053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:26:10.173053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:10.173053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:10.173053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:10.173053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:26:10.173053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:10.173748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:10.173748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:10.173748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:26:10.173748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:10.173748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:26:10.173748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:26:10.173748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:10.173748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:26:10.173748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:10.173748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:26:10.173748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:26:10.173748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:10.173748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:26:10.174371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:10.174371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:26:10.174371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:10.174371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:26:10.174371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:26:10.174371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:26:10.174371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:26:10.174371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:26:10.174371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:10.174371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:26:10.174371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:26:10.174371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:26:10.174371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:26:10.174371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:26:10.174371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:10.175136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:10.175136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:26:10.175136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:26:10.175136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:26:10.175136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:26:10.175136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:10.175136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:10.175136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:10.175136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:26:10.175136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:10.175136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:10.175136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:10.175136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:26:10.175136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:10.175136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:26:10.175136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:26:10.175136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:10.175136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:26:10.175136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:10.179975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:26:10.179975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:26:10.179975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:10.179975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:26:10.179975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:10.179975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:26:10.179975 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:26:10.179975 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:26:10.179975 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:26:10.179975 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:26:10.180759 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:26:10.180759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:10.181672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:26:10.181672 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:10.181672 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:26:10.181672 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:26:10.181672 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:10.181672 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:26:10.181672 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:10.181672 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:26:10.182351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:10.182351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:26:10.182351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:26:10.182351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:26:10.182351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:10.182351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:26:10.182351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:10.182351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:26:10.182351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:10.182351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:26:10.182351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:10.182351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:26:10.182351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:10.182351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:26:10.182351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:26:10.183022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:10.183022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:26:10.183022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:10.183022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:26:10.183022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:26:10.183022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:10.183022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:26:10.183022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:10.183022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:26:10.183022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:10.183022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:26:10.183022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:10.183022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:26:10.183022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:26:10.183558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:26:10.183558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:26:10.183558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:26:10.183558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:26:10.183558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:26:10.183558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:10.183558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:10.183558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:10.183558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:26:10.183558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:10.183558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:10.183558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:10.183558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:26:10.184043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:10.184043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:26:10.184043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:26:10.184043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:10.184043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:26:10.184043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:10.184043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:26:10.184043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:26:10.184043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:10.184043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:26:10.184043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:10.184043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:26:10.184449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:10.184449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:26:10.184449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:26:10.184449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:26:10.184449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:26:10.184449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:26:10.184449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:10.184449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:26:10.184449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:26:10.184449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:26:10.184449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:26:10.184449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:26:10.184449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:10.184449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:10.184449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:26:10.184449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:26:10.185062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:26:10.185062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:26:10.185062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:26:10.185062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:26:10.185062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:26:10.185062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:10.185062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:10.185062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:10.185062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:26:10.185062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:10.185062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:10.185062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:10.185062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:26:10.185062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:10.185062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:26:10.185062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:26:10.185062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:10.185062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:26:10.185997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:10.185997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:26:10.185997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:26:10.185997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:10.185997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:26:10.185997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:10.185997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:26:10.210088 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:26:10.223981 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:26:10.223981 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:26:10.223981 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:26:10.223981 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:26:10.223981 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:26:10.223981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:10.225001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:26:10.225001 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:10.225001 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:26:10.225001 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:26:10.225001 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:10.225001 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:26:10.225001 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:10.225001 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:26:10.225001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:10.225001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:26:10.225001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:26:10.225001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:26:10.225001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:10.225001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:26:10.225001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:10.225001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:26:10.225001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:10.225001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:26:10.226230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:10.226230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:26:10.226230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:10.226230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:26:10.226230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:26:10.226230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:10.226230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:26:10.226230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:10.226230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:26:10.226230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:26:10.226230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:10.226230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:26:10.226230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:10.226230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:26:10.226230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:10.227016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:26:10.227016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:10.227016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:26:10.227016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:26:10.227016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:26:10.227016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:26:10.227016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:26:10.227016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:26:10.227016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:10.227016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:10.227016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:10.227016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:26:10.227016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:10.227944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:10.227944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:10.227944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:26:10.227944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:10.227944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:26:10.227944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:26:10.227944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:10.227944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:26:10.227944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:10.227944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:26:10.227944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:26:10.227944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:10.227944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:26:10.228685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:10.228685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:26:10.228685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:10.228685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:26:10.228685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:26:10.228685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:26:10.228685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:26:10.228685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:26:10.228685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:10.228685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:26:10.228685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:26:10.228685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:26:10.228685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:26:10.228685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:26:10.228685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:10.229528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:10.229528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:26:10.229528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:26:10.229528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:26:10.229528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:26:10.229528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:26:10.229528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:26:10.229528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:10.229528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:10.229528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:10.229528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:26:10.229528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:10.229528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:10.229528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:10.229528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:26:10.229528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:10.229528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:26:10.229528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:26:10.229528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:10.230707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:26:10.230707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:10.230707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:26:10.230707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:26:10.230707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:10.230707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:26:10.230707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:10.230707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:26:10.230707 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:26:10.230707 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:26:10.230707 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:26:10.231424 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:26:10.231424 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:26:10.231424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:10.231424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:26:10.231424 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:10.231424 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:26:10.231424 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:26:10.231424 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:10.231424 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:26:10.231424 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:10.231424 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:26:10.231424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:10.232376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:26:10.232376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:10.232376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:26:10.232376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:26:10.232376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:26:10.232376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:26:10.232376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:10.232376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:10.232376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:10.232376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:26:10.232376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:10.232376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:10.232376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:10.232376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:26:10.232848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:10.232848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:26:10.232848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:26:10.232848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:10.232848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:26:10.232848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:10.232848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:26:10.232848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:26:10.232848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:10.232848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:26:10.232848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:10.232848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:26:10.233079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:10.233079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:26:10.233079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:26:10.233079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:26:10.233079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:26:10.233079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:26:10.233079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:10.233079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:26:10.233079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:26:10.233079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:26:10.233079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:26:10.233079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:26:10.233079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:10.233079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:10.233079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:26:10.233079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:26:10.233509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:26:10.233509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:26:10.233509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:10.233509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:10.233509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:10.233509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:26:10.233509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:10.233509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:10.233509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:10.233509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:26:10.233509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:10.233509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:26:10.233509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:26:10.233509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:10.233509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:26:10.233509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:10.233509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:26:10.233509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:26:10.233509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:10.233837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:26:10.233837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:10.233837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:26:14.956809 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:26:14.957513 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:26:14.957573 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:26:14.958560 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:26:15.007019 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:26:15.007352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:15.008259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:26:15.008259 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:15.008259 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:26:15.008259 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:26:15.008259 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:15.008259 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:26:15.008259 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:15.008259 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:26:15.009279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.051 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:15.009279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:26:15.009279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:15.009279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:26:15.009279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:26:15.009279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:26:15.009279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:26:15.009279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:15.009279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:15.009279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:15.009279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:26:15.009279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:15.009750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:15.009750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:15.009750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:26:15.009750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:15.009750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:26:15.009750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:26:15.009750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:15.009750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:26:15.009750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:15.009750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:26:15.009750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:26:15.009750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:15.009750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:26:15.010061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:15.010061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:26:15.010127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:15.010127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:26:15.010127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:26:15.010127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:26:15.010127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:26:15.010127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:26:15.010127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:15.010127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:26:15.010127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:26:15.010127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:26:15.010127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:26:15.010127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:26:15.010127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:15.010127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:15.010127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:26:15.010127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:26:15.010536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:26:15.010536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:26:15.010536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:15.010536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:15.010536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:15.010536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:26:15.010536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:15.010536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:15.010536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:15.010536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:26:15.010536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:15.010536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:26:15.010536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:26:15.010536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:15.010536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:26:15.010536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:15.010536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:26:15.010536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:26:15.010536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:15.010984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:26:15.010984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:15.010984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:26:15.170629 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:26:15.171133 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:26:15.171323 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:26:15.172430 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:26:15.172764 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:26:15.172935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:15.173918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:26:15.173918 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:15.173918 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:26:15.173918 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:26:15.173918 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:15.173918 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:26:15.173918 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:15.173918 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:26:15.175768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:15.175768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:26:15.175768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:15.175768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:26:15.175768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:26:15.175768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:26:15.175768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:26:15.175768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:15.175768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:15.175768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:15.175768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:26:15.175768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:15.176144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:15.176144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:15.176144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:26:15.176144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:15.176144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:26:15.176144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:26:15.176144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:15.176144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:26:15.176144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:15.176144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:26:15.176144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:26:15.176144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:15.176144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:26:15.176496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:15.176496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:26:15.178469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:15.178469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:26:15.178469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:26:15.178469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:26:15.178469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:26:15.178469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:26:15.178469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:15.178469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:26:15.178469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:26:15.178469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:26:15.178469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:26:15.178469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:26:15.178469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:15.178469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:15.178469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:26:15.178469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:26:15.179094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:26:15.179094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:26:15.179094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:15.179094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:15.179094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:15.179094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:26:15.179094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:15.179094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:15.179094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:15.179094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:26:15.179094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:15.179094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:26:15.179094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:26:15.179094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:15.179094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:26:15.179094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:15.179094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:26:15.179094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:26:15.179094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:15.179635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:26:15.179635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:15.179635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:26:15.179635 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:26:15.179635 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:26:15.179635 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:26:15.179635 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:26:15.179878 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:26:15.183462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:15.183462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:26:15.183462 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:15.183462 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:26:15.183462 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:26:15.183462 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:15.183462 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:26:15.183462 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:15.183462 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:26:15.183462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:15.183462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:26:15.183462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:26:15.183462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:26:15.183462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:15.183462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:26:15.183462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:15.183462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:26:15.183992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:15.183992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:26:15.183992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:15.183992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:26:15.183992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:15.183992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:26:15.183992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:26:15.183992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:15.183992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:26:15.183992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:15.183992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:26:15.183992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:26:15.183992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:15.183992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:26:15.183992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:15.183992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:26:15.184485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:15.184485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:26:15.184485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:15.184485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:26:15.184485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:26:15.184485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:26:15.184485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:26:15.184485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:26:15.184485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:26:15.184485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:26:15.184485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:15.184485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:15.184822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:15.184822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:26:15.184822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:15.184822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:15.184822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:15.184822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:26:15.184822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:15.184822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:26:15.184822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:26:15.184822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:15.184822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:26:15.184822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:15.184822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:26:15.184822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:26:15.185195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:15.185195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:26:15.185195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:15.185195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:26:15.185195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:15.185195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:26:15.185195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:26:15.185195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:26:15.185195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:26:15.185195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:26:15.185195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:15.185195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:26:15.185195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:26:15.185195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:26:15.185582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:26:15.185582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:26:15.185582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:15.185582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:15.185582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:26:15.185582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:26:15.185582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:26:15.185582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:26:15.185582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:26:15.185582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:26:15.185582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:26:15.185582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:15.185582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:15.185582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:15.185582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:26:15.185582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:15.185582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:15.185582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:15.185582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:26:15.186112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:15.186112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:26:15.186112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:26:15.186112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:15.186112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:26:15.186112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:15.186112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:26:15.186112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:26:15.186112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:15.186112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:26:15.186112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:15.186112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:26:15.210545 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:26:15.210853 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:26:15.211032 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:26:15.211275 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:26:15.212727 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:26:15.212727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:15.213666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:26:15.213666 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:15.213666 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:26:15.213666 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:26:15.213666 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:15.213666 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:26:15.213666 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:15.213666 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:26:15.215066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:15.215066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:26:15.215066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:26:15.215066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:26:15.215066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:15.215066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:26:15.215066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:15.215066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:26:15.215066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:15.215066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:26:15.215066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:15.215066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:26:15.215066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:15.215066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:26:15.215066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:26:15.215703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:15.215703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:26:15.215703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:15.215703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:26:15.215703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:26:15.215703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:15.215703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:26:15.215703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:15.215703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:26:15.216356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:15.216356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:26:15.216356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:15.216356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:26:15.216356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:26:15.216356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:26:15.216356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:26:15.216356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:26:15.216356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:26:15.216356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:15.216356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:15.216756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:15.216756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:26:15.216756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:15.216756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:15.216756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:15.216756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:26:15.216756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:15.216756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:26:15.216756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:26:15.216756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:15.216756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:26:15.216756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:15.216756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:26:15.216756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:26:15.217143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:15.217143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:26:15.217143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:15.217143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:26:15.217571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:15.217571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:26:15.217571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:26:15.217571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:26:15.217571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:26:15.217571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:26:15.217571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:15.217571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:26:15.217571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:26:15.217571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:26:15.217571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:26:15.217571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:26:15.217571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:15.217571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:15.217571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:26:15.217571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:26:15.217571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:26:15.218091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:26:15.218091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:26:15.218091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:26:15.218091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:15.218091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:15.218091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:15.218091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:26:15.218091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:15.218091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:15.218091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:15.218091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:26:15.218091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:15.218091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:26:15.218091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:26:15.218091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:15.218091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:26:15.218091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:15.218091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:26:15.218091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:26:15.218592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:15.218592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:26:15.218592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:15.218592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:26:15.218592 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:26:15.218592 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:26:15.218592 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:26:15.218592 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:26:15.218866 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:26:15.218866 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:26:15.218866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:15.219385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:26:15.219385 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:15.219385 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:26:15.219385 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:26:15.219385 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:15.219385 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:26:15.219385 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:15.219385 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:26:15.220187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:15.220187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:26:15.220187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:15.220187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:26:15.220187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:26:15.220187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:26:15.220187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:26:15.220187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:15.220187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:15.220187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:15.220187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:26:15.220187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:15.220582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:15.220582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:15.220582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:26:15.220582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:15.220582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:26:15.220582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:26:15.220582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:15.220582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:26:15.220582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:15.220582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:26:15.220582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:26:15.220582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:15.220582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:26:15.220891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:15.220891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:26:15.221133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:15.221133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:26:15.221133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:26:15.221133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:26:15.221133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:26:15.221133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:26:15.221133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:15.221133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:26:15.221133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:26:15.221133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:26:15.221133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:26:15.221133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:26:15.221133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:15.221133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:15.221133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:26:15.221133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:26:15.221591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:26:15.221591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:26:15.221591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:15.221591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:15.221591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:15.221591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:26:15.221591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:15.221591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:15.221591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:15.221591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:26:15.221591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:15.221591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:26:15.221591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:26:15.221591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:15.221591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:26:15.221591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:15.221591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:26:15.221591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:26:15.221591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:15.222055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:26:15.222055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:15.222055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:26:19.957879 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:26:19.958448 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:26:19.958448 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:26:19.959069 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:26:19.995289 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:26:19.995638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:19.996467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:26:19.996467 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:19.996467 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:26:19.996467 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:26:19.996467 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:19.996467 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:26:19.996467 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:19.996467 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:26:19.997616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:19.997616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:26:19.997616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:19.997616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:26:19.997616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:26:19.997616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:26:19.997616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:26:19.997616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:19.997616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:19.997616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:19.997616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:26:19.997616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:19.997894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:19.997894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:19.997894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:26:19.997894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:19.997894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:26:19.997894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:26:19.997894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:19.997894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:26:19.997894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:19.997894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:26:19.997894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:26:19.997894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:19.997894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:26:19.998142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:19.998142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:26:19.998800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:19.998800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:26:19.998800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:26:19.998800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:26:19.998800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:26:19.998800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:26:19.998800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:19.998800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:26:19.998800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:26:19.998800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:26:19.998800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:26:19.998800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:26:19.998800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:19.998800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:19.998800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:26:19.998800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:26:19.999140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:26:19.999140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:26:19.999140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:19.999140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:19.999140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:19.999140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:26:19.999140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:19.999140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:19.999140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:19.999140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:26:19.999140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:19.999140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:26:19.999140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:26:19.999140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:19.999140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:26:19.999140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:19.999140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:26:19.999140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:26:19.999140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:19.999498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:26:19.999498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:19.999498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:26:20.171690 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:26:20.172280 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:26:20.172498 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:26:20.173744 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:26:20.174002 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:26:20.174145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:20.174789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:26:20.174789 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:20.174789 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:26:20.174789 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:26:20.174789 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:20.174789 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:26:20.174789 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:20.174789 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:26:20.175772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:20.175772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:26:20.175772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:20.175772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:26:20.175772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:26:20.175772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:26:20.175772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:26:20.175772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:20.175772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:20.175772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:20.175772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:26:20.175772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:20.176056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:20.176056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:20.176056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:26:20.176056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:20.176056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:26:20.176056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:26:20.176056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:20.176056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:26:20.176056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:20.176056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:26:20.176056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:26:20.176056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:20.176056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:26:20.176306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:20.176306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:26:20.176587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:20.176587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:26:20.176587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:26:20.176587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:26:20.176587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:26:20.176587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:26:20.176587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:20.176587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:26:20.176587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:26:20.176587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:26:20.176587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:26:20.176587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:26:20.176587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:20.176587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:20.176587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:26:20.176587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:26:20.176912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:26:20.176912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:26:20.176912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:20.176912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:20.176912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:20.176912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:26:20.176912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:20.176912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:20.176912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:20.176912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:26:20.176912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:20.176912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:26:20.176912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:26:20.176912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:20.176912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:26:20.176912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:20.176912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:26:20.176912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:26:20.176912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:20.177265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:26:20.177265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:20.177265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:26:20.178951 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:26:20.179143 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:26:20.179252 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:26:20.179388 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:26:20.180712 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:26:20.180894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:20.181380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:26:20.181380 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:20.181380 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:26:20.181380 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:26:20.181380 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:20.181380 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:26:20.181380 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:20.181380 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:26:20.182364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:20.182364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:26:20.182364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:26:20.182364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:26:20.182364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:20.182364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:26:20.182364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:20.182364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:26:20.182364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:20.182364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:26:20.182364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:20.182364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:26:20.182364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:20.182364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:26:20.182364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:26:20.182688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:20.182688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:26:20.182688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:20.182688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:26:20.182688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:26:20.182688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:20.182688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:26:20.182688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:20.182688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:26:20.183557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:20.183557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:26:20.183557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:20.183557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:26:20.183557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:26:20.183557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:26:20.183557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:26:20.183557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:26:20.183557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:26:20.183557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:26:20.183557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:20.183557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:20.183835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:20.183835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:26:20.183835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:20.183835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:20.183835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:20.183835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:26:20.183835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:20.183835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:26:20.183835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:26:20.183835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:20.183835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:26:20.183835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:20.183835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:26:20.183835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:26:20.184095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:20.184095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:26:20.184095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:20.184095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:26:20.184836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:20.184836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:26:20.184836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:26:20.184836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:26:20.184836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:26:20.184836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:26:20.184836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:20.184836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:26:20.184836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:26:20.184836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:26:20.184836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:26:20.184836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:26:20.184836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:20.184836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:20.184836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:26:20.184836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:26:20.185165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:26:20.185165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:26:20.185165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:26:20.185165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:26:20.185165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:26:20.185165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:20.185165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:20.185165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:20.185165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:26:20.185165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:20.185165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:20.185165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:20.185165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:26:20.185165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:20.185165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:26:20.185165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:26:20.185165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:20.185165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:26:20.185503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:20.185503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:26:20.185503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:26:20.185503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:20.185503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:26:20.185503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:20.185503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:26:20.211209 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:26:20.212512 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:26:20.212580 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:26:20.212756 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:26:20.213610 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:26:20.213875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:20.214512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:26:20.214512 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:20.214512 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:26:20.214512 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:26:20.214512 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:20.214512 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:26:20.214512 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:20.214512 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:26:20.215339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:20.215339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:26:20.215339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:26:20.215339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:26:20.215339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:20.215339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:26:20.215339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:20.215339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:26:20.215339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:20.215339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:26:20.215339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:20.215339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:26:20.215339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:20.215339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:26:20.215339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:26:20.215972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:20.215972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:26:20.215972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:20.215972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:26:20.215972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:26:20.215972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:20.215972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:26:20.215972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:20.215972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:26:20.216421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:20.216421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:26:20.216421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:20.216421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:26:20.216421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:26:20.216421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:26:20.216421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:26:20.216421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:26:20.216421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:26:20.216421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:20.216421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:20.216878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:20.216878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:26:20.216878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:20.216878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:20.216878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:20.216878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:26:20.216878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:20.216878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:26:20.216878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:26:20.216878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:20.216878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:26:20.216878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:20.216878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:26:20.216878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:26:20.217496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:20.217496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:26:20.217496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:20.217496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:26:20.217496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:20.217496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:26:20.217496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:26:20.217496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:26:20.217496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:26:20.217496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:26:20.217496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:20.217496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:26:20.217496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:26:20.217496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:26:20.218102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:26:20.218102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:26:20.218102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:20.218102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:20.218102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:26:20.218102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:26:20.218102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:26:20.218102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:26:20.218102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:26:20.218102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:26:20.218102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:20.218102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:20.218102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:20.218102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:26:20.218102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:20.218102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:20.218102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:20.218102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:26:20.218102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:20.218935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:26:20.218935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:26:20.218935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:20.218935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:26:20.218935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:20.218935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:26:20.218935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:26:20.218935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:20.218935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:26:20.218935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:20.218935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:26:20.218935 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:26:20.218935 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:26:20.219938 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:26:20.219938 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:26:20.219938 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:26:20.219938 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:26:20.219938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:20.219938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:26:20.219938 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:20.219938 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:26:20.219938 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:26:20.220705 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:20.220705 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:26:20.220705 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:20.220705 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:26:20.220705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:20.220705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:26:20.220705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:20.220705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:26:20.220705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:26:20.220705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:26:20.220705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:26:20.220705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:20.220705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:20.220705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:20.221315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:26:20.221315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:20.221315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:20.221315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:20.221315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:26:20.221315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:20.221315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:26:20.221315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:26:20.221315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:20.221315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:26:20.221315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:20.221315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:26:20.221315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:26:20.221315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:20.221943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:26:20.221943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:20.221943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:26:20.221943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:20.221943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:26:20.221943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:26:20.221943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:26:20.221943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:26:20.221943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:26:20.221943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:20.221943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:26:20.221943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:26:20.221943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:26:20.221943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:26:20.221943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:26:20.221943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:20.222321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:20.222321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:26:20.222321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:26:20.222321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:26:20.222321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:26:20.222321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:20.222321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:20.222321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:20.222321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:26:20.222321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:20.222321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:20.222321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:20.222321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:26:20.222321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:20.222321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:26:20.222321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:26:20.222321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:20.222321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:26:20.222321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:20.222660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:26:20.222660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:26:20.222660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:20.222660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:26:20.222660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:20.222660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:26:24.958749 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:26:24.959247 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:26:24.959247 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:26:24.960194 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:26:24.962104 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG dbcounter [-] [105581] Writing DB stats neutron:SELECT=240 {{(pid=105581) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 14:26:24.998280 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:26:24.998544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:24.999191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:26:24.999191 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:24.999191 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:26:24.999191 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:26:24.999191 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:24.999191 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:26:24.999191 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:24.999191 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:26:25.000278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:25.000278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:26:25.000278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:25.000278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:26:25.000278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:26:25.000278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:26:25.000278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:26:25.000278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:25.000278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:25.000278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:25.000278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:26:25.000278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:25.000705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:25.000705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:25.000705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:26:25.000705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:25.000705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:26:25.000705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:26:25.000705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:25.000705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:26:25.000705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:25.000705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:26:25.000705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:26:25.000705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:25.000705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:26:25.001011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:25.001011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:26:25.001206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:25.001206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:26:25.001206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:26:25.001206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:26:25.001206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:26:25.001206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:26:25.001206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:25.001206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:26:25.001206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:26:25.001206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:26:25.001206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:26:25.001206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:26:25.001206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:25.001206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:25.001206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:26:25.001206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:26:25.001731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:26:25.001731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:26:25.001731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:25.001731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:25.001731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:25.001731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:26:25.001731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:25.001731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:25.001731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:25.001731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:26:25.001731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:25.001731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:26:25.001731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:26:25.001731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:25.001731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:26:25.001731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:25.001731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:26:25.001731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:26:25.001731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:25.002160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:26:25.002160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:25.002160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:26:25.172661 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:26:25.172833 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:26:25.172971 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:26:25.175293 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:26:25.175712 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:26:25.175897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:25.176837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:26:25.176837 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:25.176837 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:26:25.176837 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:26:25.176837 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:25.176837 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:26:25.176837 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:25.176837 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:26:25.178359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:25.178359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:26:25.178359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:25.178359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:26:25.178359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:26:25.178359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:26:25.178359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:26:25.178359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:25.178359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:25.178359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:25.178359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:26:25.178359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:25.178830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:25.178830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:25.178830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:26:25.178830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:25.178830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:26:25.178830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:26:25.178830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:25.178830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:26:25.178830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:25.178830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:26:25.178830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:26:25.178830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:25.178830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:26:25.179254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:25.179254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:26:25.181430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:25.181430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:26:25.181430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:26:25.181430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:26:25.181430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:26:25.181430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:26:25.181430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:25.181430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:26:25.181430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:26:25.181430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:26:25.181430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:26:25.181430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:26:25.181430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:25.181430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:25.181430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:26:25.181430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:26:25.181813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:26:25.181813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:26:25.181813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:25.181813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:25.181813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:25.181813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:26:25.181813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:25.181813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:25.181813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:25.181813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:26:25.181813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:25.181813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:26:25.181813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:26:25.181813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:25.181813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:26:25.181813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:25.181813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:26:25.181813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:26:25.181813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:25.182219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:26:25.182219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:25.182219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:26:25.182219 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:26:25.182219 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:26:25.182219 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:26:25.182219 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:26:25.182219 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:26:25.182466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:25.182741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:26:25.182741 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:25.182741 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:26:25.182741 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:26:25.182741 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:25.182741 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:26:25.182741 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:25.182741 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:26:25.183941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:25.183941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:26:25.183941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:26:25.183941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:26:25.183941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:25.183941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:26:25.183941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:25.183941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:26:25.183941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:25.183941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:26:25.183941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:25.183941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:26:25.183941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:25.183941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:26:25.183941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:26:25.184355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:25.184355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:26:25.184355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:25.184355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:26:25.184355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:26:25.184355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:25.184355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:26:25.184355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:25.184355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:26:25.184998 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:25.184998 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:26:25.184998 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:25.184998 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:26:25.184998 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:26:25.184998 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:26:25.184998 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:26:25.184998 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:26:25.184998 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:26:25.184998 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:26:25.184998 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:25.184998 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:25.185334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:25.185334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:26:25.185334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:25.185334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:25.185334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:25.185334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:26:25.185334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:25.185334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:26:25.185334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:26:25.185334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:25.185334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:26:25.185334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:25.185334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:26:25.185334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:26:25.185624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:25.185624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:26:25.185624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:25.185624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:26:25.185726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:25.185726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:26:25.185726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:26:25.185726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:26:25.185726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:26:25.185726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:26:25.185726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:25.185726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:26:25.185726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:26:25.185726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:26:25.185726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:26:25.185726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:26:25.185726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:25.185726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:25.185726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:26:25.185726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:26:25.186055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:26:25.186055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:26:25.186055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:26:25.186055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:26:25.186055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:26:25.186055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:25.186055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:25.186055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:25.186055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:26:25.186055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:25.186055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:25.186055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:25.186055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:26:25.186055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:25.186055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:26:25.186055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:26:25.186055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:25.186055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:26:25.186480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:25.186480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:26:25.186480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:26:25.186480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:25.186480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:26:25.186480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:25.186480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:26:25.212754 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:26:25.212959 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:26:25.213133 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:26:25.213329 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:26:25.214186 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:26:25.214529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:25.215283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:26:25.215283 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:25.215283 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:26:25.215283 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:26:25.215283 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:25.215283 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:26:25.215283 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:25.215283 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:26:25.216642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:25.216642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:26:25.216642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:26:25.216642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:26:25.216642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:25.216642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:26:25.216642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:25.216642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:26:25.216642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:25.216642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:26:25.216642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:25.216642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:26:25.216642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:25.216642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:26:25.216642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:26:25.217345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:25.217345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:26:25.217345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:25.217345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:26:25.217345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:26:25.217345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:25.217345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:26:25.217345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:25.217345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:26:25.218367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:25.218367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:26:25.218367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:25.218367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:26:25.218367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:26:25.218367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:26:25.218367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:26:25.218367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:26:25.218367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:26:25.218367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:25.218367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:25.219051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:25.219051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:26:25.219051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:25.219051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:25.219051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:25.219051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:26:25.219051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:25.219051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:26:25.219051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:26:25.219051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:25.219051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:26:25.219051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:25.219051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:26:25.219051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:26:25.219678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:25.219678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:26:25.219678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:25.219678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:26:25.220130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:25.220130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:26:25.220130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:26:25.220130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:26:25.220130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:26:25.220130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:26:25.220130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:25.220130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:26:25.220130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:26:25.220130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:26:25.220130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:26:25.220130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:26:25.220130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:25.220130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:25.220130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:26:25.220130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:26:25.220130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:26:25.221055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:26:25.221055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:26:25.221055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:26:25.221055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:25.221055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:25.221055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:25.221055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:26:25.221055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:25.221055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:25.221055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:25.221055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:26:25.221055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:25.221055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:26:25.221055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:26:25.221055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:25.221055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:26:25.221055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:25.221055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:26:25.221055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:26:25.222258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:25.222258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:26:25.222258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:25.222258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:26:25.222258 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:26:25.222258 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:26:25.222258 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:26:25.222258 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:26:25.222849 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:26:25.223099 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:26:25.223180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:25.223811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:26:25.223811 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:25.223811 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:26:25.223811 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:26:25.223811 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:25.223811 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:26:25.223811 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:25.223811 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:26:25.224698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:25.224698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:26:25.224698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:25.224698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:26:25.224698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:26:25.224698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:26:25.224698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:26:25.224698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:25.224698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:25.224698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:25.224698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:26:25.224698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:25.225145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:25.225145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:25.225145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:26:25.225145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:25.225145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:26:25.225145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:26:25.225145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:25.225145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:26:25.225145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:25.225145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:26:25.225145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:26:25.225145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:25.225145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:26:25.225527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:25.225527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:26:25.225629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:25.225629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:26:25.225629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:26:25.225629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:26:25.225629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:26:25.225629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:26:25.225629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:25.225629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:26:25.225629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:26:25.225629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:26:25.225629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:26:25.225629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:26:25.225629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:25.225629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:25.225629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:26:25.225629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:26:25.226267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:26:25.226267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:26:25.226267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:25.226267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:25.226267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:25.226267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:26:25.226267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:25.226267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:25.226267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:25.226267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:26:25.226267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:25.226267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:26:25.226267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:26:25.226267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:25.226267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:26:25.226267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:25.226267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:26:25.226267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:26:25.226267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:25.226833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:26:25.226833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:25.226833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:26:29.959611 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:26:29.960305 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:26:29.960305 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:26:29.960744 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:26:29.995342 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:26:29.995840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:29.996792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:26:29.996792 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:29.996792 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:26:29.996792 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:26:29.996792 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:29.996792 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:26:29.996792 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:29.996792 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:26:29.997659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:29.997659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:26:29.997659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:29.997659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:26:29.997659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:26:29.997659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:26:29.997659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:26:29.997659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:29.997659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:29.997659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:29.997659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:26:29.997659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:29.998133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:29.998133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:29.998133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:26:29.998133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:29.998133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:26:29.998133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:26:29.998133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:29.998133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:26:29.998133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:29.998133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:26:29.998133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:26:29.998133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:29.998133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:26:29.998536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:29.998536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:26:29.998536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:29.998536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:26:29.998536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:26:29.998536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:26:29.998536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:26:29.998536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:26:29.998536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:29.998536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:26:29.998536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:26:29.998536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:26:29.998536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:26:29.998536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:26:29.998536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:29.999018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:29.999018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:26:29.999018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:26:29.999018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:26:29.999018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:26:29.999018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:29.999018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:29.999018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:29.999018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:26:29.999018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:29.999018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:29.999018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:29.999018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:26:29.999018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:29.999018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:26:29.999018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:26:29.999018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:29.999018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:26:29.999018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:29.999560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:26:29.999560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:26:29.999560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:29.999560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:26:29.999560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:29.999560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:26:30.173224 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:26:30.173677 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:26:30.173870 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:26:30.174922 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:26:30.175176 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:26:30.175338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:30.176268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:26:30.176268 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:30.176268 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:26:30.176268 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:26:30.176268 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:30.176268 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:26:30.176268 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:30.176268 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:26:30.177474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:30.177474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:26:30.177474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:30.177474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:26:30.177474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:26:30.177474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:26:30.177474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:26:30.177474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:30.177474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:30.177474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:30.177474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:26:30.177474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:30.177788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:30.177788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:30.177788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:26:30.177788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:30.177788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:26:30.177788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:26:30.177788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:30.177788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:26:30.177788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:30.177788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:26:30.177788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:26:30.177788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:30.177788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:26:30.178087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:30.178087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:26:30.179115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:30.179115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:26:30.179115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:26:30.179115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:26:30.179115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:26:30.179115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:26:30.179115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:30.179115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:26:30.179115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:26:30.179115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:26:30.179115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:26:30.179115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:26:30.179115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:30.179115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:30.179115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:26:30.179115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:26:30.179520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:26:30.179520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:26:30.179520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:30.179520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:30.179520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:30.179520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:26:30.179520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:30.179520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:30.179520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:30.179520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:26:30.179520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:30.179520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:26:30.179520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:26:30.179520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:30.179520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:26:30.179520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:30.179520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:26:30.179520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:26:30.179520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:30.179938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:26:30.179938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:30.179938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:26:30.181339 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:26:30.181768 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:26:30.182008 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:26:30.182211 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:26:30.182610 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:26:30.182775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:30.183376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:26:30.183376 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:30.183376 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:26:30.183376 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:26:30.183376 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:30.183376 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:26:30.183376 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:30.183376 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:26:30.185131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:30.185131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:26:30.185131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:26:30.185131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:26:30.185131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:30.185131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:26:30.185131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:30.185131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:26:30.185131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:30.185131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:26:30.185131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:30.185131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:26:30.185131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:30.185131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:26:30.185131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:26:30.185513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:30.185513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:26:30.185513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:30.185513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:26:30.185513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:26:30.185513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:30.185513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:26:30.185513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:30.185513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:26:30.186795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:30.186795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:26:30.186795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:30.186795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:26:30.186795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:26:30.186795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:26:30.186795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:26:30.186795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:26:30.186795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:26:30.186795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:26:30.186795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:30.186795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:30.187109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:30.187109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:26:30.187109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:30.187109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:30.187109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:30.187109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:26:30.187109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:30.187109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:26:30.187109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:26:30.187109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:30.187109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:26:30.187109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:30.187109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:26:30.187109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:26:30.187392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:30.187392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:26:30.187392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:30.187392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:26:30.188771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:30.188771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:26:30.188771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:26:30.188771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:26:30.188771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:26:30.188771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:26:30.188771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:30.188771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:26:30.188771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:26:30.188771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:26:30.188771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:26:30.188771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:26:30.188771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:30.188771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:30.188771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:26:30.188771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:26:30.189197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:26:30.189197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:26:30.189197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:26:30.189197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:26:30.189197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:26:30.189197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:30.189197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:30.189197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:30.189197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:26:30.189197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:30.189197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:30.189197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:30.189197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:26:30.189197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:30.189197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:26:30.189197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:26:30.189197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:30.189197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:26:30.189680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:30.189680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:26:30.189680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:26:30.189680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:30.189680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:26:30.189680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:30.189680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:26:30.213685 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:26:30.213685 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:26:30.213971 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:26:30.214120 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:26:30.214594 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:26:30.215103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:30.215518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:26:30.215518 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:30.215518 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:26:30.215518 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:26:30.215518 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:30.215518 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:26:30.215518 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:30.215518 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:26:30.216861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:30.216861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:26:30.216861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:26:30.216861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:26:30.216861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:30.216861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:26:30.216861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:30.216861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:26:30.216861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:30.216861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:26:30.216861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:30.216861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:26:30.216861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:30.216861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:26:30.216861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:26:30.217377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:30.217377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:26:30.217377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:30.217377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:26:30.217377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:26:30.217377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:30.217377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:26:30.217377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:30.217377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:26:30.218470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:30.218470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:26:30.218470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:30.218470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:26:30.218470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:26:30.218470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:26:30.218470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:26:30.218470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:26:30.218470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:26:30.218470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:30.218470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:30.218885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:30.218885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:26:30.218885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:30.218885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:30.218885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:30.218885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:26:30.218885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:30.218885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:26:30.218885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:26:30.218885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:30.218885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:26:30.218885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:30.218885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:26:30.218885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:26:30.219330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:30.219330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:26:30.219330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:30.219330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:26:30.220214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:30.220214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:26:30.220214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:26:30.220214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:26:30.220214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:26:30.220214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:26:30.220214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:30.220214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:26:30.220214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:26:30.220214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:26:30.220214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:26:30.220214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:26:30.220214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:30.220214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:30.220214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:26:30.220214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:26:30.220214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:26:30.220745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:26:30.220745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:26:30.220745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:26:30.220745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:30.220745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:30.220745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:30.220745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:26:30.220745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:30.220745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:30.220745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:30.220745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:26:30.220745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:30.220745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:26:30.220745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:26:30.220745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:30.220745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:26:30.220745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:30.220745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:26:30.220745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:26:30.221255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:30.221255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:26:30.221255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:30.221255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:26:30.221255 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:26:30.221255 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:26:30.221255 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:26:30.221884 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:26:30.224775 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:26:30.224995 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:26:30.225253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:30.225705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:26:30.225705 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:30.225705 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:26:30.225705 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:26:30.225705 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:30.225705 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:26:30.225705 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:30.225705 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:26:30.226620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:30.226620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:26:30.226620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:30.226620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:26:30.226620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:26:30.226620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:26:30.226620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:26:30.226620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:30.226620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:30.226620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:30.226620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:26:30.226620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:30.226951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:30.226951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:30.226951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:26:30.226951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:30.226951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:26:30.226951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:26:30.226951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:30.226951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:26:30.226951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:30.226951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:26:30.226951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:26:30.226951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:30.226951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:26:30.227205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:30.227205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:26:30.227648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:30.227648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:26:30.227648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:26:30.227648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:26:30.227648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:26:30.227648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:26:30.227648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:30.227648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:26:30.227648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:26:30.227648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:26:30.227648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:26:30.227648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:26:30.227648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:30.227648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:30.227648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:26:30.227648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:26:30.228003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:26:30.228003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:26:30.228003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:30.228003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:30.228003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:30.228003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:26:30.228003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:30.228003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:30.228003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:30.228003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:26:30.228003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:30.228003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:26:30.228003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:26:30.228003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:30.228003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:26:30.228003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:30.228003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:26:30.228003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:26:30.228003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:30.228340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:26:30.228340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:30.228340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:26:34.960279 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:26:34.961741 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:26:34.961741 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:26:34.963050 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:26:34.997497 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:26:34.997843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:34.998626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:26:34.998626 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:34.998626 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:26:34.998626 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:26:34.998626 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:34.998626 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:26:34.998626 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:34.998626 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:26:34.999551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:34.999551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:26:34.999551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:34.999551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:26:34.999551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:26:34.999551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:26:34.999551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:26:34.999551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:34.999551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:34.999551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:34.999551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:26:34.999551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:35.000291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:35.000291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:35.000291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:26:35.000291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:35.000291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:26:35.000291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:26:35.000291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:35.000291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:26:35.000291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:35.000291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:26:35.000291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:26:35.000291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:35.000291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:26:35.001074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:35.001074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:26:35.001074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:35.001074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:26:35.001074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:26:35.001074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:26:35.001074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:26:35.001074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:26:35.001074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:35.001074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:26:35.001074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:26:35.001074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:26:35.001074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:26:35.001074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:26:35.001074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:35.002118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:35.002118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:26:35.002118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:26:35.002118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:26:35.002118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:26:35.002118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:35.002118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:35.002118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:35.002118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:26:35.002118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:35.002118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:35.002118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:35.002118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:26:35.002118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:35.002118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:26:35.002118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:26:35.002118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:35.002118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:26:35.002118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:35.003644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:26:35.003644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:26:35.003644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:35.003644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:26:35.003644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:35.003644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:26:35.174080 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:26:35.174563 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:26:35.174750 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:26:35.176176 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:26:35.176744 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:26:35.176902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:35.177999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:26:35.177999 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:35.177999 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:26:35.177999 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:26:35.177999 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:35.177999 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:26:35.177999 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:35.177999 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:26:35.179657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:35.179657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:26:35.179657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:35.179657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:26:35.179657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:26:35.179657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:26:35.179657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:26:35.179657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:35.179657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:35.179657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:35.179657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:26:35.179657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:35.180101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:35.180101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:35.180101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:26:35.180101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:35.180101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:26:35.180101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:26:35.180101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:35.180101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:26:35.180101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:35.180101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:26:35.180101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:26:35.180101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:35.180101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:26:35.180771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:35.180771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:26:35.181306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:35.181306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:26:35.181306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:26:35.181306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:26:35.181306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:26:35.181306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:26:35.181306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:35.181306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:26:35.181306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:26:35.181306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:26:35.181306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:26:35.181306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:26:35.181306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:35.181306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:35.181306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:26:35.181306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:26:35.182126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:26:35.182126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:26:35.182126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:35.182126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:35.182126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:35.182126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:26:35.182126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:35.182126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:35.182126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:35.182126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:26:35.182126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:35.182126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:26:35.182126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:26:35.182126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:35.182126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:26:35.182126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:35.182126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:26:35.182126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:26:35.182126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:35.182997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:26:35.182997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:35.182997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:26:35.182997 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:26:35.182997 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:26:35.182997 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:26:35.182997 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:26:35.182997 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:26:35.183285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:35.183333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:26:35.183333 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:35.183333 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:26:35.183333 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:26:35.183333 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:35.183333 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:26:35.183333 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:35.183333 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:26:35.184171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:35.184171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:26:35.184171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:26:35.184171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:26:35.184171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:35.184171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:26:35.184171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:35.184171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:26:35.184171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:35.184171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:26:35.184171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:35.184171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:26:35.184171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:35.184171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:26:35.184171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:26:35.184702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:35.184702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:26:35.184702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:35.184702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:26:35.184702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:26:35.184702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:35.184702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:26:35.184702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:35.184702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:26:35.184973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:35.184973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:26:35.184973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:35.184973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:26:35.184973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:26:35.184973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:26:35.184973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:26:35.184973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:26:35.184973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:26:35.184973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:26:35.184973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:35.184973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:35.185336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:35.185336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:26:35.185336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:35.185336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:35.185336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:35.185336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:26:35.185336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:35.185336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:26:35.185336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:26:35.185336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:35.185336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:26:35.185336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:35.185336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:26:35.185336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:26:35.185910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:35.185910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:26:35.185910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:35.185910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:26:35.186042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:35.186042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:26:35.186042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:26:35.186042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:26:35.186042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:26:35.186042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:26:35.186042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:35.186042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:26:35.186042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:26:35.186042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:26:35.186042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:26:35.186042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:26:35.186042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:35.186042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:35.186042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:26:35.186042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:26:35.186496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:26:35.186496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:26:35.186496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:26:35.186496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:26:35.186496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:26:35.186496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:35.186496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:35.186496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:35.186496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:26:35.186496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:35.186496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:35.186496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:35.186496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:26:35.186496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:35.186496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:26:35.186496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:26:35.186496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:35.186496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:26:35.187077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:35.187077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:26:35.187077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:26:35.187077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:35.187077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:26:35.187077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:35.187077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:26:35.213815 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:26:35.214019 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:26:35.214195 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:26:35.214427 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:26:35.215060 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:26:35.215321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:35.215977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:26:35.215977 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:35.215977 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:26:35.215977 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:26:35.215977 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:35.215977 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:26:35.215977 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:35.215977 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:26:35.217336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:35.217336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:26:35.217336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:26:35.217336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:26:35.217336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:35.217336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:26:35.217336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:35.217336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:26:35.217336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:35.217336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:26:35.217336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:35.217336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:26:35.217336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:35.217336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:26:35.217336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:26:35.218005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:35.218005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:26:35.218005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:35.218005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:26:35.218005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:26:35.218005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:35.218005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:26:35.218005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:35.218005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:26:35.219483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:35.219483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:26:35.219483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:35.219483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:26:35.219483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:26:35.219483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:26:35.219483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:26:35.219483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:26:35.219483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:26:35.219483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:35.219483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:35.219942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:35.219942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:26:35.219942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:35.219942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:35.219942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:35.219942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:26:35.219942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:35.219942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:26:35.219942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:26:35.219942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:35.219942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:26:35.219942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:35.219942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:26:35.219942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:26:35.220380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:35.220380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:26:35.220380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:35.220380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:26:35.221448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:35.221448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:26:35.221448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:26:35.221448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:26:35.221448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:26:35.221448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:26:35.221448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:35.221448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:26:35.221448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:26:35.221448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:26:35.221448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:26:35.221448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:26:35.221448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:35.221448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:35.221448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:26:35.221448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:26:35.221448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:26:35.222162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:26:35.222162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:26:35.222162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:26:35.222162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:35.222162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:35.222162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:35.222162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:26:35.222162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:35.222162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:35.222162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:35.222162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:26:35.222162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:35.222162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:26:35.222162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:26:35.222162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:35.222162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:26:35.222162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:35.222162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:26:35.222162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:26:35.222899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:35.222899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:26:35.222899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:35.222899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:26:35.222899 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:26:35.222899 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:26:35.222899 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:26:35.222899 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:26:35.223271 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:26:35.223271 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:26:35.223271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:35.224031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:26:35.224031 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:35.224031 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:26:35.224031 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:26:35.224031 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:35.224031 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:26:35.224031 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:35.224031 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:26:35.225202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:35.225202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:26:35.225202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:35.225202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:26:35.225202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:26:35.225202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:26:35.225202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:26:35.225202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:35.225202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:35.225202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:35.225202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:26:35.225202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:35.225769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:35.225769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:35.225769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:26:35.225769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:35.225769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:26:35.225769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:26:35.225769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:35.225769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:26:35.225769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:35.225769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:26:35.225769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:26:35.225769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:35.225769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:26:35.226282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:35.226282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:26:35.226557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:35.226557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:26:35.226557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:26:35.226557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:26:35.226557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:26:35.226557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:26:35.226557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:35.226557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:26:35.226557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:26:35.226557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:26:35.226557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:26:35.226557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:26:35.226557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:35.226557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:35.226557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:26:35.226557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:26:35.227053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:26:35.227053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:26:35.227053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:35.227053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:35.227053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:35.227053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:26:35.227053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:35.227053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:35.227053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:35.227053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:26:35.227053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:35.227053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:26:35.227053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:26:35.227053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:35.227053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:26:35.227053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:35.227053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:26:35.227053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:26:35.227053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:35.227599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:26:35.227599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:35.227599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:26:39.961233 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:26:39.962146 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:26:39.962146 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:26:39.963274 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:26:40.001168 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:26:40.001653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:40.002816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:26:40.002816 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:40.002816 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:26:40.002816 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:26:40.002816 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:40.002816 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:26:40.002816 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:40.002816 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:26:40.004168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:40.004168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:26:40.004168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:40.004168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:26:40.004168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:26:40.004168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:26:40.004168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:26:40.004168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:40.004168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:40.004168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:40.004168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:26:40.004168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:40.004762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:40.004762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:40.004762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:26:40.004762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:40.004762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:26:40.004762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:26:40.004762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:40.004762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:26:40.004762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:40.004762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:26:40.004762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:26:40.004762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:40.004762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:26:40.005281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:40.005281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:26:40.006446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:40.006446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:26:40.006446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:26:40.006446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:26:40.006446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:26:40.006446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:26:40.006446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:40.006446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:26:40.006446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:26:40.006446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:26:40.006446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:26:40.006446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:26:40.006446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:40.006446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:40.006446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:26:40.006446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:26:40.007110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:26:40.007110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:26:40.007110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:40.007110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:40.007110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:40.007110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:26:40.007110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:40.007110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:40.007110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:40.007110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:26:40.007110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:40.007110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:26:40.007110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:26:40.007110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:40.007110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:26:40.007110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:40.007110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:26:40.007110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:26:40.007110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:40.008054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:26:40.008054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:40.008054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:26:40.174855 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:26:40.175192 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:26:40.175339 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:26:40.176437 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:26:40.177680 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:26:40.177680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:40.177833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:26:40.177833 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:40.177833 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:26:40.177833 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:26:40.177833 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:40.177833 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:26:40.177833 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:40.177833 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:26:40.178849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:40.178849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:26:40.178849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:40.178849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:26:40.178849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:26:40.178849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:26:40.178849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:26:40.178849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:40.178849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:40.178849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:40.178849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:26:40.178849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:40.179484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:40.179484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:40.179484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:26:40.179484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:40.179484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:26:40.179484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:26:40.179484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:40.179484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:26:40.179484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:40.179484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:26:40.179484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:26:40.179484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:40.179484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:26:40.180026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:40.180026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:26:40.180026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:40.180026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:26:40.180026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:26:40.180026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:26:40.180026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:26:40.180026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:26:40.180026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:40.180026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:26:40.180026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:26:40.180026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:26:40.180026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:26:40.180026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:26:40.180026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:40.180712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:40.180712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:26:40.180712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:26:40.180712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:26:40.180712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:26:40.180712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:40.180712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:40.180712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:40.180712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:26:40.180712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:40.180712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:40.180712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:40.180712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:26:40.180712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:40.180712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:26:40.180712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:26:40.180712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:40.180712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:26:40.180712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:40.181379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:26:40.181379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:26:40.181379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:40.181379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:26:40.181379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:40.181379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:26:40.182053 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:26:40.182130 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:26:40.182620 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:26:40.182620 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:26:40.182834 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:26:40.183124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:40.183831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:26:40.183831 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:40.183831 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:26:40.183831 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:26:40.183831 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:40.183831 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:26:40.183831 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:40.183831 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:26:40.184610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:40.184610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:26:40.184610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:26:40.184610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:26:40.184610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:40.184610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:26:40.184610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:40.184610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:26:40.184610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:40.184610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:26:40.184610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:40.184610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:26:40.184610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:40.184610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:26:40.184610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:26:40.185282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:40.185282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:26:40.185282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:40.185282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:26:40.185282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:26:40.185282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:40.185282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:26:40.185282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:40.185282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:26:40.185736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:40.185736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:26:40.185736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:40.185736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:26:40.185736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:26:40.185736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:26:40.185736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:26:40.185736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:26:40.185736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:26:40.185736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:26:40.185736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:40.185736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:40.186619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:40.186619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:26:40.186619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:40.186619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:40.186619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:40.186619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:26:40.186619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:40.186619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:26:40.186619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:26:40.186619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:40.186619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:26:40.186619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:40.186619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:26:40.186619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:26:40.187235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:40.187235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:26:40.187235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:40.187235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:26:40.187235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:40.187235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:26:40.187235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:26:40.187235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:26:40.187235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:26:40.187235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:26:40.187235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:40.187235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:26:40.187235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:26:40.187235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:26:40.187896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:26:40.187896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:26:40.187896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:40.187896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:40.187896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:26:40.187896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:26:40.187896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:26:40.187896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:26:40.187896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:26:40.187896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:26:40.187896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:26:40.187896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:40.187896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:40.187896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:40.187896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:26:40.187896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:40.187896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:40.187896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:40.187896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:26:40.189525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:40.189525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:26:40.189525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:26:40.189525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:40.189525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:26:40.189525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:40.189525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:26:40.189525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:26:40.189525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:40.189525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:26:40.189525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:40.189525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:26:40.214305 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:26:40.214791 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:26:40.214917 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:26:40.215192 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:26:40.216670 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:26:40.216670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:40.216974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:26:40.216974 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:40.216974 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:26:40.216974 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:26:40.216974 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:40.216974 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:26:40.216974 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:40.216974 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:26:40.218421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:40.218421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:26:40.218421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:26:40.218421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:26:40.218421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:40.218421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:26:40.218421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:40.218421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:26:40.218421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:40.218421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:26:40.218421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:40.218421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:26:40.218421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:40.218421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:26:40.218421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:26:40.218897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:40.218897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:26:40.218897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:40.218897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:26:40.218897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:26:40.218897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:40.218897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:26:40.218897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:40.218897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:26:40.219464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:40.219464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:26:40.219464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:40.219464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:26:40.219464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:26:40.219464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:26:40.219464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:26:40.219464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:26:40.219464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:26:40.219464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:40.219464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:40.219808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:40.219808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:26:40.219808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:40.219808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:40.219808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:40.219808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:26:40.219808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:40.219808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:26:40.219808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:26:40.219808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:40.219808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:26:40.219808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:40.219808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:26:40.219808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:26:40.220244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:40.220244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:26:40.220244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:40.220244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:26:40.220593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:40.220593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:26:40.220593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:26:40.220593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:26:40.220593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:26:40.220593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:26:40.220593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:40.220593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:26:40.220593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:26:40.220593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:26:40.220593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:26:40.220593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:26:40.220593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:40.220593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:40.220593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:26:40.220593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:26:40.220593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:26:40.221193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:26:40.221193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:26:40.221193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:26:40.221193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:40.221193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:40.221193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:40.221193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:26:40.221193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:40.221193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:40.221193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:40.221193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:26:40.221193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:40.221193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:26:40.221193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:26:40.221193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:40.221193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:26:40.221193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:40.221193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:26:40.221193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:26:40.221746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:40.221746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:26:40.221746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:40.221746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:26:40.222008 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:26:40.222201 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:26:40.222296 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:26:40.222466 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:26:40.223233 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:26:40.223605 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:26:40.223821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:40.224573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:26:40.224573 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:40.224573 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:26:40.224573 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:26:40.224573 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:40.224573 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:26:40.224573 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:40.224573 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:26:40.225425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:40.225425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:26:40.225425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:40.225425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:26:40.225425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:26:40.225425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:26:40.225425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:26:40.225425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:40.225425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:40.225425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:40.225425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:26:40.225425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:40.225760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:40.225760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:40.225760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:26:40.225760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:40.225760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:26:40.225760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:26:40.225760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:40.225760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:26:40.225760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:40.225760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:26:40.225760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:26:40.225760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:40.225760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:26:40.226090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:40.226090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:26:40.226393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:40.226393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:26:40.226393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:26:40.226393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:26:40.226393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:26:40.226393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:26:40.226393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:40.226393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:26:40.226393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:26:40.226393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:26:40.226393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:26:40.226393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:26:40.226393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:40.226393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:40.226393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:26:40.226393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:26:40.226824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:26:40.226824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:26:40.226824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:40.226824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:40.226824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:40.226824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:26:40.226824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:40.226824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:40.226824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:40.226824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:26:40.226824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:40.226824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:26:40.226824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:26:40.226824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:40.226824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:26:40.226824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:40.226824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:26:40.226824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:26:40.226824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:40.227341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:26:40.227341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:40.227341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:26:44.962707 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:26:44.963592 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:26:44.963592 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:26:44.965044 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:26:45.012499 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:26:45.012752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:45.013470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:26:45.013470 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:45.013470 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:26:45.013470 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:26:45.013470 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:45.013470 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:26:45.013470 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:45.013470 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:26:45.015024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.050 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:45.015024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:26:45.015024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:45.015024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:26:45.015024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:26:45.015024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:26:45.015024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:26:45.015024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:45.015024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:45.015024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:45.015024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:26:45.015024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:45.015909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:45.015909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:45.015909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:26:45.015909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:45.015909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:26:45.015909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:26:45.015909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:45.015909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:26:45.015909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:45.015909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:26:45.015909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:26:45.015909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:45.015909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:26:45.016854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:45.016854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:26:45.016854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:45.016854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:26:45.016854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:26:45.016854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:26:45.016854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:26:45.016854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:26:45.016854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:45.016854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:26:45.016854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:26:45.016854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:26:45.016854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:26:45.016854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:26:45.016854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:45.017303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:45.017303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:26:45.017303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:26:45.017303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:26:45.017303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:26:45.017303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:45.017303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:45.017303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:45.017303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:26:45.017303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:45.017303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:45.017303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:45.017303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:26:45.017303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:45.017303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:26:45.017303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:26:45.017303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:45.017303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:26:45.017303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:45.017875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:26:45.017875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:26:45.017875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:45.017875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:26:45.017875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:45.017875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:26:45.175850 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:26:45.176744 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:26:45.176938 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:26:45.178167 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:26:45.178574 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:26:45.178864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:45.179879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:26:45.179879 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:45.179879 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:26:45.179879 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:26:45.179879 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:45.179879 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:26:45.179879 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:45.179879 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:26:45.181979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:45.181979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:26:45.181979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:45.181979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:26:45.181979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:26:45.181979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:26:45.181979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:26:45.181979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:45.181979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:45.181979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:45.181979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:26:45.181979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:45.182566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:45.182566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:45.182566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:26:45.182566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:45.182566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:26:45.182566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:26:45.182566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:45.182566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:26:45.182566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:45.182566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:26:45.182566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:26:45.182566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:45.182566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:26:45.183167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:45.183167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:26:45.184309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:45.184309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:26:45.184309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:26:45.184309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:26:45.184309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:26:45.184309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:26:45.184309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:45.184309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:26:45.184309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:26:45.184309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:26:45.184309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:26:45.184309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:26:45.184309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:45.184309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:45.184309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:26:45.184309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:26:45.185059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:26:45.185059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:26:45.185059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:45.185059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:45.185059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:45.185059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:26:45.185059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:45.185059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:45.185059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:45.185059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:26:45.185059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:45.185059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:26:45.185059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:26:45.185059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:45.185059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:26:45.185059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:45.185059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:26:45.185059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:26:45.185059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:45.185952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:26:45.185952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:45.185952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:26:45.185952 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:26:45.185952 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:26:45.185952 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:26:45.185952 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:26:45.186313 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:26:45.186567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:45.187252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:26:45.187252 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:45.187252 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:26:45.187252 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:26:45.187252 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:45.187252 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:26:45.187252 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:45.187252 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:26:45.188030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:45.188030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:26:45.188030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:26:45.188030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:26:45.188030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:45.188030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:26:45.188030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:45.188030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:26:45.188030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:45.188030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:26:45.188030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:45.188030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:26:45.188030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:45.188030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:26:45.188030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:26:45.188513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:45.188513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:26:45.188513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:45.188513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:26:45.188513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:26:45.188513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:45.188513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:26:45.188513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:45.188513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:26:45.188923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:45.188923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:26:45.188923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:45.188923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:26:45.188923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:26:45.188923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:26:45.188923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:26:45.188923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:26:45.188923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:26:45.188923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:26:45.188923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:45.188923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:45.189294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:45.189294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:26:45.189294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:45.189294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:45.189294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:45.189294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:26:45.189294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:45.189294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:26:45.189294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:26:45.189294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:45.189294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:26:45.189294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:45.189294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:26:45.189294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:26:45.189676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:45.189676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:26:45.189676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:45.189676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:26:45.190030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:45.190030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:26:45.190030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:26:45.190030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:26:45.190030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:26:45.190030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:26:45.190030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:45.190030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:26:45.190030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:26:45.190030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:26:45.190030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:26:45.190030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:26:45.190030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:45.190030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:45.190030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:26:45.190030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:26:45.190579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:26:45.190579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:26:45.190579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:26:45.190579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:26:45.190579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:26:45.190579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:45.190579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:45.190579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:45.190579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:26:45.190579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:45.190579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:45.190579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:45.190579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:26:45.190579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:45.190579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:26:45.190579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:26:45.190579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:45.190579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:26:45.191078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:45.191078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:26:45.191078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:26:45.191078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:45.191078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:26:45.191078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:45.191078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:26:45.215048 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:26:45.215536 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:26:45.215661 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:26:45.215822 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:26:45.216357 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:26:45.216745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:45.217455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:26:45.217455 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:45.217455 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:26:45.217455 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:26:45.217455 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:45.217455 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:26:45.217455 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:45.217455 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:26:45.218368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:45.218368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:26:45.218368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:26:45.218368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:26:45.218368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:45.218368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:26:45.218368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:45.218368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:26:45.218368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:45.218368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:26:45.218368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:45.218368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:26:45.218368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:45.218368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:26:45.218368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:26:45.219241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:45.219241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:26:45.219241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:45.219241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:26:45.219241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:26:45.219241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:45.219241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:26:45.219241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:45.219241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:26:45.219835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:45.219835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:26:45.219835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:45.219835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:26:45.219835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:26:45.219835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:26:45.219835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:26:45.219835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:26:45.219835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:26:45.219835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:45.219835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:45.220278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:45.220278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:26:45.220278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:45.220278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:45.220278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:45.220278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:26:45.220278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:45.220278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:26:45.220278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:26:45.220278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:45.220278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:26:45.220278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:45.220278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:26:45.220278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:26:45.220667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:45.220667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:26:45.220667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:45.220667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:26:45.220667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:45.220667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:26:45.220667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:26:45.220667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:26:45.220667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:26:45.220667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:26:45.220667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:45.220667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:26:45.220667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:26:45.220667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:26:45.221085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:26:45.221085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:26:45.221085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:45.221085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:45.221085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:26:45.221085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:26:45.221085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:26:45.221085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:26:45.221085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:26:45.221085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:26:45.221085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:45.221085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:45.221085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:45.221085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:26:45.221085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:45.221085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:45.221085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:45.221085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:26:45.221085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:45.221807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:26:45.221807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:26:45.221807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:45.221807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:26:45.221807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:45.221807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:26:45.221807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:26:45.221807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:45.221807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:26:45.221807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:45.221807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:26:45.222425 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:26:45.222758 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:26:45.222855 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:26:45.223006 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:26:45.223397 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:26:45.223777 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:26:45.223984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:45.224694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:26:45.224694 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:45.224694 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:26:45.224694 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:26:45.224694 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:45.224694 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:26:45.224694 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:45.224694 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:26:45.225660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:45.225660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:26:45.225660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:45.225660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:26:45.225660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:26:45.225660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:26:45.225660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:26:45.225660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:45.225660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:45.225660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:45.225660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:26:45.225660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:45.226060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:45.226060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:45.226060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:26:45.226060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:45.226060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:26:45.226060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:26:45.226060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:45.226060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:26:45.226060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:45.226060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:26:45.226060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:26:45.226060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:45.226060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:26:45.226448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:45.226448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:26:45.226601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:45.226601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:26:45.226601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:26:45.226601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:26:45.226601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:26:45.226601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:26:45.226601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:45.226601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:26:45.226601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:26:45.226601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:26:45.226601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:26:45.226601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:26:45.226601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:45.226601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:45.226601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:26:45.226601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:26:45.227218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:26:45.227218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:26:45.227218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:45.227218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:45.227218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:45.227218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:26:45.227218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:45.227218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:45.227218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:45.227218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:26:45.227218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:45.227218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:26:45.227218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:26:45.227218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:45.227218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:26:45.227218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:45.227218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:26:45.227218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:26:45.227218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:45.227764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:26:45.227764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:45.227764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:26:49.963565 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:26:49.964448 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:26:49.964645 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:26:49.965880 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:26:49.996565 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:26:49.996838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:49.997576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:26:49.997576 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:49.997576 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:26:49.997576 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:26:49.997576 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:49.997576 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:26:49.997576 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:49.997576 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:26:49.998581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:49.998581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:26:49.998581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:49.998581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:26:49.998581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:26:49.998581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:26:49.998581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:26:49.998581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:49.998581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:49.998581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:49.998581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:26:49.998581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:49.998871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:49.998871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:49.998871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:26:49.998871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:49.998871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:26:49.998871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:26:49.998871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:49.998871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:26:49.998871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:49.998871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:26:49.998871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:26:49.998871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:49.998871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:26:49.999111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:49.999111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:26:49.999621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:49.999621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:26:49.999621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:26:49.999621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:26:49.999621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:26:49.999621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:26:49.999621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:49.999621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:26:49.999621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:26:49.999621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:26:49.999621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:26:49.999621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:26:49.999621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:49.999621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:49.999621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:26:49.999621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:26:49.999933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:26:49.999933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:26:49.999933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:49.999933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:49.999933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:49.999933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:26:49.999933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:49.999933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:49.999933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:49.999933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:26:49.999933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:49.999933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:26:49.999933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:26:49.999933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:49.999933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:26:49.999933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:49.999933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:26:49.999933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:26:49.999933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:50.000291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:26:50.000291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:50.000291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:26:50.176780 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:26:50.177564 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:26:50.177747 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:26:50.178913 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:26:50.179337 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:26:50.179724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:50.180727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:26:50.180727 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:50.180727 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:26:50.180727 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:26:50.180727 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:50.180727 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:26:50.180727 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:50.180727 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:26:50.181782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:50.181782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:26:50.181782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:50.181782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:26:50.181782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:26:50.181782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:26:50.181782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:26:50.181782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:50.181782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:50.181782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:50.181782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:26:50.181782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:50.182276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:50.182276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:50.182276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:26:50.182276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:50.182276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:26:50.182276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:26:50.182276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:50.182276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:26:50.182276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:50.182276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:26:50.182276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:26:50.182276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:50.182276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:26:50.182624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:50.182624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:26:50.182951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:50.182951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:26:50.182951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:26:50.182951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:26:50.182951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:26:50.182951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:26:50.182951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:50.182951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:26:50.182951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:26:50.182951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:26:50.182951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:26:50.182951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:26:50.182951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:50.182951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:50.182951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:26:50.182951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:26:50.183985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:26:50.183985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:26:50.183985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:50.183985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:50.183985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:50.183985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:26:50.183985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:50.183985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:50.183985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:50.183985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:26:50.183985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:50.183985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:26:50.183985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:26:50.183985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:50.183985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:26:50.183985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:50.183985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:26:50.183985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:26:50.183985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:50.184855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:26:50.184855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:50.184855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:26:50.185159 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:26:50.185511 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:26:50.185612 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:26:50.185742 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:26:50.186133 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:26:50.186389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:50.187199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:26:50.187199 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:50.187199 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:26:50.187199 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:26:50.187199 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:50.187199 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:26:50.187199 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:50.187199 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:26:50.187951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:50.187951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:26:50.187951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:26:50.187951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:26:50.187951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:50.187951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:26:50.187951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:50.187951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:26:50.187951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:50.187951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:26:50.187951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:50.187951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:26:50.187951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:50.187951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:26:50.187951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:26:50.188330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:50.188330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:26:50.188330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:50.188330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:26:50.188330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:26:50.188330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:50.188330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:26:50.188330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:50.188330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:26:50.188814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:50.188814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:26:50.188814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:50.188814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:26:50.188814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:26:50.188814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:26:50.188814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:26:50.188814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:26:50.188814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:26:50.188814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:26:50.188814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:50.188814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:50.189737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:50.189737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:26:50.189737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:50.189737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:50.189737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:50.189737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:26:50.189737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:50.189737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:26:50.189737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:26:50.189737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:50.189737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:26:50.189737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:50.189737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:26:50.189737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:26:50.190422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:50.190422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:26:50.190422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:50.190422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:26:50.190422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:50.190422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:26:50.190422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:26:50.190422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:26:50.190422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:26:50.190422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:26:50.190422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:50.190422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:26:50.190422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:26:50.190422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:26:50.190820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:26:50.190820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:26:50.190820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:50.190820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:50.190820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:26:50.190820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:26:50.190820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:26:50.190820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:26:50.190820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:26:50.190820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:26:50.190820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:26:50.190820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:50.190820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:50.190820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:50.190820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:26:50.190820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:50.190820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:50.190820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:50.190820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:26:50.191368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:50.191368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:26:50.191368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:26:50.191368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:50.191368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:26:50.191368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:50.191368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:26:50.191368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:26:50.191368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:50.191368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:26:50.191368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:50.191368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:26:50.215599 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:26:50.215969 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:26:50.216145 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:26:50.216318 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:26:50.216936 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:26:50.217449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:50.218351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:26:50.218351 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:50.218351 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:26:50.218351 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:26:50.218351 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:50.218351 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:26:50.218351 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:50.218351 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:26:50.219538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:50.219538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:26:50.219538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:26:50.219538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:26:50.219538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:50.219538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:26:50.219538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:50.219538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:26:50.219538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:50.219538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:26:50.219538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:50.219538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:26:50.219538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:50.219538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:26:50.219538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:26:50.220740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:50.220740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:26:50.220740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:50.220740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:26:50.220740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:26:50.220740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:50.220740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:26:50.220740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:50.220740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:26:50.221209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:50.221209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:26:50.221209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:50.221209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:26:50.221209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:26:50.221209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:26:50.221209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:26:50.221209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:26:50.221209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:26:50.221209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:50.221209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:50.221833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:50.221833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:26:50.221833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:50.221833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:50.221833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:50.221833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:26:50.221833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:50.221833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:26:50.221833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:26:50.221833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:50.221833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:26:50.221833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:50.221833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:26:50.221833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:26:50.222647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:50.222647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:26:50.222647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:50.222647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:26:50.222647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:50.222647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:26:50.222647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:26:50.222647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:26:50.222647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:26:50.222647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:26:50.222647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:50.222647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:26:50.222647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:26:50.222647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:26:50.224878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:26:50.224878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:26:50.224878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:50.224878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:50.224878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:26:50.224878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:26:50.224878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:26:50.224878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:26:50.224878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:26:50.224878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:26:50.224878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:50.224878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:50.224878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:50.224878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:26:50.224878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:50.224878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:50.224878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:50.224878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:26:50.224878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:50.226765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:26:50.226765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:26:50.226765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:50.226765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:26:50.226765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:50.226765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:26:50.226765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:26:50.226765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:50.226765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:26:50.226765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:50.226765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:26:50.226765 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:26:50.226765 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:26:50.227342 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:26:50.227342 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:26:50.227342 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:26:50.227342 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:26:50.227342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:50.227342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:26:50.227342 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:50.227342 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:26:50.227342 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:26:50.227684 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:50.227684 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:26:50.227684 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:50.227684 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:26:50.227684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:50.227684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:26:50.227684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:50.227684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:26:50.227684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:26:50.227684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:26:50.227684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:26:50.227684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:50.227684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:50.227684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:50.227954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:26:50.227954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:50.227954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:50.227954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:50.227954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:26:50.227954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:50.227954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:26:50.227954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:26:50.227954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:50.227954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:26:50.227954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:50.227954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:26:50.227954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:26:50.227954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:50.228196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:26:50.228196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:50.228196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:26:50.228926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:50.228926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:26:50.228926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:26:50.228926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:26:50.228926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:26:50.228926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:26:50.228926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:50.228926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:26:50.228926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:26:50.228926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:26:50.228926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:26:50.228926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:26:50.228926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:50.228926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:50.228926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:26:50.228926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:26:50.229228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:26:50.229228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:26:50.229228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:50.229228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:50.229228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:50.229228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:26:50.229228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:50.229228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:50.229228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:50.229228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:26:50.229228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:50.229228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:26:50.229228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:26:50.229228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:50.229228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:26:50.229228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:50.229228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:26:50.229228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:26:50.229228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:50.229595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:26:50.229595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:50.229595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:26:54.964803 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:26:54.965813 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:26:54.965813 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:26:54.966685 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:26:55.008622 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:26:55.009192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:55.009998 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:26:55.009998 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:55.009998 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:26:55.009998 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:26:55.009998 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:55.009998 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:26:55.009998 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:55.009998 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:26:55.011289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:55.011289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:26:55.011289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:55.011289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:26:55.011289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:26:55.011289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:26:55.011289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:26:55.011289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:55.011289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:55.011289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:55.011289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:26:55.011289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:55.012876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:55.012876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:55.012876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:26:55.012876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:55.012876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:26:55.012876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:26:55.012876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:55.012876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:26:55.012876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:55.012876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:26:55.012876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:26:55.012876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:55.012876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:26:55.013806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:55.013806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:26:55.013806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:55.013806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:26:55.013806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:26:55.013806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:26:55.013806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:26:55.013806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:26:55.013806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:55.013806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:26:55.013806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:26:55.013806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:26:55.013806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:26:55.013806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:26:55.013806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:55.014559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:55.014559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:26:55.014559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:26:55.014559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:26:55.014559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:26:55.014559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:55.014559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:55.014559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:55.014559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:26:55.014559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:55.014559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:55.014559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:55.014559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:26:55.014559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:55.014559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:26:55.014559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:26:55.014559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:55.014559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:26:55.014559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:55.018116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:26:55.018116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:26:55.018116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:55.018116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:26:55.018116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:55.018116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:26:55.177751 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:26:55.178542 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:26:55.178733 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:26:55.179870 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:26:55.180219 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:26:55.180382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:55.181118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:26:55.181118 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:55.181118 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:26:55.181118 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:26:55.181118 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:55.181118 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:26:55.181118 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:55.181118 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:26:55.182633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:55.182633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:26:55.182633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:55.182633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:26:55.182633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:26:55.182633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:26:55.182633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:26:55.182633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:55.182633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:55.182633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:55.182633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:26:55.182633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:55.183254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:55.183254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:55.183254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:26:55.183254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:55.183254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:26:55.183254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:26:55.183254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:55.183254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:26:55.183254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:55.183254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:26:55.183254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:26:55.183254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:55.183254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:26:55.183874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:55.183874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:26:55.184259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:55.184259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:26:55.184259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:26:55.184259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:26:55.184259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:26:55.184259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:26:55.184259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:55.184259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:26:55.184259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:26:55.184259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:26:55.184259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:26:55.184259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:26:55.184259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:55.184259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:55.184259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:26:55.184259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:26:55.185022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:26:55.185022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:26:55.185022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:55.185022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:55.185022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:55.185022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:26:55.185022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:55.185022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:55.185022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:55.185022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:26:55.185022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:55.185022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:26:55.185022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:26:55.185022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:55.185022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:26:55.185022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:55.185022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:26:55.185022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:26:55.185022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:55.185455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:26:55.185455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:55.185455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:26:55.185658 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:26:55.186017 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:26:55.186187 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:26:55.186366 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:26:55.186773 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:26:55.186965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:55.187503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:26:55.187503 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:55.187503 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:26:55.187503 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:26:55.187503 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:55.187503 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:26:55.187503 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:55.187503 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:26:55.188247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:55.188247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:26:55.188247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:26:55.188247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:26:55.188247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:55.188247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:26:55.188247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:55.188247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:26:55.188247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:55.188247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:26:55.188247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:55.188247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:26:55.188247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:55.188247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:26:55.188247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:26:55.188586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:55.188586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:26:55.188586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:55.188586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:26:55.188586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:26:55.188586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:55.188586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:26:55.188586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:55.188586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:26:55.189055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:55.189055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:26:55.189055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:55.189055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:26:55.189055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:26:55.189055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:26:55.189055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:26:55.189055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:26:55.189055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:26:55.189055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:26:55.189055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:55.189055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:55.189311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:55.189311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:26:55.189311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:55.189311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:55.189311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:55.189311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:26:55.189311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:55.189311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:26:55.189311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:26:55.189311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:55.189311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:26:55.189311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:55.189311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:26:55.189311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:26:55.189698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:55.189698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:26:55.189698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:55.189698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:26:55.190116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:55.190116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:26:55.190116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:26:55.190116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:26:55.190116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:26:55.190116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:26:55.190116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:55.190116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:26:55.190116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:26:55.190116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:26:55.190116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:26:55.190116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:26:55.190116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:55.190116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:55.190116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:26:55.190116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:26:55.190542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:26:55.190542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:26:55.190542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:26:55.190542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:26:55.190542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:26:55.190542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:55.190542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:55.190542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:55.190542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:26:55.190542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:55.190542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:55.190542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:55.190542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:26:55.190542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:55.190542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:26:55.190542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:26:55.190542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:55.190542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:26:55.191614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:55.191614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:26:55.191614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:26:55.191614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:55.191614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:26:55.191614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:55.191614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:26:55.216247 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:26:55.216990 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:26:55.217162 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:26:55.217360 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:26:55.218005 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:26:55.218304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:55.219004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:26:55.219004 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:55.219004 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:26:55.219004 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:26:55.219004 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:55.219004 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:26:55.219004 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:55.219004 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:26:55.220263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:55.220263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:26:55.220263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:26:55.220263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:26:55.220263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:55.220263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:26:55.220263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:55.220263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:26:55.220263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:55.220263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:26:55.220263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:55.220263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:26:55.220263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:55.220263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:26:55.220263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:26:55.220981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:55.220981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:26:55.220981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:55.220981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:26:55.220981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:26:55.220981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:55.220981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:26:55.220981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:55.220981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:26:55.221561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:55.221561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:26:55.221561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:55.221561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:26:55.221561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:26:55.221561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:26:55.221561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:26:55.221561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:26:55.221561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:26:55.221561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:55.221561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:55.222190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:55.222190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:26:55.222190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:55.222190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:55.222190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:55.222190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:26:55.222190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:55.222190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:26:55.222190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:26:55.222190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:55.222190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:26:55.222190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:55.222190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:26:55.222190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:26:55.222900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:55.222900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:26:55.222900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:55.222900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:26:55.222900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:55.222900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:26:55.222900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:26:55.222900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:26:55.222900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:26:55.222900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:26:55.222900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:55.222900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:26:55.222900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:26:55.222900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:26:55.224077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:26:55.224077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:26:55.224077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:55.224077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:55.224077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:26:55.224077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:26:55.224077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:26:55.224077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:26:55.224077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:26:55.224077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:26:55.224077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:55.224077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:55.224077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:55.224077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:26:55.224077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:55.224077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:55.224077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:55.224077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:26:55.224077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:55.225966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:26:55.225966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:26:55.225966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:55.225966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:26:55.225966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:55.225966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:26:55.225966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:26:55.225966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:55.225966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:26:55.225966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:55.225966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:26:55.225966 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:26:55.225966 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:26:55.227247 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:26:55.227247 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:26:55.227247 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:26:55.227247 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:26:55.227247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:55.227247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:26:55.227247 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:55.227247 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:26:55.227247 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:26:55.228340 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:55.228340 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:26:55.228340 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:55.228340 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:26:55.228340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:55.228340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:26:55.228340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:55.228340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:26:55.228340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:26:55.228340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:26:55.228340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:26:55.228340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:55.228340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:55.228340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:55.229106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:26:55.229106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:55.229106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:26:55.229106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:55.229106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:26:55.229106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:55.229106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:26:55.229106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:26:55.229106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:55.229106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:26:55.229106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:55.229106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:26:55.229106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:26:55.229106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:55.229717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:26:55.229717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:55.229717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:26:55.229717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:55.229717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:26:55.229717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:26:55.229717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:26:55.229717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:26:55.229717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:26:55.229717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:55.229717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:26:55.229717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:26:55.229717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:26:55.229717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:26:55.229717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:26:55.229717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:26:55.230398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:26:55.230398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:26:55.230398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:26:55.230398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:26:55.230398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:26:55.230398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:55.230398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:55.230398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:26:55.230398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:26:55.230398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:26:55.230398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:26:55.230398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:26:55.230398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:26:55.230398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:26:55.230398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:26:55.230398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:26:55.230398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:26:55.230398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:26:55.230398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:26:55.231212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:26:55.231212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:26:55.231212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:26:55.231212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:26:55.231212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:26:55.231212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:26:59.966239 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:26:59.968883 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:26:59.968988 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:26:59.970278 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:27:00.004028 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:27:00.004028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:00.004807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:27:00.004807 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:00.004807 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:27:00.004807 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:27:00.004807 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:00.004807 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:27:00.004807 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:00.004807 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:27:00.005621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:00.005621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:27:00.005621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:00.005621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:27:00.005621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:27:00.005621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:27:00.005621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:27:00.005621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:00.005621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:00.005621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:00.005621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:27:00.005621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:00.006126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:00.006126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:00.006126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:27:00.006126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:00.006126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:27:00.006126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:27:00.006126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:00.006126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:27:00.006126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:00.006126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:27:00.006126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:27:00.006126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:00.006126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:27:00.006546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:00.006546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:27:00.006546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:00.006546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:27:00.006546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:27:00.006546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:27:00.006546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:27:00.006546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:27:00.006546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:00.006546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:27:00.006546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:27:00.006546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:27:00.006546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:27:00.006546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:27:00.006546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:00.006999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:00.006999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:27:00.006999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:27:00.006999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:27:00.006999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:27:00.006999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:00.006999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:00.006999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:00.006999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:27:00.006999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:00.006999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:00.006999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:00.006999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:27:00.006999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:00.006999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:27:00.006999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:27:00.006999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:00.006999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:27:00.006999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:00.007649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:27:00.007649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:27:00.007649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:00.007649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:27:00.007649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:00.007649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:27:00.179084 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:27:00.180293 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:27:00.180293 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:27:00.181701 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:27:00.182209 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:27:00.182680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:00.183786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:27:00.183786 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:00.183786 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:27:00.183786 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:27:00.183786 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:00.183786 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:27:00.183786 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:00.183786 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:27:00.184897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:00.184897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:27:00.184897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:00.184897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:27:00.184897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:27:00.184897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:27:00.184897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:27:00.184897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:00.184897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:00.184897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:00.184897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:27:00.184897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:00.185283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:00.185283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:00.185283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:27:00.185283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:00.185283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:27:00.185283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:27:00.185283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:00.185283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:27:00.185283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:00.185283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:27:00.185283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:27:00.185283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:00.185283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:27:00.185663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:00.185663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:27:00.185918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:00.185918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:27:00.185918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:27:00.185918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:27:00.185918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:27:00.185918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:27:00.185918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:00.185918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:27:00.185918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:27:00.185918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:27:00.185918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:27:00.185918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:27:00.185918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:00.185918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:00.185918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:27:00.185918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:27:00.186369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:27:00.186369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:27:00.186369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:00.186369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:00.186369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:00.186369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:27:00.186369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:00.186369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:00.186369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:00.186369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:27:00.186369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:00.186369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:27:00.186369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:27:00.186369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:00.186369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:27:00.186369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:00.186369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:27:00.186369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:27:00.186369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:00.186867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:27:00.186867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:00.186867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:27:00.186867 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:27:00.187007 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:27:00.187124 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:27:00.187252 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:27:00.187637 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:27:00.187841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:00.188362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:27:00.188362 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:00.188362 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:27:00.188362 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:27:00.188362 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:00.188362 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:27:00.188362 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:00.188362 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:27:00.189186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:00.189186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:27:00.189186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:27:00.189186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:27:00.189186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:00.189186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:27:00.189186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:00.189186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:27:00.189186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:00.189186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:27:00.189186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:00.189186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:27:00.189186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:00.189186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:27:00.189186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:27:00.189710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:00.189710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:27:00.189710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:00.189710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:27:00.189710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:27:00.189710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:00.189710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:27:00.189710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:00.189710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:27:00.189942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:00.189942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:27:00.189942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:00.189942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:27:00.189942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:27:00.189942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:27:00.189942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:27:00.189942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:27:00.189942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:27:00.189942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:27:00.189942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:00.189942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:00.190293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:00.190293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:27:00.190293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:00.190293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:00.190293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:00.190293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:27:00.190293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:00.190293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:27:00.190293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:27:00.190293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:00.190293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:27:00.190293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:00.190293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:27:00.190293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:27:00.190602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:00.190602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:27:00.190602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:00.190602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:27:00.190936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:00.190936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:27:00.190936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:27:00.190936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:27:00.190936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:27:00.190936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:27:00.190936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:00.190936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:27:00.190936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:27:00.190936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:27:00.190936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:27:00.190936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:27:00.190936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:00.190936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:00.190936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:27:00.190936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:27:00.191430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:27:00.191430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:27:00.191430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:27:00.191430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:27:00.191430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:27:00.191430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:00.191430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:00.191430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:00.191430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:27:00.191430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:00.191430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:00.191430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:00.191430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:27:00.191430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:00.191430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:27:00.191430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:27:00.191430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:00.191430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:27:00.191773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:00.191773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:27:00.191773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:27:00.191773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:00.191773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:27:00.191773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:00.191773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:27:00.216658 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:27:00.217050 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:27:00.217140 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:27:00.217339 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:27:00.217678 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:27:00.217890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:00.218435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:27:00.218435 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:00.218435 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:27:00.218435 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:27:00.218435 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:00.218435 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:27:00.218435 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:00.218435 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:27:00.219723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:00.219723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:27:00.219723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:27:00.219723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:27:00.219723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:00.219723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:27:00.219723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:00.219723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:27:00.219723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:00.219723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:27:00.219723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:00.219723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:27:00.219723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:00.219723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:27:00.219723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:27:00.220156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:00.220156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:27:00.220156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:00.220156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:27:00.220156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:27:00.220156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:00.220156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:27:00.220156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:00.220156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:27:00.221672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:00.221672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:27:00.221672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:00.221672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:27:00.221672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:27:00.221672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:27:00.221672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:27:00.221672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:27:00.221672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:27:00.221672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:00.221672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:00.222061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:00.222061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:27:00.222061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:00.222061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:00.222061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:00.222061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:27:00.222061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:00.222061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:27:00.222061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:27:00.222061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:00.222061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:27:00.222061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:00.222061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:27:00.222061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:27:00.222411 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:00.222411 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:27:00.222411 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:00.222411 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:27:00.223088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:00.223088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:27:00.223088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:27:00.223088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:27:00.223088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:27:00.223088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:27:00.223088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:00.223088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:27:00.223088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:27:00.223088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:27:00.223088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:27:00.223088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:27:00.223088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:00.223088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:00.223088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:27:00.223088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:27:00.223088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:27:00.223579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:27:00.223579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:27:00.223579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:27:00.223579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:00.223579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:00.223579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:00.223579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:27:00.223579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:00.223579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:00.223579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:00.223579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:27:00.223579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:00.223579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:27:00.223579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:27:00.223579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:00.223579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:27:00.223579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:00.223579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:27:00.223579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:27:00.224082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:00.224082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:27:00.224082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:00.224082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:27:00.224082 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:27:00.224082 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:27:00.224082 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:27:00.224489 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:27:00.224568 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:27:00.224791 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:27:00.224944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:00.225427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:27:00.225427 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:00.225427 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:27:00.225427 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:27:00.225427 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:00.225427 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:27:00.225427 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:00.225427 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:27:00.226252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:00.226252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:27:00.226252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:00.226252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:27:00.226252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:27:00.226252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:27:00.226252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:27:00.226252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:00.226252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:00.226252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:00.226252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:27:00.226252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:00.226747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:00.226747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:00.226747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:27:00.226747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:00.226747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:27:00.226747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:27:00.226747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:00.226747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:27:00.226747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:00.226747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:27:00.226747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:27:00.226747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:00.226747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:27:00.227081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:00.227081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:27:00.227184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:00.227184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:27:00.227184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:27:00.227184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:27:00.227184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:27:00.227184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:27:00.227184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:00.227184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:27:00.227184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:27:00.227184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:27:00.227184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:27:00.227184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:27:00.227184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:00.227184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:00.227184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:27:00.227184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:27:00.227663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:27:00.227663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:27:00.227663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:00.227663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:00.227663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:00.227663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:27:00.227663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:00.227663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:00.227663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:00.227663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:27:00.227663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:00.227663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:27:00.227663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:27:00.227663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:00.227663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:27:00.227663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:00.227663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:27:00.227663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:27:00.227663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:00.228148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:27:00.228148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:00.228148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:27:04.968970 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:27:04.969830 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:27:04.970242 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:27:04.971314 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:27:05.010800 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:27:05.011169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:05.011976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:27:05.011976 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:05.011976 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:27:05.011976 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:27:05.011976 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:05.011976 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:27:05.011976 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:05.011976 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:27:05.012900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:05.012900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:27:05.012900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:05.012900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:27:05.012900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:27:05.012900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:27:05.012900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:27:05.012900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:05.012900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:05.012900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:05.012900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:27:05.012900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:05.013680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:05.013680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:05.013680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:27:05.013680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:05.013680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:27:05.013680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:27:05.013680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:05.013680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:27:05.013680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:05.013680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:27:05.013680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:27:05.013680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:05.013680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:27:05.014419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:05.014419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:27:05.014419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:05.014419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:27:05.014419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:27:05.014419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:27:05.014419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:27:05.014419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:27:05.014419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:05.014419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:27:05.014419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:27:05.014419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:27:05.014419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:27:05.014419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:27:05.014419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:05.015323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:05.015323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:27:05.015323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:27:05.015323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:27:05.015323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:27:05.015323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:05.015323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:05.015323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:05.015323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:27:05.015323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:05.015323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:05.015323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:05.015323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:27:05.015323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:05.015323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:27:05.015323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:27:05.015323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:05.015323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:27:05.015323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:05.016561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:27:05.016561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:27:05.016561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:05.016561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:27:05.016561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:05.016561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:27:05.179761 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:27:05.180252 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:27:05.180475 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:27:05.181519 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:27:05.181816 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:27:05.181929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:05.182826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:27:05.182826 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:05.182826 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:27:05.182826 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:27:05.182826 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:05.182826 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:27:05.182826 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:05.182826 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:27:05.184647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:05.184647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:27:05.184647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:05.184647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:27:05.184647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:27:05.184647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:27:05.184647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:27:05.184647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:05.184647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:05.184647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:05.184647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:27:05.184647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:05.185018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:05.185018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:05.185018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:27:05.185018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:05.185018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:27:05.185018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:27:05.185018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:05.185018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:27:05.185018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:05.185018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:27:05.185018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:27:05.185018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:05.185018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:27:05.185412 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:05.185412 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:27:05.186188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:05.186188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:27:05.186188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:27:05.186188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:27:05.186188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:27:05.186188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:27:05.186188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:05.186188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:27:05.186188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:27:05.186188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:27:05.186188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:27:05.186188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:27:05.186188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:05.186188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:05.186188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:27:05.186188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:27:05.186705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:27:05.186705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:27:05.186705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:05.186705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:05.186705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:05.186705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:27:05.186705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:05.186705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:05.186705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:05.186705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:27:05.186705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:05.186705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:27:05.186705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:27:05.186705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:05.186705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:27:05.186705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:05.186705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:27:05.186705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:27:05.186705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:05.187468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:27:05.187468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:05.187468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:27:05.187852 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:27:05.188124 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:27:05.188212 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:27:05.188336 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:27:05.188669 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:27:05.188849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:05.189358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:27:05.189358 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:05.189358 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:27:05.189358 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:27:05.189358 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:05.189358 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:27:05.189358 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:05.189358 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:27:05.190078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:05.190078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:27:05.190078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:27:05.190078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:27:05.190078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:05.190078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:27:05.190078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:05.190078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:27:05.190078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:05.190078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:27:05.190078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:05.190078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:27:05.190078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:05.190078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:27:05.190078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:27:05.190918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:05.190918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:27:05.190918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:05.190918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:27:05.190918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:27:05.190918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:05.190918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:27:05.190918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:05.190918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:27:05.190918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:05.190918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:27:05.190918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:05.190918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:27:05.190918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:27:05.191557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:27:05.191557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:27:05.191557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:27:05.191557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:27:05.191557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:27:05.191557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:05.191557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:05.191557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:05.191557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:27:05.191557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:05.191557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:05.191557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:05.191557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:27:05.192019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:05.192019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:27:05.192019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:27:05.192019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:05.192019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:27:05.192019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:05.192019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:27:05.192019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:27:05.192019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:05.192019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:27:05.192019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:05.192019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:27:05.192387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:05.192387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:27:05.192387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:27:05.192387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:27:05.192387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:27:05.192387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:27:05.192387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:05.192387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:27:05.192387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:27:05.192387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:27:05.192387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:27:05.192387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:27:05.192387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:05.192387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:05.192387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:27:05.192387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:27:05.192911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:27:05.192911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:27:05.192911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:27:05.192911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:27:05.192911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:27:05.192911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:05.192911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:05.192911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:05.192911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:27:05.192911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:05.192911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:05.192911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:05.192911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:27:05.192911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:05.192911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:27:05.192911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:27:05.192911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:05.192911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:27:05.193537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:05.193537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:27:05.193537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:27:05.193537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:05.193537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:27:05.193537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:05.193537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:27:05.217108 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:27:05.217945 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:27:05.218172 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:27:05.218457 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:27:05.219680 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:27:05.220034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:05.220919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:27:05.220919 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:05.220919 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:27:05.220919 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:27:05.220919 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:05.220919 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:27:05.220919 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:05.220919 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:27:05.222070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:05.222070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:27:05.222070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:27:05.222070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:27:05.222070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:05.222070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:27:05.222070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:05.222070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:27:05.222070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:05.222070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:27:05.222070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:05.222070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:27:05.222070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:05.222070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:27:05.222070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:27:05.222681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:05.222681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:27:05.222681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:05.222681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:27:05.222681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:27:05.222681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:05.222681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:27:05.222681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:05.222681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:27:05.222972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:05.222972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:27:05.222972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:05.222972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:27:05.222972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:27:05.222972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:27:05.222972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:27:05.222972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:27:05.222972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:27:05.222972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:05.222972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:05.223305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:05.223305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:27:05.223305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:05.223305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:05.223305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:05.223305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:27:05.223305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:05.223305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:27:05.223305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:27:05.223305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:05.223305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:27:05.223305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:05.223305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:27:05.223305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:27:05.223620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:05.223620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:27:05.223620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:05.223620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:27:05.224121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:05.224121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:27:05.224121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:27:05.224121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:27:05.224121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:27:05.224121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:27:05.224121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:05.224121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:27:05.224121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:27:05.224121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:27:05.224121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:27:05.224121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:27:05.224121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:05.224121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:05.224121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:27:05.224121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:27:05.224121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:27:05.224625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:27:05.224625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:27:05.224625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:27:05.224625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:05.224625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:05.224625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:05.224625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:27:05.224625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:05.224625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:05.224625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:05.224625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:27:05.224625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:05.224625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:27:05.224625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:27:05.224625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:05.224625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:27:05.224625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:05.224625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:27:05.224625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:27:05.225096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:05.225096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:27:05.225096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:05.225096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:27:05.225226 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:27:05.225391 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:27:05.225499 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:27:05.225687 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:27:05.226194 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:27:05.226429 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:27:05.226598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:05.227096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:27:05.227096 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:05.227096 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:27:05.227096 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:27:05.227096 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:05.227096 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:27:05.227096 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:05.227096 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:27:05.228004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:05.228004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:27:05.228004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:05.228004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:27:05.228004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:27:05.228004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:27:05.228004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:27:05.228004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:05.228004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:05.228004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:05.228004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:27:05.228004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:05.228291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:05.228291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:05.228291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:27:05.228291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:05.228291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:27:05.228291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:27:05.228291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:05.228291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:27:05.228291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:05.228291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:27:05.228291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:27:05.228291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:05.228291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:27:05.228657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:05.228657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:27:05.229071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:05.229071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:27:05.229071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:27:05.229071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:27:05.229071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:27:05.229071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:27:05.229071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:05.229071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:27:05.229071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:27:05.229071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:27:05.229071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:27:05.229071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:27:05.229071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:05.229071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:05.229071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:27:05.229071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:27:05.229547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:27:05.229547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:27:05.229547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:05.229547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:05.229547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:05.229547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:27:05.229547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:05.229547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:05.229547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:05.229547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:27:05.229547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:05.229547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:27:05.229547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:27:05.229547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:05.229547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:27:05.229547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:05.229547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:27:05.229547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:27:05.229547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:05.230147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:27:05.230147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:05.230147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:27:09.969834 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:27:09.970847 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:27:09.970847 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:27:09.971187 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:27:10.005289 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:27:10.005460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:10.005907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:27:10.005907 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:10.005907 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:27:10.005907 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:27:10.005907 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:10.005907 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:27:10.005907 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:10.005907 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:27:10.006617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:10.006617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:27:10.006617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:10.006617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:27:10.006617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:27:10.006617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:27:10.006617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:27:10.006617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:10.006617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:10.006617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:10.006617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:27:10.006617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:10.006924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:10.006924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:10.006924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:27:10.006924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:10.006924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:27:10.006924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:27:10.006924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:10.006924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:27:10.006924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:10.006924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:27:10.006924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:27:10.006924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:10.006924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:27:10.007272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:10.007272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:27:10.007403 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:10.007403 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:27:10.007403 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:27:10.007403 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:27:10.007403 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:27:10.007403 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:27:10.007403 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:10.007403 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:27:10.007403 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:27:10.007403 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:27:10.007403 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:27:10.007403 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:27:10.007403 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:10.007403 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:10.007403 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:27:10.007403 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:27:10.008240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:27:10.008240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:27:10.008240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:10.008240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:10.008240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:10.008240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:27:10.008240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:10.008240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:10.008240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:10.008240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:27:10.008240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:10.008240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:27:10.008240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:27:10.008240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:10.008240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:27:10.008240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:10.008240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:27:10.008240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:27:10.008240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:10.009542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:27:10.009542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:10.009542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:27:10.180652 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:27:10.181162 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:27:10.181420 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:27:10.182654 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:27:10.182976 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:27:10.183312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:10.184512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:27:10.184512 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:10.184512 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:27:10.184512 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:27:10.184512 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:10.184512 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:27:10.184512 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:10.184512 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:27:10.186057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:10.186057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:27:10.186057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:10.186057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:27:10.186057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:27:10.186057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:27:10.186057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:27:10.186057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:10.186057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:10.186057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:10.186057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:27:10.186057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:10.186493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:10.186493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:10.186493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:27:10.186493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:10.186493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:27:10.186493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:27:10.186493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:10.186493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:27:10.186493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:10.186493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:27:10.186493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:27:10.186493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:10.186493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:27:10.186842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:10.186842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:27:10.187709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:10.187709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:27:10.187709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:27:10.187709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:27:10.187709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:27:10.187709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:27:10.187709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:10.187709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:27:10.187709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:27:10.187709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:27:10.187709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:27:10.187709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:27:10.187709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:10.187709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:10.187709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:27:10.187709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:27:10.188203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:27:10.188203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:27:10.188203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:10.188203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:10.188203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:10.188203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:27:10.188203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:10.188203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:10.188203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:10.188203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:27:10.188203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:10.188203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:27:10.188203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:27:10.188203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:10.188203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:27:10.188203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:10.188203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:27:10.188203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:27:10.188203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:10.188785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:27:10.188785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:10.188785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:27:10.188785 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:27:10.188785 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:27:10.188785 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:27:10.188986 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:27:10.189248 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:27:10.189457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:10.189882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:27:10.189882 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:10.189882 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:27:10.189882 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:27:10.189882 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:10.189882 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:27:10.189882 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:10.189882 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:27:10.190658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:10.190658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:27:10.190658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:27:10.190658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:27:10.190658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:10.190658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:27:10.190658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:10.190658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:27:10.190658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:10.190658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:27:10.190658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:10.190658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:27:10.190658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:10.190658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:27:10.190658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:27:10.191138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:10.191138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:27:10.191138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:10.191138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:27:10.191138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:27:10.191138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:10.191138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:27:10.191138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:10.191138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:27:10.191507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:10.191507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:27:10.191507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:10.191507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:27:10.191507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:27:10.191507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:27:10.191507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:27:10.191507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:27:10.191507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:27:10.191507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:27:10.191507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:10.191507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:10.191883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:10.191883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:27:10.191883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:10.191883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:10.191883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:10.191883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:27:10.191883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:10.191883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:27:10.191883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:27:10.191883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:10.191883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:27:10.191883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:10.191883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:27:10.191883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:27:10.192247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:10.192247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:27:10.192247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:10.192247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:27:10.192544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:10.192544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:27:10.192544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:27:10.192544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:27:10.192544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:27:10.192544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:27:10.192544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:10.192544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:27:10.192544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:27:10.192544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:27:10.192544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:27:10.192544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:27:10.192544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:10.192544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:10.192544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:27:10.192544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:27:10.193014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:27:10.193014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:27:10.193014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:27:10.193014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:27:10.193014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:27:10.193014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:10.193014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:10.193014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:10.193014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:27:10.193014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:10.193014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:10.193014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:10.193014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:27:10.193014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:10.193014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:27:10.193014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:27:10.193014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:10.193014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:27:10.193535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:10.193535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:27:10.193535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:27:10.193535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:10.193535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:27:10.193535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:10.193535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:27:10.218215 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:27:10.218536 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:27:10.218628 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:27:10.218775 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:27:10.219278 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:27:10.219739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:10.220822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:27:10.220822 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:10.220822 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:27:10.220822 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:27:10.220822 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:10.220822 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:27:10.220822 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:10.220822 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:27:10.222019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:10.222019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:27:10.222019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:27:10.222019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:27:10.222019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:10.222019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:27:10.222019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:10.222019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:27:10.222019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:10.222019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:27:10.222019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:10.222019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:27:10.222019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:10.222019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:27:10.222019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:27:10.222778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:10.222778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:27:10.222778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:10.222778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:27:10.222778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:27:10.222778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:10.222778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:27:10.222778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:10.222778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:27:10.223518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:10.223518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:27:10.223518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:10.223518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:27:10.223518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:27:10.223518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:27:10.223518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:27:10.223518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:27:10.223518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:27:10.223518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:10.223518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:10.224083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:10.224083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:27:10.224083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:10.224083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:10.224083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:10.224083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:27:10.224083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:10.224083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:27:10.224083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:27:10.224083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:10.224083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:27:10.224083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:10.224083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:27:10.224083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:27:10.224759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:10.224759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:27:10.224759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:10.224759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:27:10.225271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:10.225271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:27:10.225271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:27:10.225271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:27:10.225271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:27:10.225271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:27:10.225271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:10.225271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:27:10.225271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:27:10.225271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:27:10.225271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:27:10.225271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:27:10.225271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:10.225271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:10.225271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:27:10.225271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:27:10.225271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:27:10.225783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:27:10.225783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:27:10.225783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:27:10.225783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:10.225783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:10.225783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:10.225783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:27:10.225783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:10.225783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:10.225783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:10.225783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:27:10.225783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:10.225783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:27:10.225783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:27:10.225783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:10.225783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:27:10.225783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:10.225783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:27:10.225783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:27:10.226577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:10.226577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:27:10.226577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:10.226577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:27:10.226577 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:27:10.226577 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:27:10.226577 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:27:10.226577 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:27:10.227022 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:27:10.227022 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:27:10.227022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:10.227602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:27:10.227602 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:10.227602 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:27:10.227602 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:27:10.227602 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:10.227602 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:27:10.227602 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:10.227602 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:27:10.229192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:10.229192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:27:10.229192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:10.229192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:27:10.229192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:27:10.229192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:27:10.229192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:27:10.229192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:10.229192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:10.229192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:10.229192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:27:10.229192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:10.229844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:10.229844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:10.229844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:27:10.229844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:10.229844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:27:10.229844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:27:10.229844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:10.229844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:27:10.229844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:10.229844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:27:10.229844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:27:10.229844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:10.229844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:27:10.230462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:10.230462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:27:10.230930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:10.230930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:27:10.230930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:27:10.230930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:27:10.230930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:27:10.230930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:27:10.230930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:10.230930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:27:10.230930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:27:10.230930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:27:10.230930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:27:10.230930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:27:10.230930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:10.230930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:10.230930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:27:10.230930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:27:10.231737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:27:10.231737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:27:10.231737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:10.231737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:10.231737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:10.231737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:27:10.231737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:10.231737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:10.231737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:10.231737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:27:10.231737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:10.231737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:27:10.231737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:27:10.231737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:10.231737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:27:10.231737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:10.231737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:27:10.231737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:27:10.231737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:10.232662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:27:10.232662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:10.232662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:27:14.970475 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:27:14.971328 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:27:14.971328 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:27:14.971977 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:27:15.004092 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:27:15.004340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:15.005195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:27:15.005195 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:15.005195 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:27:15.005195 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:27:15.005195 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:15.005195 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:27:15.005195 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:15.005195 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:27:15.005975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:15.005975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:27:15.005975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:15.005975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:27:15.005975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:27:15.005975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:27:15.005975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:27:15.005975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:15.005975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:15.005975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:15.005975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:27:15.005975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:15.006637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:15.006637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:15.006637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:27:15.006637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:15.006637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:27:15.006637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:27:15.006637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:15.006637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:27:15.006637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:15.006637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:27:15.006637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:27:15.006637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:15.006637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:27:15.007316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:15.007316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:27:15.007316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:15.007316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:27:15.007316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:27:15.007316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:27:15.007316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:27:15.007316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:27:15.007316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:15.007316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:27:15.007316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:27:15.007316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:27:15.007316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:27:15.007316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:27:15.007316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:15.008163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:15.008163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:27:15.008163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:27:15.008163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:27:15.008163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:27:15.008163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:15.008163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:15.008163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:15.008163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:27:15.008163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:15.008163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:15.008163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:15.008163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:27:15.008163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:15.008163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:27:15.008163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:27:15.008163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:15.008163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:27:15.008163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:15.009104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:27:15.009104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:27:15.009104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:15.009104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:27:15.009104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:15.009104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:27:15.181188 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:27:15.181938 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:27:15.182101 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:27:15.183061 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:27:15.183388 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:27:15.183660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:15.184581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:27:15.184581 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:15.184581 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:27:15.184581 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:27:15.184581 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:15.184581 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:27:15.184581 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:15.184581 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:27:15.185884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:15.185884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:27:15.185884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:15.185884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:27:15.185884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:27:15.185884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:27:15.185884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:27:15.185884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:15.185884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:15.185884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:15.185884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:27:15.185884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:15.186527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:15.186527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:15.186527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:27:15.186527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:15.186527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:27:15.186527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:27:15.186527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:15.186527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:27:15.186527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:15.186527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:27:15.186527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:27:15.186527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:15.186527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:27:15.187177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:15.187177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:27:15.187606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:15.187606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:27:15.187606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:27:15.187606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:27:15.187606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:27:15.187606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:27:15.187606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:15.187606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:27:15.187606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:27:15.187606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:27:15.187606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:27:15.187606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:27:15.187606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:15.187606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:15.187606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:27:15.187606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:27:15.188391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:27:15.188391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:27:15.188391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:15.188391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:15.188391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:15.188391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:27:15.188391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:15.188391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:15.188391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:15.188391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:27:15.188391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:15.188391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:27:15.188391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:27:15.188391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:15.188391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:27:15.188391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:15.188391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:27:15.188391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:27:15.188391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:15.189368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:27:15.189368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:15.189368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:27:15.189368 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:27:15.189368 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:27:15.189673 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:27:15.189742 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:27:15.190366 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:27:15.190702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:15.191333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:27:15.191333 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:15.191333 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:27:15.191333 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:27:15.191333 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:15.191333 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:27:15.191333 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:15.191333 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:27:15.191909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:15.191909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:27:15.191909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:27:15.191909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:27:15.191909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:15.191909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:27:15.191909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:15.191909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:27:15.191909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:15.191909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:27:15.191909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:15.191909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:27:15.191909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:15.191909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:27:15.191909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:27:15.192374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:15.192374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:27:15.192374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:15.192374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:27:15.192374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:27:15.192374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:15.192374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:27:15.192374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:15.192374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:27:15.192733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:15.192733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:27:15.192733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:15.192733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:27:15.192733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:27:15.192733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:27:15.192733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:27:15.192733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:27:15.192733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:27:15.192733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:27:15.192733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:15.192733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:15.193088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:15.193088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:27:15.193088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:15.193088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:15.193088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:15.193088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:27:15.193088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:15.193088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:27:15.193088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:27:15.193088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:15.193088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:27:15.193088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:15.193088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:27:15.193088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:27:15.193487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:15.193487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:27:15.193487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:15.193487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:27:15.193487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:15.193487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:27:15.193487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:27:15.193487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:27:15.193487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:27:15.193487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:27:15.193487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:15.193487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:27:15.193487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:27:15.193487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:27:15.193871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:27:15.193871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:27:15.193871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:15.193871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:15.193871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:27:15.193871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:27:15.193871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:27:15.193871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:27:15.193871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:27:15.193871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:27:15.193871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:27:15.193871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:15.193871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:15.193871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:15.193871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:27:15.193871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:15.193871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:15.193871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:15.193871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:27:15.194395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:15.194395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:27:15.194395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:27:15.194395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:15.194395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:27:15.194395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:15.194395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:27:15.194395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:27:15.194395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:15.194395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:27:15.194395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:15.194395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:27:15.218754 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:27:15.219112 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:27:15.219286 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:27:15.219539 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:27:15.220187 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:27:15.220660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:15.221566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:27:15.221566 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:15.221566 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:27:15.221566 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:27:15.221566 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:15.221566 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:27:15.221566 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:15.221566 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:27:15.222839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:15.222839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:27:15.222839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:27:15.222839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:27:15.222839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:15.222839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:27:15.222839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:15.222839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:27:15.222839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:15.222839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:27:15.222839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:15.222839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:27:15.222839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:15.222839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:27:15.222839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:27:15.223535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:15.223535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:27:15.223535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:15.223535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:27:15.223535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:27:15.223535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:15.223535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:27:15.223535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:15.223535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:27:15.225147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:15.225147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:27:15.225147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:15.225147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:27:15.225147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:27:15.225147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:27:15.225147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:27:15.225147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:27:15.225147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:27:15.225147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:15.225147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:15.225749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:15.225749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:27:15.225749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:15.225749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:15.225749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:15.225749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:27:15.225749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:15.225749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:27:15.225749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:27:15.225749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:15.225749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:27:15.225749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:15.225749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:27:15.225749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:27:15.226512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:15.226512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:27:15.226512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:15.226512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:27:15.226512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:15.226512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:27:15.226512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:27:15.226512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:27:15.226512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:27:15.226512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:27:15.226512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:15.226512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:27:15.226512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:27:15.226512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:27:15.227165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:27:15.227165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:27:15.227165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:15.227165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:15.227165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:27:15.227165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:27:15.227165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:27:15.227165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:27:15.227165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:27:15.227165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:27:15.227165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:15.227165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:15.227165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:15.227165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:27:15.227165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:15.227165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:15.227165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:15.227165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:27:15.227165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:15.228067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:27:15.228067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:27:15.228067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:15.228067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:27:15.228067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:15.228067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:27:15.228067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:27:15.228067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:15.228067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:27:15.228067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:15.228067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:27:15.228067 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:27:15.228067 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:27:15.228855 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:27:15.228855 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:27:15.228855 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:27:15.228855 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:27:15.228855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:15.228855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:27:15.228855 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:15.228855 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:27:15.228855 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:27:15.229220 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:15.229220 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:27:15.229220 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:15.229220 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:27:15.229364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:15.229364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:27:15.229364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:15.229364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:27:15.229364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:27:15.229364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:27:15.229364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:27:15.229364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:15.229364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:15.229364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:15.229364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:27:15.229364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:15.229623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:15.229623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:15.229623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:27:15.229623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:15.229623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:27:15.229623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:27:15.229623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:15.229623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:27:15.229623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:15.229623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:27:15.229623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:27:15.229623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:15.229623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:27:15.229862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:15.229862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:27:15.230392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:15.230392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:27:15.230392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:27:15.230392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:27:15.230392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:27:15.230392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:27:15.230392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:15.230392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:27:15.230392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:27:15.230392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:27:15.230392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:27:15.230392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:27:15.230392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:15.230392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:15.230392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:27:15.230392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:27:15.230723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:27:15.230723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:27:15.230723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:15.230723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:15.230723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:15.230723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:27:15.230723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:15.230723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:15.230723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:15.230723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:27:15.230723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:15.230723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:27:15.230723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:27:15.230723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:15.230723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:27:15.230723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:15.230723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:27:15.230723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:27:15.230723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:15.231083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:27:15.231083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:15.231083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:27:19.971215 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:27:19.972258 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:27:19.972258 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:27:19.973517 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:27:20.006040 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:27:20.006624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:20.007606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:27:20.007606 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:20.007606 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:27:20.007606 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:27:20.007606 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:20.007606 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:27:20.007606 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:20.007606 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:27:20.008399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:20.008399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:27:20.008399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:20.008399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:27:20.008399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:27:20.008399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:27:20.008399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:27:20.008399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:20.008399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:20.008399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:20.008399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:27:20.008399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:20.009233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:20.009233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:20.009233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:27:20.009233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:20.009233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:27:20.009233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:27:20.009233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:20.009233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:27:20.009233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:20.009233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:27:20.009233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:27:20.009233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:20.009233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:27:20.010112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:20.010112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:27:20.010112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:20.010112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:27:20.010112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:27:20.010112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:27:20.010112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:27:20.010112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:27:20.010112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:20.010112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:27:20.010112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:27:20.010112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:27:20.010112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:27:20.010112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:27:20.010112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:20.011009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:20.011009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:27:20.011009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:27:20.011009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:27:20.011009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:27:20.011009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:20.011009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:20.011009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:20.011009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:27:20.011009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:20.011009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:20.011009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:20.011009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:27:20.011009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:20.011009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:27:20.011009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:27:20.011009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:20.011009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:27:20.011009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:20.012060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:27:20.012060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:27:20.012060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:20.012060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:27:20.012060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:20.012060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:27:20.182099 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:27:20.182507 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:27:20.182633 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:27:20.183649 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:27:20.183996 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:27:20.184251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:20.185114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:27:20.185114 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:20.185114 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:27:20.185114 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:27:20.185114 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:20.185114 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:27:20.185114 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:20.185114 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:27:20.186131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:20.186131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:27:20.186131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:20.186131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:27:20.186131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:27:20.186131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:27:20.186131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:27:20.186131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:20.186131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:20.186131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:20.186131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:27:20.186131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:20.186811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:20.186811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:20.186811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:27:20.186811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:20.186811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:27:20.186811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:27:20.186811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:20.186811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:27:20.186811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:20.186811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:27:20.186811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:27:20.186811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:20.186811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:27:20.187368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:20.187368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:27:20.187368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:20.187368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:27:20.187368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:27:20.187368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:27:20.187368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:27:20.187368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:27:20.187368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:20.187368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:27:20.187368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:27:20.187368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:27:20.187368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:27:20.187368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:27:20.187368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:20.188118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:20.188118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:27:20.188118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:27:20.188118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:27:20.188118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:27:20.188118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:20.188118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:20.188118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:20.188118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:27:20.188118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:20.188118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:20.188118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:20.188118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:27:20.188118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:20.188118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:27:20.188118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:27:20.188118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:20.188118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:27:20.188118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:20.189009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:27:20.189009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:27:20.189009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:20.189009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:27:20.189009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:20.189009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:27:20.189313 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:27:20.189510 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:27:20.189604 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:27:20.189717 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:27:20.190178 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:27:20.190538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:20.191205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:27:20.191205 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:20.191205 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:27:20.191205 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:27:20.191205 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:20.191205 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:27:20.191205 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:20.191205 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:27:20.191997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:20.191997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:27:20.191997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:27:20.191997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:27:20.191997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:20.191997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:27:20.191997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:20.191997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:27:20.191997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:20.191997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:27:20.191997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:20.191997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:27:20.191997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:20.191997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:27:20.191997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:27:20.192493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:20.192493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:27:20.192493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:20.192493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:27:20.192493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:27:20.192493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:20.192493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:27:20.192493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:20.192493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:27:20.192838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:20.192838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:27:20.192838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:20.192838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:27:20.192838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:27:20.192838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:27:20.192838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:27:20.192838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:27:20.192838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:27:20.192838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:27:20.192838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:20.192838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:20.193191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:20.193191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:27:20.193191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:20.193191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:20.193191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:20.193191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:27:20.193191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:20.193191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:27:20.193191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:27:20.193191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:20.193191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:27:20.193191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:20.193191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:27:20.193191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:27:20.193562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:20.193562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:27:20.193562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:20.193562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:27:20.193881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:20.193881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:27:20.193881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:27:20.193881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:27:20.193881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:27:20.193881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:27:20.193881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:20.193881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:27:20.193881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:27:20.193881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:27:20.193881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:27:20.193881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:27:20.193881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:20.193881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:20.193881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:27:20.193881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:27:20.194327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:27:20.194327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:27:20.194327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:27:20.194327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:27:20.194327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:27:20.194327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:20.194327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:20.194327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:20.194327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:27:20.194327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:20.194327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:20.194327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:20.194327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:27:20.194327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:20.194327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:27:20.194327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:27:20.194327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:20.194327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:27:20.194812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:20.194812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:27:20.194812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:27:20.194812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:20.194812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:27:20.194812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:20.194812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:27:20.219283 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:27:20.219607 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:27:20.219734 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:27:20.219888 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:27:20.220334 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:27:20.220707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:20.221495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:27:20.221495 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:20.221495 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:27:20.221495 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:27:20.221495 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:20.221495 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:27:20.221495 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:20.221495 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:27:20.222541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:20.222541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:27:20.222541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:27:20.222541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:27:20.222541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:20.222541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:27:20.222541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:20.222541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:27:20.222541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:20.222541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:27:20.222541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:20.222541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:27:20.222541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:20.222541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:27:20.222541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:27:20.223177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:20.223177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:27:20.223177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:20.223177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:27:20.223177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:27:20.223177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:20.223177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:27:20.223177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:20.223177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:27:20.223720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:20.223720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:27:20.223720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:20.223720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:27:20.223720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:27:20.223720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:27:20.223720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:27:20.223720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:27:20.223720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:27:20.223720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:20.223720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:20.224228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:20.224228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:27:20.224228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:20.224228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:20.224228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:20.224228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:27:20.224228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:20.224228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:27:20.224228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:27:20.224228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:20.224228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:27:20.224228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:20.224228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:27:20.224228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:27:20.224820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:20.224820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:27:20.224820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:20.224820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:27:20.225272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:20.225272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:27:20.225272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:27:20.225272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:27:20.225272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:27:20.225272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:27:20.225272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:20.225272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:27:20.225272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:27:20.225272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:27:20.225272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:27:20.225272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:27:20.225272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:20.225272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:20.225272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:27:20.225272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:27:20.225272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:27:20.226610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:27:20.226610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:27:20.226610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:27:20.226610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:20.226610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:20.226610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:20.226610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:27:20.226610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:20.226610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:20.226610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:20.226610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:27:20.226610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:20.226610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:27:20.226610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:27:20.226610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:20.226610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:27:20.226610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:20.226610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:27:20.226610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:27:20.227718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:20.227718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:27:20.227718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:20.227718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:27:20.227718 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:27:20.227718 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:27:20.227718 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:27:20.228339 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:27:20.228339 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:27:20.228634 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:27:20.228859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:20.229569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:27:20.229569 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:20.229569 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:27:20.229569 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:27:20.229569 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:20.229569 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:27:20.229569 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:20.229569 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:27:20.230481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:20.230481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:27:20.230481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:20.230481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:27:20.230481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:27:20.230481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:27:20.230481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:27:20.230481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:20.230481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:20.230481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:20.230481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:27:20.230481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:20.231063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:20.231063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:20.231063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:27:20.231063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:20.231063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:27:20.231063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:27:20.231063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:20.231063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:27:20.231063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:20.231063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:27:20.231063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:27:20.231063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:20.231063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:27:20.231499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:20.231499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:27:20.231499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:20.231499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:27:20.231499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:27:20.231499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:27:20.231499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:27:20.231499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:27:20.231499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:20.231499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:27:20.231499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:27:20.231499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:27:20.231499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:27:20.231499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:27:20.231499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:20.231921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:20.231921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:27:20.231921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:27:20.231921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:27:20.231921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:27:20.231921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:20.231921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:20.231921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:20.231921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:27:20.231921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:20.231921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:20.231921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:20.231921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:27:20.231921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:20.231921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:27:20.231921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:27:20.231921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:20.231921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:27:20.231921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:20.232393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:27:20.232393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:27:20.232393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:20.232393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:27:20.232393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:20.232393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:27:24.972307 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:27:24.973047 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:27:24.973244 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:27:24.974611 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:27:24.976803 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG dbcounter [-] [105581] Writing DB stats neutron:SELECT=240 {{(pid=105581) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 14:27:25.007417 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:27:25.007818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:25.008716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:27:25.008716 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:25.008716 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:27:25.008716 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:27:25.008716 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:25.008716 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:27:25.008716 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:25.008716 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:27:25.009690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:25.009690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:27:25.009690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:25.009690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:27:25.009690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:27:25.009690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:27:25.009690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:27:25.009690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:25.009690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:25.009690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:25.009690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:27:25.009690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:25.009956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:25.009956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:25.009956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:27:25.009956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:25.009956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:27:25.009956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:27:25.009956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:25.009956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:27:25.009956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:25.009956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:27:25.009956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:27:25.009956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:25.009956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:27:25.010194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:25.010194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:27:25.010745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:25.010745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:27:25.010745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:27:25.010745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:27:25.010745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:27:25.010745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:27:25.010745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:25.010745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:27:25.010745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:27:25.010745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:27:25.010745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:27:25.010745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:27:25.010745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:25.010745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:25.010745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:27:25.010745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:27:25.011062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:27:25.011062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:27:25.011062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:25.011062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:25.011062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:25.011062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:27:25.011062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:25.011062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:25.011062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:25.011062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:27:25.011062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:25.011062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:27:25.011062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:27:25.011062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:25.011062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:27:25.011062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:25.011062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:27:25.011062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:27:25.011062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:25.011437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:27:25.011437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:25.011437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:27:25.182771 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:27:25.183421 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:27:25.183596 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:27:25.184365 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:27:25.184855 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:27:25.185122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:25.185975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:27:25.185975 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:25.185975 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:27:25.185975 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:27:25.185975 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:25.185975 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:27:25.185975 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:25.185975 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:27:25.186854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:25.186854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:27:25.186854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:25.186854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:27:25.186854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:27:25.186854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:27:25.186854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:27:25.186854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:25.186854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:25.186854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:25.186854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:27:25.186854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:25.187401 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:25.187401 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:25.187401 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:27:25.187401 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:25.187401 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:27:25.187401 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:27:25.187401 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:25.187401 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:27:25.187401 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:25.187401 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:27:25.187401 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:27:25.187401 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:25.187401 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:27:25.188022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:25.188022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:27:25.188022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:25.188022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:27:25.188022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:27:25.188022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:27:25.188022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:27:25.188022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:27:25.188022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:25.188022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:27:25.188022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:27:25.188022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:27:25.188022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:27:25.188022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:27:25.188022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:25.188763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:25.188763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:27:25.188763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:27:25.188763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:27:25.188763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:27:25.188763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:25.188763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:25.188763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:25.188763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:27:25.188763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:25.188763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:25.188763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:25.188763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:27:25.188763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:25.188763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:27:25.188763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:27:25.188763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:25.188763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:27:25.188763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:25.189874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:27:25.189874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:27:25.189874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:25.189874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:27:25.189874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:25.189874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:27:25.189874 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:27:25.190233 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:27:25.190445 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:27:25.190640 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:27:25.190992 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:27:25.191188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:25.191679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:27:25.191679 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:25.191679 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:27:25.191679 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:27:25.191679 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:25.191679 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:27:25.191679 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:25.191679 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:27:25.192423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:25.192423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:27:25.192423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:27:25.192423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:27:25.192423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:25.192423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:27:25.192423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:25.192423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:27:25.192423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:25.192423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:27:25.192423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:25.192423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:27:25.192423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:25.192423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:27:25.192423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:27:25.193125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:25.193125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:27:25.193125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:25.193125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:27:25.193125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:27:25.193125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:25.193125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:27:25.193125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:25.193125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:27:25.193507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:25.193507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:27:25.193507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:25.193507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:27:25.193507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:27:25.193507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:27:25.193507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:27:25.193507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:27:25.193507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:27:25.193507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:27:25.193507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:25.193507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:25.193831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:25.193831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:27:25.193831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:25.193831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:25.193831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:25.193831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:27:25.193831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:25.193831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:27:25.193831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:27:25.193831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:25.193831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:27:25.193831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:25.193831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:27:25.193831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:27:25.194344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:25.194344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:27:25.194344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:25.194344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:27:25.194344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:25.194344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:27:25.194344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:27:25.194344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:27:25.194344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:27:25.194344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:27:25.194344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:25.194344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:27:25.194344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:27:25.194344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:27:25.194837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:27:25.194837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:27:25.194837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:25.194837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:25.194837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:27:25.194837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:27:25.194837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:27:25.194837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:27:25.194837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:27:25.194837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:27:25.194837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:27:25.194837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:25.194837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:25.194837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:25.194837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:27:25.194837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:25.194837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:25.194837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:25.194837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:27:25.195333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:25.195333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:27:25.195333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:27:25.195333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:25.195333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:27:25.195333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:25.195333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:27:25.195333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:27:25.195333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:25.195333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:27:25.195333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:25.195333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:27:25.219771 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:27:25.220144 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:27:25.220475 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:27:25.220539 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:27:25.221035 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:27:25.221385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:25.222181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:27:25.222181 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:25.222181 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:27:25.222181 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:27:25.222181 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:25.222181 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:27:25.222181 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:25.222181 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:27:25.223377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:25.223377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:27:25.223377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:27:25.223377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:27:25.223377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:25.223377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:27:25.223377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:25.223377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:27:25.223377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:25.223377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:27:25.223377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:25.223377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:27:25.223377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:25.223377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:27:25.223377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:27:25.223819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:25.223819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:27:25.223819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:25.223819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:27:25.223819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:27:25.223819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:25.223819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:27:25.223819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:25.223819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:27:25.224779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:25.224779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:27:25.224779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:25.224779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:27:25.224779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:27:25.224779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:27:25.224779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:27:25.224779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:27:25.224779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:27:25.224779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:25.224779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:25.225307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:25.225307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:27:25.225307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:25.225307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:25.225307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:25.225307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:27:25.225307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:25.225307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:27:25.225307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:27:25.225307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:25.225307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:27:25.225307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:25.225307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:27:25.225307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:27:25.226127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:25.226127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:27:25.226127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:25.226127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:27:25.226332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:25.226332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:27:25.226332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:27:25.226332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:27:25.226332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:27:25.226332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:27:25.226332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:25.226332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:27:25.226332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:27:25.226332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:27:25.226332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:27:25.226332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:27:25.226332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:25.226332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:25.226332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:27:25.226332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:27:25.226332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:27:25.226914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:27:25.226914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:27:25.226914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:27:25.226914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:25.226914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:25.226914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:25.226914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:27:25.226914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:25.226914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:25.226914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:25.226914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:27:25.226914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:25.226914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:27:25.226914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:27:25.226914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:25.226914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:27:25.226914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:25.226914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:27:25.226914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:27:25.227416 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:25.227416 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:27:25.227416 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:25.227416 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:27:25.227416 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:27:25.227862 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:27:25.227981 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:27:25.228170 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:27:25.228477 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:27:25.228739 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:27:25.228885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:25.229359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:27:25.229359 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:25.229359 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:27:25.229359 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:27:25.229359 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:25.229359 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:27:25.229359 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:25.229359 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:27:25.230232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:25.230232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:27:25.230232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:25.230232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:27:25.230232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:27:25.230232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:27:25.230232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:27:25.230232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:25.230232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:25.230232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:25.230232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:27:25.230232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:25.230876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:25.230876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:25.230876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:27:25.230876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:25.230876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:27:25.230876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:27:25.230876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:25.230876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:27:25.230876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:25.230876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:27:25.230876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:27:25.230876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:25.230876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:27:25.231324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:25.231324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:27:25.231324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:25.231324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:27:25.231324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:27:25.231324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:27:25.231324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:27:25.231324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:27:25.231324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:25.231324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:27:25.231324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:27:25.231324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:27:25.231324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:27:25.231324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:27:25.231324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:25.231808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:25.231808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:27:25.231808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:27:25.231808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:27:25.231808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:27:25.231808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:25.231808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:25.231808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:25.231808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:27:25.231808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:25.231808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:25.231808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:25.231808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:27:25.231808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:25.231808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:27:25.231808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:27:25.231808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:25.231808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:27:25.231808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:25.232314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:27:25.232314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:27:25.232314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:25.232314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:27:25.232314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:25.232314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:27:29.973260 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:27:29.974053 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:27:29.974204 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:27:29.975694 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:27:30.007777 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:27:30.008141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:30.008935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:27:30.008935 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:30.008935 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:27:30.008935 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:27:30.008935 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:30.008935 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:27:30.008935 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:30.008935 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:27:30.009690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:30.009690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:27:30.009690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:30.009690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:27:30.009690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:27:30.009690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:27:30.009690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:27:30.009690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:30.009690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:30.009690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:30.009690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:27:30.009690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:30.010280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:30.010280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:30.010280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:27:30.010280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:30.010280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:27:30.010280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:27:30.010280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:30.010280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:27:30.010280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:30.010280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:27:30.010280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:27:30.010280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:30.010280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:27:30.010931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:30.010931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:27:30.010931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:30.010931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:27:30.010931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:27:30.010931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:27:30.010931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:27:30.010931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:27:30.010931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:30.010931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:27:30.010931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:27:30.010931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:27:30.010931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:27:30.010931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:27:30.010931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:30.011576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:30.011576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:27:30.011576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:27:30.011576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:27:30.011576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:27:30.011576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:30.011576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:30.011576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:30.011576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:27:30.011576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:30.011576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:30.011576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:30.011576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:27:30.011576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:30.011576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:27:30.011576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:27:30.011576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:30.011576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:27:30.011576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:30.012393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:27:30.012393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:27:30.012393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:30.012393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:27:30.012393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:30.012393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:27:30.185190 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:27:30.185190 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:27:30.185190 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:27:30.187125 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:27:30.187125 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:27:30.187125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:30.188394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:27:30.188394 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:30.188394 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:27:30.188394 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:27:30.188394 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:30.188394 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:27:30.188394 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:30.188394 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:27:30.189704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:30.189704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:27:30.189704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:30.189704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:27:30.189704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:27:30.189704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:27:30.189704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:27:30.189704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:30.189704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:30.189704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:30.189704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:27:30.189704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:30.190271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:30.190271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:30.190271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:27:30.190271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:30.190271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:27:30.190271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:27:30.190271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:30.190271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:27:30.190271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:30.190271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:27:30.190271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:27:30.190271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:30.190271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:27:30.191048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:30.191048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:27:30.191048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:30.191048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:27:30.191048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:27:30.191048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:27:30.191048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:27:30.191048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:27:30.191048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:30.191048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:27:30.191048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:27:30.191048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:27:30.191048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:27:30.191048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:27:30.191048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:30.191774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:30.191774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:27:30.191774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:27:30.191774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:27:30.191774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:27:30.191774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:30.191774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:30.191774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:30.191774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:27:30.191774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:30.191774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:30.191774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:30.191774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:27:30.191774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:30.191774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:27:30.191774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:27:30.191774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:30.191774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:27:30.191774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:30.193624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:27:30.193624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:27:30.193624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:30.193624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:27:30.193624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:30.193624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:27:30.193624 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:27:30.193624 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:27:30.193624 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:27:30.193624 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:27:30.194051 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:27:30.194051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:30.194051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:27:30.194051 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:30.194051 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:27:30.194051 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:27:30.194051 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:30.194051 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:27:30.194051 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:30.194051 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:27:30.194051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:30.194051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:27:30.194051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:27:30.194051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:27:30.194684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:30.194684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:27:30.194684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:30.194684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:27:30.194684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:30.194684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:27:30.194684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:30.194684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:27:30.194684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:30.194684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:27:30.194684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:27:30.194684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:30.194684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:27:30.194684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:30.194684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:27:30.194684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:27:30.194684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:30.194684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:27:30.195483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:30.195483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:27:30.195483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:30.195483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:27:30.195483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:30.195483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:27:30.195483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:27:30.195483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:27:30.195483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:27:30.195483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:27:30.195483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:27:30.195483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:27:30.196008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:30.196008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:30.196008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:30.196008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:27:30.196008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:30.196008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:30.196008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:30.196008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:27:30.196008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:30.196008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:27:30.196008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:27:30.196008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:30.196008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:27:30.196008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:30.196584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:27:30.196584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:27:30.196584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:30.196584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:27:30.196584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:30.196584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:27:30.196584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:30.196584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:27:30.196584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:27:30.196584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:27:30.196584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:27:30.196584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:27:30.196584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:30.196584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:27:30.196861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:27:30.196861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:27:30.196861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:27:30.196861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:27:30.196861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:30.196861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:30.196861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:27:30.196861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:27:30.196861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:27:30.196861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:27:30.196861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:27:30.196861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:27:30.196861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:27:30.196861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:30.196861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:30.196861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:30.196861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:27:30.196861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:30.196861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:30.197212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:30.197212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:27:30.197212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:30.197212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:27:30.197212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:27:30.197212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:30.197212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:27:30.197212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:30.197212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:27:30.197212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:27:30.197212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:30.197212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:27:30.197212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:30.197212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:27:30.220744 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:27:30.221315 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:27:30.221536 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:27:30.222719 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:27:30.223266 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:27:30.223669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:30.224494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:27:30.224494 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:30.224494 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:27:30.224494 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:27:30.224494 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:30.224494 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:27:30.224494 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:30.224494 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:27:30.225858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:30.225858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:27:30.225858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:27:30.225858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:27:30.225858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:30.225858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:27:30.225858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:30.225858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:27:30.225858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:30.225858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:27:30.225858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:30.225858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:27:30.225858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:30.225858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:27:30.225858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:27:30.226471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:30.226471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:27:30.226471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:30.226471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:27:30.226471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:27:30.226471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:30.226471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:27:30.226471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:30.226471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:27:30.227271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:30.227271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:27:30.227271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:30.227271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:27:30.227271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:27:30.227271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:27:30.227271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:27:30.227271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:27:30.227271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:27:30.227271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:30.227271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:30.228354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:30.228354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:27:30.228354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:30.228354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:30.228354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:30.228354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:27:30.228354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:30.228354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:27:30.228354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:27:30.228354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:30.228354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:27:30.228354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:30.228354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:27:30.228354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:27:30.229841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:30.229841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:27:30.229841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:30.229841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:27:30.229841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:30.229841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:27:30.229841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:27:30.229841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:27:30.229841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:27:30.229841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:27:30.229841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:30.229841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:27:30.229841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:27:30.229841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:27:30.230269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:27:30.230269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:27:30.230269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:30.230269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:30.230269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:27:30.230269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:27:30.230269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:27:30.230269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:27:30.230269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:27:30.230269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:27:30.230269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:30.230269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:30.230269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:30.230269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:27:30.230269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:30.230269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:30.230269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:30.230269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:27:30.230269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:30.230785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:27:30.230785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:27:30.230785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:30.230785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:27:30.230785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:30.230785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:27:30.230785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:27:30.230785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:30.230785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:27:30.230785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:30.230785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:27:30.230785 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:27:30.230785 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:27:30.231176 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:27:30.231176 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:27:30.231238 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:27:30.231768 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:27:30.231998 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:30.233105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:27:30.233105 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:30.233105 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:27:30.233105 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:27:30.233105 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:30.233105 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:27:30.233105 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:30.233105 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:27:30.234062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:30.234062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:27:30.234062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:30.234062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:27:30.234062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:27:30.234062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:27:30.234062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:27:30.234062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:30.234062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:30.234062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:30.234062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:27:30.234062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:30.234462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:30.234462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:30.234462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:27:30.234462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:30.234462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:27:30.234462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:27:30.234462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:30.234462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:27:30.234462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:30.234462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:27:30.234462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:27:30.234462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:30.234462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:27:30.234783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:30.234783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:27:30.234783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:30.234783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:27:30.234783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:27:30.234783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:27:30.234783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:27:30.234783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:27:30.234783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:30.234783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:27:30.234783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:27:30.234783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:27:30.234783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:27:30.234783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:27:30.234783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:30.235157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:30.235157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:27:30.235157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:27:30.235157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:27:30.235157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:27:30.235157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:30.235157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:30.235157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:30.235157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:27:30.235157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:30.235157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:30.235157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:30.235157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:27:30.235157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:30.235157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:27:30.235157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:27:30.235157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:30.235157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:27:30.235157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:30.235637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:27:30.235637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:27:30.235637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:30.235637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:27:30.235637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:30.235637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:27:34.973988 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:27:34.974689 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:27:34.974882 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:27:34.976163 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:27:35.009157 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:27:35.009438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:35.010082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:27:35.010082 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:35.010082 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:27:35.010082 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:27:35.010082 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:35.010082 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:27:35.010082 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:35.010082 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:27:35.010903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:35.010903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:27:35.010903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:35.010903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:27:35.010903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:27:35.010903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:27:35.010903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:27:35.010903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:35.010903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:35.010903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:35.010903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:27:35.010903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:35.011590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:35.011590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:35.011590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:27:35.011590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:35.011590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:27:35.011590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:27:35.011590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:35.011590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:27:35.011590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:35.011590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:27:35.011590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:27:35.011590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:35.011590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:27:35.012202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:35.012202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:27:35.012202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:35.012202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:27:35.012202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:27:35.012202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:27:35.012202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:27:35.012202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:27:35.012202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:35.012202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:27:35.012202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:27:35.012202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:27:35.012202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:27:35.012202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:27:35.012202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:35.012942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:35.012942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:27:35.012942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:27:35.012942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:27:35.012942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:27:35.012942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:35.012942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:35.012942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:35.012942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:27:35.012942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:35.012942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:35.012942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:35.012942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:27:35.012942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:35.012942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:27:35.012942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:27:35.012942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:35.012942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:27:35.012942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:35.013832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:27:35.013832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:27:35.013832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:35.013832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:27:35.013832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:35.013832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:27:35.184980 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:27:35.184980 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:27:35.185199 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:27:35.186339 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:27:35.186760 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:27:35.187073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:35.188126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:27:35.188126 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:35.188126 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:27:35.188126 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:27:35.188126 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:35.188126 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:27:35.188126 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:35.188126 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:27:35.189695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:35.189695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:27:35.189695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:35.189695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:27:35.189695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:27:35.189695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:27:35.189695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:27:35.189695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:35.189695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:35.189695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:35.189695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:27:35.189695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:35.190324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:35.190324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:35.190324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:27:35.190324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:35.190324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:27:35.190324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:27:35.190324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:35.190324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:27:35.190324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:35.190324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:27:35.190324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:27:35.190324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:35.190324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:27:35.190949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:35.190949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:27:35.190949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:35.190949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:27:35.190949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:27:35.190949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:27:35.190949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:27:35.190949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:27:35.190949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:35.190949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:27:35.190949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:27:35.190949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:27:35.190949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:27:35.190949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:27:35.190949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:35.191714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:35.191714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:27:35.191714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:27:35.191714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:27:35.191714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:27:35.191714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:35.191714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:35.191714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:35.191714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:27:35.191714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:35.191714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:35.191714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:35.191714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:27:35.191714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:35.191714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:27:35.191714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:27:35.191714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:35.191714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:27:35.191714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:35.192635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:27:35.192635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:27:35.192635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:35.192635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:27:35.192635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:35.192635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:27:35.192635 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:27:35.192635 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:27:35.192635 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:27:35.192635 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:27:35.193138 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:27:35.193350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:35.193937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:27:35.193937 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:35.193937 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:27:35.193937 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:27:35.193937 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:35.193937 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:27:35.193937 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:35.193937 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:27:35.194689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:35.194689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:27:35.194689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:27:35.194689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:27:35.194689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:35.194689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:27:35.194689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:35.194689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:27:35.194689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:35.194689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:27:35.194689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:35.194689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:27:35.194689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:35.194689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:27:35.194689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:27:35.195156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:35.195156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:27:35.195156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:35.195156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:27:35.195156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:27:35.195156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:35.195156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:27:35.195156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:35.195156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:27:35.195660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:35.195660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:27:35.195660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:35.195660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:27:35.195660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:27:35.195660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:27:35.195660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:27:35.195660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:27:35.195660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:27:35.195660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:27:35.195660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:35.195660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:35.196317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:35.196317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:27:35.196317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:35.196317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:35.196317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:35.196317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:27:35.196317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:35.196317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:27:35.196317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:27:35.196317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:35.196317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:27:35.196317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:35.196317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:27:35.196317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:27:35.196702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:35.196702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:27:35.196702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:35.196702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:27:35.196702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:35.196702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:27:35.196702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:27:35.196702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:27:35.196702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:27:35.196702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:27:35.196702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:35.196702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:27:35.196702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:27:35.196702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:27:35.197059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:27:35.197059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:27:35.197059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:35.197059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:35.197059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:27:35.197059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:27:35.197059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:27:35.197059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:27:35.197059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:27:35.197059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:27:35.197059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:27:35.197059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:35.197059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:35.197059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:35.197059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:27:35.197059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:35.197059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:35.197059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:35.197059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:27:35.197567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:35.197567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:27:35.197567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:27:35.197567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:35.197567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:27:35.197567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:35.197567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:27:35.197567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:27:35.197567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:35.197567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:27:35.197567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:35.197567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:27:35.221601 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:27:35.222010 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:27:35.222192 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:27:35.222429 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:27:35.223043 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:27:35.223536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:35.224444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:27:35.224444 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:35.224444 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:27:35.224444 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:27:35.224444 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:35.224444 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:27:35.224444 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:35.224444 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:27:35.225547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:35.225547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:27:35.225547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:27:35.225547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:27:35.225547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:35.225547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:27:35.225547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:35.225547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:27:35.225547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:35.225547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:27:35.225547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:35.225547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:27:35.225547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:35.225547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:27:35.225547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:27:35.227242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:35.227242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:27:35.227242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:35.227242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:27:35.227242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:27:35.227242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:35.227242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:27:35.227242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:35.227242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:27:35.227242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:35.227242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:27:35.227242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:35.227242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:27:35.227242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:27:35.227984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:27:35.227984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:27:35.227984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:27:35.227984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:27:35.227984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:35.227984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:35.227984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:35.227984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:27:35.227984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:35.227984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:35.227984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:35.227984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:27:35.227984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:35.228909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:27:35.228909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:27:35.228909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:35.228909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:27:35.228909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:35.228909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:27:35.228909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:27:35.228909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:35.228909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:27:35.228909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:35.228909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:27:35.228909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:35.229371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:27:35.229371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:27:35.229371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:27:35.229371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:27:35.229371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:27:35.229371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:35.229371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:27:35.229371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:27:35.229371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:27:35.229371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:27:35.229371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:27:35.229371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:35.229371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:35.229371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:27:35.229371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:27:35.229371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:27:35.229371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:27:35.229371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:27:35.229371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:27:35.230574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:35.230574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:35.230574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:35.230574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:27:35.230574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:35.230574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:35.230574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:35.230574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:27:35.230574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:35.230574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:27:35.230574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:27:35.230574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:35.230574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:27:35.230574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:35.230574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:27:35.230574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:27:35.230574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:35.230574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:27:35.230574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:35.230574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:27:35.231379 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:27:35.231379 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:27:35.231379 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:27:35.231379 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:27:35.231379 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:27:35.231379 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:27:35.231647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:35.232582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:27:35.232582 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:35.232582 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:27:35.232582 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:27:35.232582 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:35.232582 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:27:35.232582 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:35.232582 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:27:35.232891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:35.232891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:27:35.232891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:35.232891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:27:35.232891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:27:35.232891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:27:35.232891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:27:35.232891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:35.232891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:35.232891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:35.232891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:27:35.232891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:35.233194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:35.233194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:35.233194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:27:35.233194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:35.233194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:27:35.233194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:27:35.233194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:35.233194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:27:35.233194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:35.233194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:27:35.233194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:27:35.233194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:35.233194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:27:35.233463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:35.233463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:27:35.233733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:35.233733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:27:35.233733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:27:35.233733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:27:35.233733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:27:35.233733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:27:35.233733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:35.233733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:27:35.233733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:27:35.233733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:27:35.233733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:27:35.233733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:27:35.233733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:35.233733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:35.233733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:27:35.233733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:27:35.234076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:27:35.234076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:27:35.234076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:35.234076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:35.234076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:35.234076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:27:35.234076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:35.234076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:35.234076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:35.234076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:27:35.234076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:35.234076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:27:35.234076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:27:35.234076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:35.234076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:27:35.234076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:35.234076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:27:35.234076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:27:35.234076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:35.234618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:27:35.234618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:35.234618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:27:39.974771 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:27:39.975647 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:27:39.975647 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:27:39.976134 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:27:40.008547 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:27:40.008878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:40.009594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:27:40.009594 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:40.009594 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:27:40.009594 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:27:40.009594 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:40.009594 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:27:40.009594 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:40.009594 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:27:40.010244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:40.010244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:27:40.010244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:40.010244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:27:40.010244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:27:40.010244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:27:40.010244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:27:40.010244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:40.010244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:40.010244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:40.010244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:27:40.010244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:40.010659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:40.010659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:40.010659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:27:40.010659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:40.010659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:27:40.010659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:27:40.010659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:40.010659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:27:40.010659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:40.010659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:27:40.010659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:27:40.010659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:40.010659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:27:40.011125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:40.011125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:27:40.011125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:40.011125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:27:40.011125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:27:40.011125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:27:40.011125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:27:40.011125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:27:40.011125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:40.011125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:27:40.011125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:27:40.011125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:27:40.011125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:27:40.011125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:27:40.011125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:40.011971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:40.011971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:27:40.011971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:27:40.011971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:27:40.011971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:27:40.011971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:40.011971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:40.011971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:40.011971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:27:40.011971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:40.011971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:40.011971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:40.011971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:27:40.011971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:40.011971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:27:40.011971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:27:40.011971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:40.011971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:27:40.011971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:40.013011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:27:40.013011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:27:40.013011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:40.013011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:27:40.013011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:40.013011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:27:40.185128 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:27:40.185818 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:27:40.186029 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:27:40.186929 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:27:40.187116 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:27:40.187258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:40.187962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:27:40.187962 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:40.187962 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:27:40.187962 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:27:40.187962 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:40.187962 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:27:40.187962 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:40.187962 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:27:40.189595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:40.189595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:27:40.189595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:40.189595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:27:40.189595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:27:40.189595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:27:40.189595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:27:40.189595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:40.189595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:40.189595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:40.189595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:27:40.189595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:40.190173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:40.190173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:40.190173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:27:40.190173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:40.190173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:27:40.190173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:27:40.190173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:40.190173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:27:40.190173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:40.190173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:27:40.190173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:27:40.190173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:40.190173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:27:40.190698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:40.190698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:27:40.191397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:40.191397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:27:40.191397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:27:40.191397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:27:40.191397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:27:40.191397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:27:40.191397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:40.191397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:27:40.191397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:27:40.191397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:27:40.191397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:27:40.191397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:27:40.191397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:40.191397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:40.191397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:27:40.191397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:27:40.192081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:27:40.192081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:27:40.192081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:40.192081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:40.192081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:40.192081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:27:40.192081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:40.192081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:40.192081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:40.192081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:27:40.192081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:40.192081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:27:40.192081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:27:40.192081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:40.192081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:27:40.192081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:40.192081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:27:40.192081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:27:40.192081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:40.192832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:27:40.192832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:40.192832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:27:40.192832 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:27:40.193026 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:27:40.193091 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:27:40.193269 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:27:40.193585 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:27:40.193763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:40.194285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:27:40.194285 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:40.194285 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:27:40.194285 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:27:40.194285 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:40.194285 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:27:40.194285 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:40.194285 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:27:40.195627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:40.195627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:27:40.195627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:27:40.195627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:27:40.195627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:40.195627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:27:40.195627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:40.195627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:27:40.195627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:40.195627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:27:40.195627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:40.195627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:27:40.195627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:40.195627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:27:40.195627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:27:40.196277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:40.196277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:27:40.196277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:40.196277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:27:40.196277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:27:40.196277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:40.196277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:27:40.196277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:40.196277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:27:40.196681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:40.196681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:27:40.196681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:40.196681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:27:40.196681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:27:40.196681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:27:40.196681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:27:40.196681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:27:40.196681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:27:40.196681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:27:40.196681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:40.196681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:40.197177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:40.197177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:27:40.197177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:40.197177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:40.197177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:40.197177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:27:40.197177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:40.197177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:27:40.197177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:27:40.197177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:40.197177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:27:40.197177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:40.197177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:27:40.197177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:27:40.197719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:40.197719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:27:40.197719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:40.197719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:27:40.197719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:40.197719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:27:40.197719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:27:40.197719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:27:40.197719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:27:40.197719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:27:40.197719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:40.197719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:27:40.197719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:27:40.197719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:27:40.198255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:27:40.198255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:27:40.198255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:40.198255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:40.198255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:27:40.198255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:27:40.198255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:27:40.198255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:27:40.198255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:27:40.198255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:27:40.198255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:27:40.198255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:40.198255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:40.198255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:40.198255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:27:40.198255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:40.198255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:40.198255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:40.198255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:27:40.199009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:40.199009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:27:40.199009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:27:40.199009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:40.199009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:27:40.199009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:40.199009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:27:40.199009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:27:40.199009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:40.199009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:27:40.199009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:40.199009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:27:40.222117 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:27:40.222458 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:27:40.222578 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:27:40.222719 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:27:40.223256 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:27:40.223681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:40.224485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:27:40.224485 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:40.224485 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:27:40.224485 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:27:40.224485 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:40.224485 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:27:40.224485 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:40.224485 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:27:40.225235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:40.225235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:27:40.225235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:27:40.225235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:27:40.225235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:40.225235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:27:40.225235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:40.225235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:27:40.225235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:40.225235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:27:40.225235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:40.225235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:27:40.225235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:40.225235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:27:40.225235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:27:40.225840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:40.225840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:27:40.225840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:40.225840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:27:40.225840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:27:40.225840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:40.225840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:27:40.225840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:40.225840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:27:40.226211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:40.226211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:27:40.226211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:40.226211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:27:40.226211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:27:40.226211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:27:40.226211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:27:40.226211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:27:40.226211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:27:40.226211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:40.226211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:40.226648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:40.226648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:27:40.226648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:40.226648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:40.226648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:40.226648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:27:40.226648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:40.226648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:27:40.226648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:27:40.226648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:40.226648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:27:40.226648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:40.226648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:27:40.226648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:27:40.227147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:40.227147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:27:40.227147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:40.227147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:27:40.227293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:40.227293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:27:40.227293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:27:40.227293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:27:40.227293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:27:40.227293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:27:40.227293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:40.227293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:27:40.227293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:27:40.227293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:27:40.227293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:27:40.227293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:27:40.227293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:40.227293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:40.227293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:27:40.227293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:27:40.227293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:27:40.228016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:27:40.228016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:27:40.228016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:27:40.228016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:40.228016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:40.228016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:40.228016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:27:40.228016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:40.228016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:40.228016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:40.228016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:27:40.228016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:40.228016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:27:40.228016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:27:40.228016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:40.228016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:27:40.228016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:40.228016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:27:40.228016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:27:40.228802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:40.228802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:27:40.228802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:40.228802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:27:40.230641 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:27:40.230905 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:27:40.231006 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:27:40.231209 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:27:40.231693 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:27:40.232148 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:27:40.232391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:40.233166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:27:40.233166 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:40.233166 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:27:40.233166 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:27:40.233166 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:40.233166 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:27:40.233166 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:40.233166 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:27:40.233973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:40.233973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:27:40.233973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:40.233973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:27:40.233973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:27:40.233973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:27:40.233973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:27:40.233973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:40.233973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:40.233973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:40.233973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:27:40.233973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:40.234365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:40.234365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:40.234365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:27:40.234365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:40.234365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:27:40.234365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:27:40.234365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:40.234365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:27:40.234365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:40.234365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:27:40.234365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:27:40.234365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:40.234365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:27:40.234794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:40.234794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:27:40.235036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:40.235036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:27:40.235036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:27:40.235036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:27:40.235036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:27:40.235036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:27:40.235036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:40.235036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:27:40.235036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:27:40.235036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:27:40.235036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:27:40.235036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:27:40.235036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:40.235036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:40.235036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:27:40.235036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:27:40.235846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:27:40.235846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:27:40.235846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:40.235846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:40.235846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:40.235846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:27:40.235846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:40.235846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:40.235846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:40.235846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:27:40.235846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:40.235846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:27:40.235846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:27:40.235846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:40.235846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:27:40.235846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:40.235846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:27:40.235846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:27:40.235846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:40.236618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:27:40.236618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:40.236618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:27:44.975369 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:27:44.976520 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:27:44.976520 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:27:44.977761 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:27:45.015433 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:27:45.015669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:45.016439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:27:45.016439 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:45.016439 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:27:45.016439 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:27:45.016439 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:45.016439 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:27:45.016439 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:45.016439 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:27:45.017926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:45.017926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:27:45.017926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:45.017926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:27:45.017926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:27:45.017926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:27:45.017926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:27:45.017926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:45.017926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:45.017926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:45.017926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:27:45.017926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:45.018464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:45.018464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:45.018464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:27:45.018464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:45.018464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:27:45.018464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:27:45.018464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:45.018464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:27:45.018464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:45.018464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:27:45.018464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:27:45.018464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:45.018464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:27:45.018958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:45.018958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:27:45.019701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:45.019701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:27:45.019701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:27:45.019701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:27:45.019701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:27:45.019701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:27:45.019701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:45.019701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:27:45.019701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:27:45.019701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:27:45.019701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:27:45.019701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:27:45.019701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:45.019701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:45.019701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:27:45.019701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:27:45.020336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:27:45.020336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:27:45.020336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:45.020336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:45.020336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:45.020336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:27:45.020336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:45.020336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:45.020336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:45.020336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:27:45.020336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:45.020336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:27:45.020336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:27:45.020336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:45.020336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:27:45.020336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:45.020336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:27:45.020336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:27:45.020336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:45.020975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:27:45.020975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:45.020975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:27:45.185845 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:27:45.186541 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:27:45.186905 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:27:45.187741 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:27:45.188334 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:27:45.188334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:45.189371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:27:45.189371 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:45.189371 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:27:45.189371 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:27:45.189371 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:45.189371 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:27:45.189371 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:45.189371 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:27:45.191091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:45.191091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:27:45.191091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:45.191091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:27:45.191091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:27:45.191091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:27:45.191091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:27:45.191091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:45.191091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:45.191091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:45.191091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:27:45.191091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:45.191516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:45.191516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:45.191516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:27:45.191516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:45.191516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:27:45.191516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:27:45.191516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:45.191516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:27:45.191516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:45.191516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:27:45.191516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:27:45.191516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:45.191516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:27:45.191878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:45.191878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:27:45.193420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:45.193420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:27:45.193420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:27:45.193420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:27:45.193420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:27:45.193420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:27:45.193420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:45.193420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:27:45.193420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:27:45.193420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:27:45.193420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:27:45.193420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:27:45.193420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:45.193420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:45.193420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:27:45.193420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:27:45.193878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:27:45.193878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:27:45.193878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:45.193878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:45.193878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:45.193878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:27:45.193878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:45.193878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:45.193878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:45.193878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:27:45.193878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:45.193878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:27:45.193878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:27:45.193878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:45.193878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:27:45.193878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:45.193878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:27:45.193878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:27:45.193878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:45.194361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:27:45.194361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:45.194361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:27:45.194361 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:27:45.194361 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:27:45.194361 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:27:45.194361 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:27:45.194666 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:27:45.194891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:45.195293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:27:45.195293 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:45.195293 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:27:45.195293 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:27:45.195293 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:45.195293 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:27:45.195293 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:45.195293 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:27:45.196030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:45.196030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:27:45.196030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:27:45.196030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:27:45.196030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:45.196030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:27:45.196030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:45.196030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:27:45.196030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:45.196030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:27:45.196030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:45.196030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:27:45.196030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:45.196030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:27:45.196030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:27:45.196879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:45.196879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:27:45.196879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:45.196879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:27:45.196879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:27:45.196879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:45.196879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:27:45.196879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:45.196879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:27:45.196879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:45.196879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:27:45.196879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:45.196879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:27:45.196879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:27:45.197300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:27:45.197300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:27:45.197300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:27:45.197300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:27:45.197300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:27:45.197300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:45.197300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:45.197300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:45.197300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:27:45.197300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:45.197300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:45.197300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:45.197300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:27:45.197679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:45.197679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:27:45.197679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:27:45.197679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:45.197679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:27:45.197679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:45.197679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:27:45.197679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:27:45.197679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:45.197679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:27:45.197679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:45.197679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:27:45.197983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:45.197983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:27:45.197983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:27:45.197983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:27:45.197983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:27:45.197983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:27:45.197983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:45.197983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:27:45.197983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:27:45.197983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:27:45.197983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:27:45.197983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:27:45.197983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:45.197983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:45.197983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:27:45.197983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:27:45.198396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:27:45.198396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:27:45.198396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:27:45.198396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:27:45.198396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:27:45.198396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:45.198396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:45.198396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:45.198396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:27:45.198396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:45.198396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:45.198396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:45.198396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:27:45.198396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:45.198396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:27:45.198396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:27:45.198396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:45.198396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:27:45.198894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:45.198894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:27:45.198894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:27:45.198894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:45.198894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:27:45.198894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:45.198894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:27:45.222566 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:27:45.223097 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:27:45.223291 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:27:45.223545 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:27:45.224567 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:27:45.224996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:45.226034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:27:45.226034 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:45.226034 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:27:45.226034 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:27:45.226034 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:45.226034 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:27:45.226034 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:45.226034 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:27:45.227357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:45.227357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:27:45.227357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:27:45.227357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:27:45.227357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:45.227357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:27:45.227357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:45.227357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:27:45.227357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:45.227357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:27:45.227357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:45.227357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:27:45.227357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:45.227357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:27:45.227357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:27:45.228270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:45.228270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:27:45.228270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:45.228270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:27:45.228270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:27:45.228270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:45.228270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:27:45.228270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:45.228270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:27:45.228883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:45.228883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:27:45.228883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:45.228883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:27:45.228883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:27:45.228883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:27:45.228883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:27:45.228883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:27:45.228883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:27:45.228883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:45.228883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:45.229530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:45.229530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:27:45.229530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:45.229530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:45.229530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:45.229530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:27:45.229530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:45.229530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:27:45.229530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:27:45.229530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:45.229530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:27:45.229530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:45.229530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:27:45.229530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:27:45.230140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:45.230140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:27:45.230140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:45.230140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:27:45.230865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:45.230865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:27:45.230865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:27:45.230865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:27:45.230865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:27:45.230865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:27:45.230865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:45.230865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:27:45.230865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:27:45.230865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:27:45.230865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:27:45.230865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:27:45.230865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:45.230865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:45.230865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:27:45.230865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:27:45.230865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:27:45.231341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:27:45.231341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:27:45.231341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:27:45.231341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:45.231341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:45.231341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:45.231341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:27:45.231341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:45.231341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:45.231341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:45.231341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:27:45.231341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:45.231341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:27:45.231341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:27:45.231341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:45.231341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:27:45.231341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:45.231341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:27:45.231341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:27:45.231868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:45.231868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:27:45.231868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:45.231868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:27:45.231868 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:27:45.231868 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:27:45.231868 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:27:45.232117 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:27:45.232564 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:27:45.232717 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:27:45.232826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:45.233583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:27:45.233583 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:45.233583 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:27:45.233583 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:27:45.233583 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:45.233583 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:27:45.233583 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:45.233583 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:27:45.234939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:45.234939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:27:45.234939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:45.234939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:27:45.234939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:27:45.234939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:27:45.234939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:27:45.234939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:45.234939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:45.234939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:45.234939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:27:45.234939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:45.235300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:45.235300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:45.235300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:27:45.235300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:45.235300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:27:45.235300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:27:45.235300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:45.235300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:27:45.235300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:45.235300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:27:45.235300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:27:45.235300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:45.235300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:27:45.235654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:45.235654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:27:45.236647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:45.236647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:27:45.236647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:27:45.236647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:27:45.236647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:27:45.236647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:27:45.236647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:45.236647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:27:45.236647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:27:45.236647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:27:45.236647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:27:45.236647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:27:45.236647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:45.236647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:45.236647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:27:45.236647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:27:45.237104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:27:45.237104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:27:45.237104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:45.237104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:45.237104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:45.237104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:27:45.237104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:45.237104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:45.237104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:45.237104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:27:45.237104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:45.237104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:27:45.237104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:27:45.237104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:45.237104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:27:45.237104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:45.237104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:27:45.237104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:27:45.237104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:45.237673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:27:45.237673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:45.237673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:27:49.976274 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:27:49.977208 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:27:49.977395 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:27:49.978724 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:27:50.012881 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:27:50.013295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:50.014204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:27:50.014204 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:50.014204 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:27:50.014204 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:27:50.014204 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:50.014204 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:27:50.014204 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:50.014204 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:27:50.015078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:50.015078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:27:50.015078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:50.015078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:27:50.015078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:27:50.015078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:27:50.015078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:27:50.015078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:50.015078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:50.015078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:50.015078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:27:50.015078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:50.015519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:50.015519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:50.015519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:27:50.015519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:50.015519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:27:50.015519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:27:50.015519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:50.015519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:27:50.015519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:50.015519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:27:50.015519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:27:50.015519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:50.015519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:27:50.015847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:50.015847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:27:50.016187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:50.016187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:27:50.016187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:27:50.016187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:27:50.016187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:27:50.016187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:27:50.016187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:50.016187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:27:50.016187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:27:50.016187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:27:50.016187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:27:50.016187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:27:50.016187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:50.016187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:50.016187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:27:50.016187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:27:50.017236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:27:50.017236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:27:50.017236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:50.017236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:50.017236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:50.017236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:27:50.017236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:50.017236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:50.017236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:50.017236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:27:50.017236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:50.017236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:27:50.017236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:27:50.017236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:50.017236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:27:50.017236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:50.017236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:27:50.017236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:27:50.017236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:50.017930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:27:50.017930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:50.017930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:27:50.186548 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:27:50.188100 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:27:50.188192 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:27:50.189250 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:27:50.189645 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:27:50.189887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:50.190915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:27:50.190915 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:50.190915 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:27:50.190915 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:27:50.190915 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:50.190915 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:27:50.190915 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:50.190915 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:27:50.191925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:50.191925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:27:50.191925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:50.191925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:27:50.191925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:27:50.191925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:27:50.191925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:27:50.191925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:50.191925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:50.191925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:50.191925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:27:50.191925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:50.192534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:50.192534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:50.192534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:27:50.192534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:50.192534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:27:50.192534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:27:50.192534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:50.192534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:27:50.192534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:50.192534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:27:50.192534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:27:50.192534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:50.192534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:27:50.193076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:50.193076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:27:50.193076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:50.193076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:27:50.193076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:27:50.193076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:27:50.193076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:27:50.193076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:27:50.193076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:50.193076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:27:50.193076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:27:50.193076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:27:50.193076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:27:50.193076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:27:50.193076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:50.193759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:50.193759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:27:50.193759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:27:50.193759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:27:50.193759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:27:50.193759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:50.193759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:50.193759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:50.193759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:27:50.193759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:50.193759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:50.193759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:50.193759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:27:50.193759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:50.193759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:27:50.193759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:27:50.193759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:50.193759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:27:50.193759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:50.194572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:27:50.194572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:27:50.194572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:50.194572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:27:50.194572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:50.194572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:27:50.194572 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:27:50.194572 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:27:50.194572 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:27:50.195003 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:27:50.195003 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:27:50.196163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:50.196163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:27:50.196163 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:50.196163 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:27:50.196163 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:27:50.196163 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:50.196163 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:27:50.196163 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:50.196163 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:27:50.196824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:50.196824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:27:50.196824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:27:50.196824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:27:50.196824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:50.196824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:27:50.196824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:50.196824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:27:50.196824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:50.196824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:27:50.196824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:50.196824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:27:50.196824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:50.196824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:27:50.196824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:27:50.197750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:50.197750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:27:50.197750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:50.197750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:27:50.197750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:27:50.197750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:50.197750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:27:50.197750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:50.197750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:27:50.197750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:50.197750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:27:50.197750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:50.197750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:27:50.197750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:27:50.198626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:27:50.198626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:27:50.198626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:27:50.198626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:27:50.198626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:27:50.198626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:50.198626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:50.198626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:50.198626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:27:50.198626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:50.198626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:50.198626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:50.198626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:27:50.199527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:50.199527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:27:50.199527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:27:50.199527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:50.199527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:27:50.199527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:50.199527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:27:50.199527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:27:50.199527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:50.199527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:27:50.199527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:50.199527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:27:50.200283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:50.200283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:27:50.200283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:27:50.200283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:27:50.200283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:27:50.200283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:27:50.200283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:50.200283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:27:50.200283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:27:50.200283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:27:50.200283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:27:50.200283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:27:50.200283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:50.200283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:50.200283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:27:50.200283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:27:50.201261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:27:50.201261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:27:50.201261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:27:50.201261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:27:50.201261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:27:50.201261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:50.201261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:50.201261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:50.201261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:27:50.201261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:50.201261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:50.201261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:50.201261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:27:50.201261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:50.201261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:27:50.201261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:27:50.201261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:50.201261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:27:50.201810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:50.201810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:27:50.201810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:27:50.201810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:50.201810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:27:50.201810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:50.201810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:27:50.223130 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:27:50.223527 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:27:50.223757 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:27:50.223933 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:27:50.224450 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:27:50.224821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:50.225533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:27:50.225533 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:50.225533 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:27:50.225533 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:27:50.225533 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:50.225533 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:27:50.225533 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:50.225533 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:27:50.226361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:50.226361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:27:50.226361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:27:50.226361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:27:50.226361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:50.226361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:27:50.226361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:50.226361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:27:50.226361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:50.226361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:27:50.226361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:50.226361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:27:50.226361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:50.226361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:27:50.226361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:27:50.227039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:50.227039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:27:50.227039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:50.227039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:27:50.227039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:27:50.227039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:50.227039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:27:50.227039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:50.227039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:27:50.227039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:50.227039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:27:50.227039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:50.227039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:27:50.227039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:27:50.227732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:27:50.227732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:27:50.227732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:27:50.227732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:27:50.227732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:50.227732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:50.227732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:50.227732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:27:50.227732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:50.227732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:50.227732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:50.227732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:27:50.227732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:50.228552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:27:50.228552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:27:50.228552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:50.228552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:27:50.228552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:50.228552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:27:50.228552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:27:50.228552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:50.228552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:27:50.228552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:50.228552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:27:50.228552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:50.229701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:27:50.229701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:27:50.229701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:27:50.229701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:27:50.229701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:27:50.229701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:50.229701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:27:50.229701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:27:50.229701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:27:50.229701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:27:50.229701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:27:50.229701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:50.229701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:50.229701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:27:50.229701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:27:50.229701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:27:50.229701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:27:50.229701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:27:50.229701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:27:50.231362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:50.231362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:50.231362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:50.231362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:27:50.231362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:50.231362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:50.231362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:50.231362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:27:50.231362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:50.231362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:27:50.231362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:27:50.231362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:50.231362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:27:50.231362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:50.231362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:27:50.231362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:27:50.231362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:50.231362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:27:50.231362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:50.231362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:27:50.232463 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:27:50.232463 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:27:50.232463 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:27:50.232463 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:27:50.232463 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:27:50.232837 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:27:50.233040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:50.233699 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:27:50.233699 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:50.233699 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:27:50.233699 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:27:50.233699 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:50.233699 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:27:50.233699 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:50.233699 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:27:50.234493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:50.234493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:27:50.234493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:50.234493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:27:50.234493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:27:50.234493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:27:50.234493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:27:50.234493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:50.234493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:50.234493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:50.234493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:27:50.234493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:50.235130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:50.235130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:50.235130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:27:50.235130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:50.235130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:27:50.235130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:27:50.235130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:50.235130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:27:50.235130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:50.235130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:27:50.235130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:27:50.235130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:50.235130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:27:50.235927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:50.235927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:27:50.235927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:50.235927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:27:50.235927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:27:50.235927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:27:50.235927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:27:50.235927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:27:50.235927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:50.235927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:27:50.235927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:27:50.235927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:27:50.235927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:27:50.235927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:27:50.235927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:50.236469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:50.236469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:27:50.236469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:27:50.236469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:27:50.236469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:27:50.236469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:50.236469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:50.236469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:50.236469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:27:50.236469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:50.236469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:50.236469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:50.236469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:27:50.236469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:50.236469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:27:50.236469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:27:50.236469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:50.236469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:27:50.236469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:50.236949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:27:50.236949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:27:50.236949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:50.236949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:27:50.236949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:50.236949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:27:54.977254 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:27:54.978351 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:27:54.978351 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:27:54.980585 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:27:55.014072 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:27:55.014484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:55.015514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:27:55.015514 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:55.015514 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:27:55.015514 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:27:55.015514 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:55.015514 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:27:55.015514 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:55.015514 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:27:55.017026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:55.017026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:27:55.017026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:55.017026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:27:55.017026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:27:55.017026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:27:55.017026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:27:55.017026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:55.017026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:55.017026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:55.017026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:27:55.017026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:55.017777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:55.017777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:55.017777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:27:55.017777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:55.017777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:27:55.017777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:27:55.017777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:55.017777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:27:55.017777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:55.017777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:27:55.017777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:27:55.017777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:55.017777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:27:55.018582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:55.018582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:27:55.018830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:55.018830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:27:55.018830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:27:55.018830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:27:55.018830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:27:55.018830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:27:55.018830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:55.018830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:27:55.018830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:27:55.018830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:27:55.018830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:27:55.018830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:27:55.018830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:55.018830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:55.018830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:27:55.018830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:27:55.019941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:27:55.019941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:27:55.019941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:55.019941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:55.019941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:55.019941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:27:55.019941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:55.019941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:55.019941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:55.019941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:27:55.019941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:55.019941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:27:55.019941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:27:55.019941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:55.019941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:27:55.019941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:55.019941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:27:55.019941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:27:55.019941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:55.020921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:27:55.020921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:55.020921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:27:55.188685 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:27:55.189158 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:27:55.189354 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:27:55.190572 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:27:55.190975 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:27:55.191289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:55.192330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:27:55.192330 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:55.192330 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:27:55.192330 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:27:55.192330 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:55.192330 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:27:55.192330 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:55.192330 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:27:55.193807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:55.193807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:27:55.193807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:55.193807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:27:55.193807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:27:55.193807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:27:55.193807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:27:55.193807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:55.193807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:55.193807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:55.193807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:27:55.193807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:55.194476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:55.194476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:55.194476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:27:55.194476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:55.194476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:27:55.194476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:27:55.194476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:55.194476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:27:55.194476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:55.194476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:27:55.194476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:27:55.194476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:55.194476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:27:55.194955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:55.194955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:27:55.195433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:55.195433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:27:55.195433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:27:55.195433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:27:55.195433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:27:55.195433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:27:55.195433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:55.195433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:27:55.195433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:27:55.195433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:27:55.195433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:27:55.195433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:27:55.195433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:55.195433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:55.195433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:27:55.195433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:27:55.196082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:27:55.196082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:27:55.196082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:55.196082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:55.196082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:55.196082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:27:55.196082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:55.196082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:55.196082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:55.196082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:27:55.196082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:55.196082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:27:55.196082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:27:55.196082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:55.196082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:27:55.196082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:55.196082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:27:55.196082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:27:55.196082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:55.196665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:27:55.196665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:55.196665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:27:55.196665 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:27:55.196665 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:27:55.196665 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:27:55.196665 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:27:55.196665 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:27:55.196939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:55.197421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:27:55.197421 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:55.197421 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:27:55.197421 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:27:55.197421 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:55.197421 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:27:55.197421 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:55.197421 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:27:55.198128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:55.198128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:27:55.198128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:27:55.198128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:27:55.198128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:55.198128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:27:55.198128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:55.198128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:27:55.198128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:55.198128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:27:55.198128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:55.198128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:27:55.198128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:55.198128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:27:55.198128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:27:55.198663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:55.198663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:27:55.198663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:55.198663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:27:55.198663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:27:55.198663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:55.198663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:27:55.198663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:55.198663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:27:55.198971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:55.198971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:27:55.198971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:55.198971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:27:55.198971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:27:55.198971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:27:55.198971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:27:55.198971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:27:55.198971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:27:55.198971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:27:55.198971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:55.198971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:55.199340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:55.199340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:27:55.199340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:55.199340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:55.199340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:55.199340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:27:55.199340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:55.199340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:27:55.199340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:27:55.199340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:55.199340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:27:55.199340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:55.199340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:27:55.199340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:27:55.199737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:55.199737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:27:55.199737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:55.199737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:27:55.199944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:55.199944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:27:55.199944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:27:55.199944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:27:55.199944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:27:55.199944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:27:55.199944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:55.199944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:27:55.199944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:27:55.199944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:27:55.199944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:27:55.199944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:27:55.199944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:55.199944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:55.199944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:27:55.199944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:27:55.200465 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:27:55.200465 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:27:55.200465 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:27:55.200465 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:27:55.200465 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:27:55.200465 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:55.200465 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:55.200465 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:55.200465 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:27:55.200465 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:55.200465 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:55.200465 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:55.200465 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:27:55.200465 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:55.200465 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:27:55.200465 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:27:55.200465 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:55.200465 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:27:55.201020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:55.201020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:27:55.201020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:27:55.201020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:55.201020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:27:55.201020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:55.201020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:27:55.223599 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:27:55.223915 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:27:55.224096 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:27:55.224283 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:27:55.224975 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:27:55.225373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:55.226218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:27:55.226218 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:55.226218 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:27:55.226218 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:27:55.226218 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:55.226218 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:27:55.226218 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:55.226218 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:27:55.227820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:55.227820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:27:55.227820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:27:55.227820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:27:55.227820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:55.227820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:27:55.227820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:55.227820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:27:55.227820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:55.227820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:27:55.227820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:55.227820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:27:55.227820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:55.227820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:27:55.227820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:27:55.228278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:55.228278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:27:55.228278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:55.228278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:27:55.228278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:27:55.228278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:55.228278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:27:55.228278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:55.228278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:27:55.229187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:55.229187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:27:55.229187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:55.229187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:27:55.229187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:27:55.229187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:27:55.229187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:27:55.229187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:27:55.229187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:27:55.229187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:55.229187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:55.229632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:55.229632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:27:55.229632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:55.229632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:55.229632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:55.229632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:27:55.229632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:55.229632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:27:55.229632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:27:55.229632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:55.229632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:27:55.229632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:55.229632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:27:55.229632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:27:55.230063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:55.230063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:27:55.230063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:55.230063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:27:55.232221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:55.232221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:27:55.232221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:27:55.232221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:27:55.232221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:27:55.232221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:27:55.232221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:55.232221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:27:55.232221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:27:55.232221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:27:55.232221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:27:55.232221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:27:55.232221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:55.232221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:55.232221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:27:55.232221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:27:55.232221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:27:55.232604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:27:55.232604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:27:55.232604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:27:55.232604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:55.232604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:55.232604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:55.232604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:27:55.232604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:55.232604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:55.232604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:55.232604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:27:55.232604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:55.232604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:27:55.232604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:27:55.232604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:55.232604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:27:55.232604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:55.232604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:27:55.232604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:27:55.232968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:55.232968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:27:55.232968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:55.232968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:27:55.233251 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:27:55.234319 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:27:55.234319 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:27:55.234319 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:27:55.234654 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:27:55.235039 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:27:55.235261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:55.235974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:27:55.235974 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:55.235974 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:27:55.235974 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:27:55.235974 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:55.235974 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:27:55.235974 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:55.235974 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:27:55.236865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:55.236865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:27:55.236865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:55.236865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:27:55.236865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:27:55.236865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:27:55.236865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:27:55.236865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:55.236865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:55.236865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:55.236865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:27:55.236865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:55.237126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:27:55.237126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:55.237126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:27:55.237126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:55.237126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:27:55.237126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:27:55.237126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:55.237126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:27:55.237126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:55.237126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:27:55.237126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:27:55.237126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:55.237126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:27:55.237377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:55.237377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:27:55.237711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:55.237711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:27:55.237711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:27:55.237711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:27:55.237711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:27:55.237711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:27:55.237711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:55.237711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:27:55.237711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:27:55.237711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:27:55.237711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:27:55.237711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:27:55.237711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:27:55.237711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:27:55.237711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:27:55.237711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:27:55.238034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:27:55.238034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:27:55.238034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:55.238034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:55.238034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:27:55.238034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:27:55.238034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:27:55.238034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:27:55.238034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:27:55.238034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:27:55.238034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:27:55.238034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:27:55.238034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:27:55.238034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:27:55.238034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:27:55.238034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:27:55.238034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:27:55.238034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:27:55.238034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:27:55.238468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:27:55.238468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:27:55.238468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:27:59.978229 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:27:59.979149 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:27:59.979149 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:27:59.979605 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:28:00.014352 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:28:00.014657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:00.015351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:28:00.015351 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:00.015351 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:28:00.015351 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:28:00.015351 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:00.015351 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:28:00.015351 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:00.015351 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:28:00.016653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:00.016653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:28:00.016653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:00.016653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:28:00.016653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:28:00.016653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:28:00.016653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:28:00.016653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:00.016653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:00.016653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:00.016653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:28:00.016653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:00.017616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:00.017616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:00.017616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:28:00.017616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:00.017616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:28:00.017616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:28:00.017616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:00.017616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:28:00.017616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:00.017616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:28:00.017616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:28:00.017616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:00.017616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:28:00.018478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:00.018478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:28:00.018478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:00.018478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:28:00.018478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:28:00.018478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:28:00.018478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:28:00.018478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:28:00.018478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:00.018478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:28:00.018478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:28:00.018478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:28:00.018478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:28:00.018478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:28:00.018478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:00.019342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:00.019342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:28:00.019342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:28:00.019342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:28:00.019342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:28:00.019342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:00.019342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:00.019342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:00.019342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:28:00.019342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:00.019342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:00.019342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:00.019342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:28:00.019342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:00.019342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:28:00.019342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:28:00.019342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:00.019342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:28:00.019342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:00.020374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:28:00.020374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:28:00.020374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:00.020374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:28:00.020374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:00.020374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:28:00.189303 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:28:00.189956 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:28:00.190097 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:28:00.191136 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:28:00.191546 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:28:00.191791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:00.193197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:28:00.193197 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:00.193197 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:28:00.193197 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:28:00.193197 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:00.193197 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:28:00.193197 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:00.193197 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:28:00.194368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:00.194368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:28:00.194368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:00.194368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:28:00.194368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:28:00.194368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:28:00.194368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:28:00.194368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:00.194368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:00.194368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:00.194368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:28:00.194368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:00.195178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:00.195178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:00.195178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:28:00.195178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:00.195178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:28:00.195178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:28:00.195178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:00.195178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:28:00.195178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:00.195178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:28:00.195178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:28:00.195178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:00.195178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:28:00.196041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:00.196041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:28:00.196041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:00.196041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:28:00.196041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:28:00.196041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:28:00.196041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:28:00.196041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:28:00.196041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:00.196041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:28:00.196041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:28:00.196041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:28:00.196041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:28:00.196041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:28:00.196041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:00.197050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:00.197050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:28:00.197050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:28:00.197050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:28:00.197050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:28:00.197050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:00.197050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:00.197050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:00.197050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:28:00.197050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:00.197050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:00.197050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:00.197050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:28:00.197050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:00.197050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:28:00.197050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:28:00.197050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:00.197050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:28:00.197050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:00.198069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:28:00.198069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:28:00.198069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:00.198069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:28:00.198069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:00.198069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:28:00.198069 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:28:00.198069 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:28:00.198069 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:28:00.198069 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:28:00.198538 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:28:00.198538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:00.198538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:28:00.198538 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:00.198538 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:28:00.198538 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:28:00.198538 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:00.198538 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:28:00.198538 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:00.198538 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:28:00.198538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:00.198538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:28:00.198538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:28:00.198538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:28:00.199127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:00.199127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:28:00.199127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:00.199127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:28:00.199127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:00.199127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:28:00.199127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:00.199127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:28:00.199127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:00.199127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:28:00.199127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:28:00.199127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:00.199127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:28:00.199127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:00.199127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:28:00.199127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:28:00.199127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:00.199127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:28:00.199948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:00.199948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:28:00.199948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:00.199948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:28:00.199948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:00.199948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:28:00.199948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:28:00.199948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:28:00.199948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:28:00.199948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:28:00.199948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:28:00.199948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:28:00.200638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:00.200638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:00.200638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:00.200638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:28:00.200638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:00.200638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:00.200638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:00.200638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:28:00.200638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:00.200638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:28:00.200638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:28:00.200638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:00.200638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:28:00.200638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:00.201129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:28:00.201129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:28:00.201129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:00.201129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:28:00.201129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:00.201129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:28:00.201129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:00.201129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:28:00.201129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:28:00.201129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:28:00.201129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:28:00.201129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:28:00.201129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:00.201129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:28:00.201686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:28:00.201686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:28:00.201686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:28:00.201686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:28:00.201686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:00.201686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:00.201686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:28:00.201686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:28:00.201686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:28:00.201686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:28:00.201686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:28:00.201686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:28:00.201686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:28:00.201686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:00.201686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:00.201686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:00.201686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:28:00.201686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:00.201686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:00.202490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:00.202490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:28:00.202490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:00.202490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:28:00.202490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:28:00.202490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:00.202490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:28:00.202490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:00.202490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:28:00.202490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:28:00.202490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:00.202490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:28:00.202490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:00.202490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:28:00.224064 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:28:00.224704 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:28:00.224704 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:28:00.224879 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:28:00.225337 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:28:00.225905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:00.226610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:28:00.226610 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:00.226610 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:28:00.226610 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:28:00.226610 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:00.226610 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:28:00.226610 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:00.226610 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:28:00.227821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:00.227821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:28:00.227821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:28:00.227821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:28:00.227821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:00.227821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:28:00.227821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:00.227821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:28:00.227821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:00.227821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:28:00.227821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:00.227821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:28:00.227821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:00.227821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:28:00.227821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:28:00.228450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:00.228450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:28:00.228450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:00.228450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:28:00.228450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:28:00.228450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:00.228450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:28:00.228450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:00.228450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:28:00.228450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:00.228450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:28:00.228450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:00.228450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:28:00.228450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:28:00.228980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:28:00.228980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:28:00.228980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:28:00.228980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:28:00.228980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:00.228980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:00.228980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:00.228980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:28:00.228980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:00.228980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:00.228980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:00.228980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:28:00.228980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:00.229597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:28:00.229597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:28:00.229597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:00.229597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:28:00.229597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:00.229597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:28:00.229597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:28:00.229597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:00.229597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:28:00.229597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:00.229597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:28:00.229597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:00.230146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:28:00.230146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:28:00.230146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:28:00.230146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:28:00.230146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:28:00.230146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:00.230146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:28:00.230146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:28:00.230146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:28:00.230146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:28:00.230146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:28:00.230146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:00.230146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:00.230146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:28:00.230146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:28:00.230146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:28:00.230146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:28:00.230146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:28:00.230146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:28:00.230855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:00.230855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:00.230855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:00.230855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:28:00.230855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:00.230855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:00.230855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:00.230855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:28:00.230855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:00.230855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:28:00.230855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:28:00.230855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:00.230855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:28:00.230855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:00.230855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:28:00.230855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:28:00.230855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:00.230855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:28:00.230855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:00.230855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:28:00.234623 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:28:00.234623 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:28:00.234623 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:28:00.234623 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:28:00.234623 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:28:00.234935 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:28:00.235109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:00.235812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:28:00.235812 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:00.235812 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:28:00.235812 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:28:00.235812 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:00.235812 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:28:00.235812 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:00.235812 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:28:00.236417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:00.236417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:28:00.236417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:00.236417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:28:00.236417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:28:00.236417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:28:00.236417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:28:00.236417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:00.236417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:00.236417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:00.236417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:28:00.236417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:00.236778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:00.236778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:00.236778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:28:00.236778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:00.236778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:28:00.236778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:28:00.236778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:00.236778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:28:00.236778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:00.236778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:28:00.236778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:28:00.236778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:00.236778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:28:00.237192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:00.237192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:28:00.237192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:00.237192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:28:00.237192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:28:00.237192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:28:00.237192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:28:00.237192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:28:00.237192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:00.237192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:28:00.237192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:28:00.237192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:28:00.237192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:28:00.237192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:28:00.237192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:00.237668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:00.237668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:28:00.237668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:28:00.237668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:28:00.237668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:28:00.237668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:00.237668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:00.237668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:00.237668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:28:00.237668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:00.237668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:00.237668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:00.237668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:28:00.237668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:00.237668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:28:00.237668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:28:00.237668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:00.237668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:28:00.237668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:00.238050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:28:00.238050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:28:00.238050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:00.238050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:28:00.238050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:00.238050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:28:04.978971 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:28:04.980090 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:28:04.980090 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:28:04.981231 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:28:05.021430 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:28:05.022019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:05.023060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:28:05.023060 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:05.023060 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:28:05.023060 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:28:05.023060 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:05.023060 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:28:05.023060 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:05.023060 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:28:05.024306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:05.024306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:28:05.024306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:05.024306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:28:05.024306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:28:05.024306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:28:05.024306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:28:05.024306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:05.024306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:05.024306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:05.024306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:28:05.024306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:05.024979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:05.024979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:05.024979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:28:05.024979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:05.024979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:28:05.024979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:28:05.024979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:05.024979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:28:05.024979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:05.024979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:28:05.024979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:28:05.024979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:05.024979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:28:05.025687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:05.025687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:28:05.025898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:05.025898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:28:05.025898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:28:05.025898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:28:05.025898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:28:05.025898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:28:05.025898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:05.025898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:28:05.025898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:28:05.025898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:28:05.025898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:28:05.025898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:28:05.025898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:05.025898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:05.025898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:28:05.025898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:28:05.026546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:28:05.026546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:28:05.026546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:05.026546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:05.026546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:05.026546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:28:05.026546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:05.026546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:05.026546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:05.026546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:28:05.026546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:05.026546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:28:05.026546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:28:05.026546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:05.026546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:28:05.026546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:05.026546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:28:05.026546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:28:05.026546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:05.027168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:28:05.027168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:05.027168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:28:05.189795 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:28:05.190105 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:28:05.190262 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:28:05.191197 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:28:05.191469 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:28:05.191629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:05.192384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:28:05.192384 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:05.192384 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:28:05.192384 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:28:05.192384 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:05.192384 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:28:05.192384 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:05.192384 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:28:05.194052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:05.194052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:28:05.194052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:05.194052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:28:05.194052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:28:05.194052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:28:05.194052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:28:05.194052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:05.194052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:05.194052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:05.194052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:28:05.194052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:05.194950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:05.194950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:05.194950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:28:05.194950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:05.194950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:28:05.194950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:28:05.194950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:05.194950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:28:05.194950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:05.194950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:28:05.194950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:28:05.194950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:05.194950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:28:05.195825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:05.195825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:28:05.196542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:05.196542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:28:05.196542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:28:05.196542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:28:05.196542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:28:05.196542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:28:05.196542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:05.196542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:28:05.196542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:28:05.196542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:28:05.196542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:28:05.196542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:28:05.196542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:05.196542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:05.196542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:28:05.196542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:28:05.197157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:28:05.197157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:28:05.197157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:05.197157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:05.197157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:05.197157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:28:05.197157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:05.197157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:05.197157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:05.197157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:28:05.197157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:05.197157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:28:05.197157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:28:05.197157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:05.197157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:28:05.197157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:05.197157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:28:05.197157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:28:05.197157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:05.197932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:28:05.197932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:05.197932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:28:05.197932 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:28:05.197932 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:28:05.197932 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:28:05.197932 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:28:05.198201 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:28:05.198367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:05.198890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:28:05.198890 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:05.198890 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:28:05.198890 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:28:05.198890 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:05.198890 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:28:05.198890 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:05.198890 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:28:05.199775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:05.199775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:28:05.199775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:28:05.199775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:28:05.199775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:05.199775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:28:05.199775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:05.199775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:28:05.199775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:05.199775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:28:05.199775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:05.199775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:28:05.199775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:05.199775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:28:05.199775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:28:05.202935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:05.202935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:28:05.202935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:05.202935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:28:05.202935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:28:05.202935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:05.202935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:28:05.202935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:05.202935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:28:05.202935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:05.202935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:28:05.202935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:05.202935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:28:05.202935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:28:05.203364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:28:05.203364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:28:05.203364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:28:05.203364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:28:05.203364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:28:05.203364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:05.203364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:05.203364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:05.203364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:28:05.203364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:05.203364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:05.203364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:05.203364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:28:05.203751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:05.203751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:28:05.203751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:28:05.203751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:05.203751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:28:05.203751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:05.203751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:28:05.203751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:28:05.203751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:05.203751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:28:05.203751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:05.203751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:28:05.204080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:05.204080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:28:05.204080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:28:05.204080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:28:05.204080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:28:05.204080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:28:05.204080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:05.204080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:28:05.204080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:28:05.204080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:28:05.204080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:28:05.204080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:28:05.204080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:05.204080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:05.204080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:28:05.204080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:28:05.204562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:28:05.204562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:28:05.204562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:28:05.204562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:28:05.204562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:28:05.204562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:05.204562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:05.204562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:05.204562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:28:05.204562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:05.204562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:05.204562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:05.204562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:28:05.204562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:05.204562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:28:05.204562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:28:05.204562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:05.204562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:28:05.205114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:05.205114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:28:05.205114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:28:05.205114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:05.205114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:28:05.205114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:05.205114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:28:05.224773 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:28:05.225220 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:28:05.225509 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:28:05.225803 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:28:05.226225 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:28:05.226492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:05.227097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:28:05.227097 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:05.227097 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:28:05.227097 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:28:05.227097 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:05.227097 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:28:05.227097 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:05.227097 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:28:05.228786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:05.228786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:28:05.228786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:28:05.228786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:28:05.228786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:05.228786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:28:05.228786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:05.228786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:28:05.228786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:05.228786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:28:05.228786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:05.228786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:28:05.228786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:05.228786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:28:05.228786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:28:05.229767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:05.229767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:28:05.229767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:05.229767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:28:05.229767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:28:05.229767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:05.229767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:28:05.229767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:05.229767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:28:05.230758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:05.230758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:28:05.230758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:05.230758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:28:05.230758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:28:05.230758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:28:05.230758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:28:05.230758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:28:05.230758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:28:05.230758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:05.230758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:05.231590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:05.231590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:28:05.231590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:05.231590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:05.231590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:05.231590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:28:05.231590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:05.231590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:28:05.231590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:28:05.231590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:05.231590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:28:05.231590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:05.231590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:28:05.231590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:28:05.232584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:05.232584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:28:05.232584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:05.232584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:28:05.233197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:05.233197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:28:05.233197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:28:05.233197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:28:05.233197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:28:05.233197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:28:05.233197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:05.233197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:28:05.233197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:28:05.233197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:28:05.233197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:28:05.233197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:28:05.233197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:05.233197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:05.233197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:28:05.233197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:28:05.233197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:28:05.234140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:28:05.234140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:28:05.234140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:28:05.234140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:05.234140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:05.234140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:05.234140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:28:05.234140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:05.234140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:05.234140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:05.234140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:28:05.234140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:05.234140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:28:05.234140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:28:05.234140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:05.234140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:28:05.234140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:05.234140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:28:05.234140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:28:05.235089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:05.235089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:28:05.235089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:05.235089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:28:05.235089 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:28:05.235089 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:28:05.235089 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:28:05.235089 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:28:05.235283 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:28:05.235632 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:28:05.235812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:05.236329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:28:05.236329 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:05.236329 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:28:05.236329 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:28:05.236329 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:05.236329 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:28:05.236329 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:05.236329 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:28:05.237030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:05.237030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:28:05.237030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:05.237030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:28:05.237030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:28:05.237030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:28:05.237030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:28:05.237030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:05.237030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:05.237030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:05.237030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:28:05.237030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:05.237433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:05.237433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:05.237433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:28:05.237433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:05.237433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:28:05.237433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:28:05.237433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:05.237433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:28:05.237433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:05.237433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:28:05.237433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:28:05.237433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:05.237433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:28:05.237842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:05.237842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:28:05.238005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:05.238005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:28:05.238005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:28:05.238005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:28:05.238005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:28:05.238005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:28:05.238005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:05.238005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:28:05.238005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:28:05.238005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:28:05.238005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:28:05.238005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:28:05.238005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:05.238005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:05.238005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:28:05.238005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:28:05.238350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:28:05.238350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:28:05.238350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:05.238350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:05.238350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:05.238350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:28:05.238350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:05.238350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:05.238350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:05.238350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:28:05.238350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:05.238350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:28:05.238350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:28:05.238350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:05.238350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:28:05.238350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:05.238350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:28:05.238350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:28:05.238350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:05.238745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:28:05.238745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:05.238745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:28:09.979841 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:28:09.980895 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:28:09.980895 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:28:09.982177 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:28:10.015334 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:28:10.016363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:10.017373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:28:10.017373 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:10.017373 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:28:10.017373 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:28:10.017373 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:10.017373 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:28:10.017373 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:10.017373 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:28:10.018395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:10.018395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:28:10.018395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:10.018395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:28:10.018395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:28:10.018395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:28:10.018395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:28:10.018395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:10.018395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:10.018395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:10.018395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:28:10.018395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:10.019264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:10.019264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:10.019264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:28:10.019264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:10.019264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:28:10.019264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:28:10.019264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:10.019264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:28:10.019264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:10.019264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:28:10.019264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:28:10.019264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:10.019264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:28:10.020070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:10.020070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:28:10.020070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:10.020070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:28:10.020070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:28:10.020070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:28:10.020070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:28:10.020070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:28:10.020070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:10.020070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:28:10.020070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:28:10.020070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:28:10.020070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:28:10.020070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:28:10.020070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:10.020592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:10.020592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:28:10.020592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:28:10.020592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:28:10.020592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:28:10.020592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:10.020592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:10.020592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:10.020592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:28:10.020592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:10.020592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:10.020592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:10.020592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:28:10.020592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:10.020592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:28:10.020592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:28:10.020592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:10.020592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:28:10.020592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:10.021206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:28:10.021206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:28:10.021206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:10.021206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:28:10.021206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:10.021206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:28:10.190525 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:28:10.191177 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:28:10.191440 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:28:10.192636 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:28:10.192996 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:28:10.193290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:10.194245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:28:10.194245 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:10.194245 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:28:10.194245 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:28:10.194245 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:10.194245 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:28:10.194245 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:10.194245 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:28:10.195911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:10.195911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:28:10.195911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:10.195911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:28:10.195911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:28:10.195911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:28:10.195911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:28:10.195911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:10.195911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:10.195911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:10.195911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:28:10.195911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:10.196548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:10.196548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:10.196548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:28:10.196548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:10.196548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:28:10.196548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:28:10.196548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:10.196548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:28:10.196548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:10.196548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:28:10.196548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:28:10.196548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:10.196548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:28:10.197125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:10.197125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:28:10.197125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:10.197125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:28:10.197125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:28:10.197125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:28:10.197125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:28:10.197125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:28:10.197125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:10.197125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:28:10.197125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:28:10.197125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:28:10.197125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:28:10.197125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:28:10.197125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:10.197827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:10.197827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:28:10.197827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:28:10.197827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:28:10.197827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:28:10.197827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:10.197827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:10.197827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:10.197827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:28:10.197827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:10.197827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:10.197827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:10.197827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:28:10.197827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:10.197827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:28:10.197827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:28:10.197827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:10.197827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:28:10.197827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:10.198781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:28:10.198781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:28:10.198781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:10.198781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:28:10.198781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:10.198781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:28:10.198781 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:28:10.198781 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:28:10.198781 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:28:10.198781 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:28:10.199366 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:28:10.199366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:10.199745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:28:10.199745 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:10.199745 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:28:10.199745 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:28:10.199745 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:10.199745 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:28:10.199745 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:10.199745 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:28:10.200506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:10.200506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:28:10.200506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:28:10.200506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:28:10.200506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:10.200506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:28:10.200506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:10.200506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:28:10.200506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:10.200506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:28:10.200506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:10.200506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:28:10.200506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:10.200506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:28:10.200506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:28:10.200991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:10.200991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:28:10.200991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:10.200991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:28:10.200991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:28:10.200991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:10.200991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:28:10.200991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:10.200991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:28:10.201326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:10.201326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:28:10.201326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:10.201326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:28:10.201326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:28:10.201326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:28:10.201326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:28:10.201326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:28:10.201326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:28:10.201326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:28:10.201326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:10.201326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:10.201721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:10.201721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:28:10.201721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:10.201721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:10.201721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:10.201721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:28:10.201721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:10.201721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:28:10.201721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:28:10.201721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:10.201721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:28:10.201721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:10.201721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:28:10.201721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:28:10.202110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:10.202110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:28:10.202110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:10.202110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:28:10.202300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:10.202300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:28:10.202300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:28:10.202300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:28:10.202300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:28:10.202300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:28:10.202300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:10.202300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:28:10.202300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:28:10.202300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:28:10.202300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:28:10.202300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:28:10.202300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:10.202300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:10.202300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:28:10.202300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:28:10.202774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:28:10.202774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:28:10.202774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:28:10.202774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:28:10.202774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:28:10.202774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:10.202774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:10.202774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:10.202774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:28:10.202774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:10.202774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:10.202774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:10.202774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:28:10.202774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:10.202774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:28:10.202774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:28:10.202774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:10.202774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:28:10.203330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:10.203330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:28:10.203330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:28:10.203330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:10.203330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:28:10.203330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:10.203330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:28:10.225589 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:28:10.225589 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:28:10.225886 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:28:10.225886 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:28:10.226314 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:28:10.226703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:10.227529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:28:10.227529 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:10.227529 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:28:10.227529 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:28:10.227529 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:10.227529 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:28:10.227529 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:10.227529 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:28:10.228319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:10.228319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:28:10.228319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:28:10.228319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:28:10.228319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:10.228319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:28:10.228319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:10.228319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:28:10.228319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:10.228319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:28:10.228319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:10.228319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:28:10.228319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:10.228319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:28:10.228319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:28:10.228784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:10.228784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:28:10.228784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:10.228784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:28:10.228784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:28:10.228784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:10.228784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:28:10.228784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:10.228784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:28:10.230009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:10.230009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:28:10.230009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:10.230009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:28:10.230009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:28:10.230009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:28:10.230009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:28:10.230009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:28:10.230009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:28:10.230009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:10.230009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:10.230462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:10.230462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:28:10.230462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:10.230462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:10.230462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:10.230462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:28:10.230462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:10.230462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:28:10.230462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:28:10.230462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:10.230462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:28:10.230462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:10.230462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:28:10.230462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:28:10.230886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:10.230886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:28:10.230886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:10.230886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:28:10.231075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:10.231075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:28:10.231075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:28:10.231075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:28:10.231075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:28:10.231075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:28:10.231075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:10.231075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:28:10.231075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:28:10.231075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:28:10.231075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:28:10.231075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:28:10.231075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:10.231075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:10.231075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:28:10.231075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:28:10.231075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:28:10.231690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:28:10.231690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:28:10.231690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:28:10.231690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:10.231690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:10.231690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:10.231690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:28:10.231690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:10.231690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:10.231690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:10.231690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:28:10.231690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:10.231690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:28:10.231690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:28:10.231690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:10.231690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:28:10.231690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:10.231690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:28:10.231690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:28:10.232422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:10.232422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:28:10.232422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:10.232422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:28:10.234567 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:28:10.234848 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:28:10.235015 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:28:10.235183 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:28:10.235664 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:28:10.236032 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:28:10.236291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:10.237046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:28:10.237046 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:10.237046 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:28:10.237046 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:28:10.237046 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:10.237046 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:28:10.237046 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:10.237046 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:28:10.237852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:10.237852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:28:10.237852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:10.237852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:28:10.237852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:28:10.237852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:28:10.237852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:28:10.237852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:10.237852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:10.237852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:10.237852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:28:10.237852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:10.238218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:10.238218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:10.238218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:28:10.238218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:10.238218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:28:10.238218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:28:10.238218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:10.238218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:28:10.238218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:10.238218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:28:10.238218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:28:10.238218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:10.238218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:28:10.238592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:10.238592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:28:10.238806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:10.238806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:28:10.238806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:28:10.238806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:28:10.238806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:28:10.238806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:28:10.238806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:10.238806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:28:10.238806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:28:10.238806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:28:10.238806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:28:10.238806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:28:10.238806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:10.238806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:10.238806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:28:10.238806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:28:10.239334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:28:10.239334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:28:10.239334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:10.239334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:10.239334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:10.239334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:28:10.239334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:10.239334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:10.239334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:10.239334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:28:10.239334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:10.239334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:28:10.239334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:28:10.239334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:10.239334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:28:10.239334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:10.239334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:28:10.239334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:28:10.239334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:10.239880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:28:10.239880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:10.239880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:28:14.980861 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:28:14.981712 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:28:14.981712 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:28:14.982606 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:28:15.016169 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:28:15.016600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:15.017557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:28:15.017557 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:15.017557 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:28:15.017557 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:28:15.017557 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:15.017557 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:28:15.017557 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:15.017557 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:28:15.018435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:15.018435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:28:15.018435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:15.018435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:28:15.018435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:28:15.018435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:28:15.018435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:28:15.018435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:15.018435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:15.018435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:15.018435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:28:15.018435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:15.019156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:15.019156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:15.019156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:28:15.019156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:15.019156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:28:15.019156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:28:15.019156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:15.019156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:28:15.019156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:15.019156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:28:15.019156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:28:15.019156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:15.019156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:28:15.020123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:15.020123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:28:15.020123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:15.020123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:28:15.020123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:28:15.020123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:28:15.020123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:28:15.020123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:28:15.020123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:15.020123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:28:15.020123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:28:15.020123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:28:15.020123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:28:15.020123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:28:15.020123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:15.021170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:15.021170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:28:15.021170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:28:15.021170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:28:15.021170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:28:15.021170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:15.021170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:15.021170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:15.021170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:28:15.021170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:15.021170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:15.021170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:15.021170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:28:15.021170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:15.021170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:28:15.021170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:28:15.021170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:15.021170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:28:15.021170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:15.022392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:28:15.022392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:28:15.022392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:15.022392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:28:15.022392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:15.022392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:28:15.191280 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:28:15.191773 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:28:15.192002 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:28:15.192992 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:28:15.193370 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:28:15.193763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:15.194747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:28:15.194747 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:15.194747 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:28:15.194747 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:28:15.194747 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:15.194747 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:28:15.194747 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:15.194747 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:28:15.196433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:15.196433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:28:15.196433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:15.196433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:28:15.196433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:28:15.196433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:28:15.196433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:28:15.196433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:15.196433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:15.196433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:15.196433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:28:15.196433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:15.197153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:15.197153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:15.197153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:28:15.197153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:15.197153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:28:15.197153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:28:15.197153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:15.197153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:28:15.197153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:15.197153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:28:15.197153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:28:15.197153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:15.197153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:28:15.197840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:15.197840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:28:15.198369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:15.198369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:28:15.198369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:28:15.198369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:28:15.198369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:28:15.198369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:28:15.198369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:15.198369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:28:15.198369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:28:15.198369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:28:15.198369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:28:15.198369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:28:15.198369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:15.198369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:15.198369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:28:15.198369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:28:15.199226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:28:15.199226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:28:15.199226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:15.199226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:15.199226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:15.199226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:28:15.199226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:15.199226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:15.199226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:15.199226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:28:15.199226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:15.199226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:28:15.199226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:28:15.199226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:15.199226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:28:15.199226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:15.199226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:28:15.199226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:28:15.199226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:15.200189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:28:15.200189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:15.200189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:28:15.200189 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:28:15.200189 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:28:15.200189 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:28:15.200189 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:28:15.200189 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:28:15.200659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:15.200659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:28:15.200659 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:15.200659 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:28:15.200659 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:28:15.200659 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:15.200659 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:28:15.200659 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:15.200659 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:28:15.201162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:15.201162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:28:15.201162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:28:15.201162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:28:15.201162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:15.201162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:28:15.201162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:15.201162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:28:15.201162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:15.201162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:28:15.201162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:15.201162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:28:15.201162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:15.201162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:28:15.201162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:28:15.202133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:15.202133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:28:15.202133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:15.202133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:28:15.202133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:28:15.202133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:15.202133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:28:15.202133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:15.202133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:28:15.202133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:15.202133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:28:15.202133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:15.202133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:28:15.202133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:28:15.202647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:28:15.202647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:28:15.202647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:28:15.202647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:28:15.202647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:28:15.202647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:15.202647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:15.202647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:15.202647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:28:15.202647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:15.202647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:15.202647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:15.202647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:28:15.203070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:15.203070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:28:15.203070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:28:15.203070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:15.203070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:28:15.203070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:15.203070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:28:15.203070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:28:15.203070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:15.203070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:28:15.203070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:15.203070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:28:15.203487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:15.203487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:28:15.203487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:28:15.203487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:28:15.203487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:28:15.203487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:28:15.203487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:15.203487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:28:15.203487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:28:15.203487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:28:15.203487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:28:15.203487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:28:15.203487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:15.203487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:15.203487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:28:15.203487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:28:15.203964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:28:15.203964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:28:15.203964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:28:15.203964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:28:15.203964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:28:15.203964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:15.203964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:15.203964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:15.203964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:28:15.203964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:15.203964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:15.203964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:15.203964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:28:15.203964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:15.203964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:28:15.203964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:28:15.203964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:15.203964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:28:15.204542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:15.204542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:28:15.204542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:28:15.204542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:15.204542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:28:15.204542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:15.204542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:28:15.225740 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:28:15.226119 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:28:15.226291 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:28:15.226573 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:28:15.226867 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:28:15.227040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:15.227571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:28:15.227571 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:15.227571 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:28:15.227571 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:28:15.227571 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:15.227571 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:28:15.227571 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:15.227571 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:28:15.228727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:15.228727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:28:15.228727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:28:15.228727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:28:15.228727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:15.228727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:28:15.228727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:15.228727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:28:15.228727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:15.228727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:28:15.228727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:15.228727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:28:15.228727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:15.228727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:28:15.228727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:28:15.229386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:15.229386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:28:15.229386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:15.229386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:28:15.229386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:28:15.229386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:15.229386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:28:15.229386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:15.229386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:28:15.230821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:15.230821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:28:15.230821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:15.230821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:28:15.230821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:28:15.230821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:28:15.230821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:28:15.230821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:28:15.230821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:28:15.230821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:15.230821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:15.231322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:15.231322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:28:15.231322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:15.231322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:15.231322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:15.231322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:28:15.231322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:15.231322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:28:15.231322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:28:15.231322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:15.231322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:28:15.231322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:15.231322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:28:15.231322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:28:15.232092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:15.232092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:28:15.232092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:15.232092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:28:15.232092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:15.232092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:28:15.232092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:28:15.232092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:28:15.232092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:28:15.232092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:28:15.232092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:15.232092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:28:15.232092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:28:15.232092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:28:15.232839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:28:15.232839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:28:15.232839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:15.232839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:15.232839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:28:15.232839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:28:15.232839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:28:15.232839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:28:15.232839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:28:15.232839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:28:15.232839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:15.232839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:15.232839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:15.232839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:28:15.232839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:15.232839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:15.232839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:15.232839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:28:15.232839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:15.233955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:28:15.233955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:28:15.233955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:15.233955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:28:15.233955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:15.233955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:28:15.233955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:28:15.233955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:15.233955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:28:15.233955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:15.233955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:28:15.234896 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:28:15.235307 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:28:15.235498 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:28:15.235779 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:28:15.236100 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:28:15.236337 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:28:15.236505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:15.237001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:28:15.237001 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:15.237001 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:28:15.237001 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:28:15.237001 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:15.237001 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:28:15.237001 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:15.237001 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:28:15.237878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:15.237878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:28:15.237878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:15.237878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:28:15.237878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:28:15.237878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:28:15.237878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:28:15.237878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:15.237878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:15.237878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:15.237878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:28:15.237878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:15.238279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:15.238279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:15.238279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:28:15.238279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:15.238279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:28:15.238279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:28:15.238279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:15.238279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:28:15.238279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:15.238279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:28:15.238279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:28:15.238279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:15.238279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:28:15.238671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:15.238671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:28:15.238860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:15.238860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:28:15.238860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:28:15.238860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:28:15.238860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:28:15.238860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:28:15.238860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:15.238860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:28:15.238860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:28:15.238860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:28:15.238860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:28:15.238860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:28:15.238860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:15.238860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:15.238860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:28:15.238860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:28:15.239443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:28:15.239443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:28:15.239443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:15.239443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:15.239443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:15.239443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:28:15.239443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:15.239443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:15.239443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:15.239443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:28:15.239443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:15.239443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:28:15.239443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:28:15.239443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:15.239443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:28:15.239443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:15.239443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:28:15.239443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:28:15.239443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:15.240051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:28:15.240051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:15.240051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:28:19.981692 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:28:19.982560 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:28:19.982560 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:28:19.984018 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:28:20.016694 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:28:20.017081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:20.017923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:28:20.017923 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:20.017923 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:28:20.017923 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:28:20.017923 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:20.017923 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:28:20.017923 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:20.017923 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:28:20.018699 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:20.018699 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:28:20.018699 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:20.018699 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:28:20.018699 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:28:20.018699 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:28:20.018699 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:28:20.018699 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:20.018699 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:20.018699 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:20.018699 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:28:20.018699 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:20.019296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:20.019296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:20.019296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:28:20.019296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:20.019296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:28:20.019296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:28:20.019296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:20.019296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:28:20.019296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:20.019296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:28:20.019296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:28:20.019296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:20.019296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:28:20.019941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:20.019941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:28:20.019941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:20.019941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:28:20.019941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:28:20.019941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:28:20.019941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:28:20.019941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:28:20.019941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:20.019941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:28:20.019941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:28:20.019941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:28:20.019941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:28:20.019941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:28:20.019941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:20.020695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:20.020695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:28:20.020695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:28:20.020695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:28:20.020695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:28:20.020695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:20.020695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:20.020695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:20.020695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:28:20.020695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:20.020695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:20.020695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:20.020695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:28:20.020695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:20.020695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:28:20.020695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:28:20.020695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:20.020695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:28:20.020695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:20.022512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:28:20.022512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:28:20.022512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:20.022512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:28:20.022512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:20.022512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:28:20.191802 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:28:20.192377 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:28:20.192614 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:28:20.193780 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:28:20.194144 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:28:20.194488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:20.195305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:28:20.195305 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:20.195305 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:28:20.195305 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:28:20.195305 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:20.195305 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:28:20.195305 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:20.195305 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:28:20.197238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:20.197238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:28:20.197238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:20.197238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:28:20.197238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:28:20.197238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:28:20.197238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:28:20.197238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:20.197238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:20.197238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:20.197238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:28:20.197238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:20.198105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:20.198105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:20.198105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:28:20.198105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:20.198105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:28:20.198105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:28:20.198105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:20.198105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:28:20.198105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:20.198105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:28:20.198105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:28:20.198105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:20.198105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:28:20.198465 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:20.198465 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:28:20.198983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:20.198983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:28:20.198983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:28:20.198983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:28:20.198983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:28:20.198983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:28:20.198983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:20.198983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:28:20.198983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:28:20.198983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:28:20.198983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:28:20.198983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:28:20.198983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:20.198983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:20.198983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:28:20.198983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:28:20.199446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:28:20.199446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:28:20.199446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:20.199446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:20.199446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:20.199446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:28:20.199446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:20.199446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:20.199446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:20.199446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:28:20.199446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:20.199446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:28:20.199446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:28:20.199446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:20.199446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:28:20.199446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:20.199446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:28:20.199446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:28:20.199446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:20.199936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:28:20.199936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:20.199936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:28:20.199936 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:28:20.199936 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:28:20.199936 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:28:20.199936 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:28:20.200497 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:28:20.200569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:20.201102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:28:20.201102 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:20.201102 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:28:20.201102 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:28:20.201102 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:20.201102 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:28:20.201102 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:20.201102 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:28:20.202245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:20.202245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:28:20.202245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:28:20.202245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:28:20.202245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:20.202245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:28:20.202245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:20.202245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:28:20.202245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:20.202245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:28:20.202245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:20.202245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:28:20.202245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:20.202245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:28:20.202245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:28:20.203097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:20.203097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:28:20.203097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:20.203097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:28:20.203097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:28:20.203097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:20.203097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:28:20.203097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:20.203097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:28:20.203770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:20.203770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:28:20.203770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:20.203770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:28:20.203770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:28:20.203770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:28:20.203770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:28:20.203770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:28:20.203770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:28:20.203770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:28:20.203770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:20.203770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:20.204509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:20.204509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:28:20.204509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:20.204509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:20.204509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:20.204509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:28:20.204509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:20.204509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:28:20.204509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:28:20.204509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:20.204509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:28:20.204509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:20.204509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:28:20.204509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:28:20.205081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:20.205081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:28:20.205081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:20.205081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:28:20.205746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:20.205746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:28:20.205746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:28:20.205746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:28:20.205746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:28:20.205746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:28:20.205746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:20.205746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:28:20.205746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:28:20.205746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:28:20.205746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:28:20.205746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:28:20.205746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:20.205746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:20.205746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:28:20.205746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:28:20.206200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:28:20.206200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:28:20.206200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:28:20.206200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:28:20.206200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:28:20.206200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:20.206200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:20.206200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:20.206200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:28:20.206200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:20.206200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:20.206200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:20.206200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:28:20.206200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:20.206200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:28:20.206200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:28:20.206200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:20.206200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:28:20.206701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:20.206701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:28:20.206701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:28:20.206701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:20.206701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:28:20.206701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:20.206701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:28:20.227129 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:28:20.227767 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:28:20.227959 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:28:20.228198 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:28:20.228766 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:28:20.229127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:20.229883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:28:20.229883 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:20.229883 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:28:20.229883 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:28:20.229883 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:20.229883 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:28:20.229883 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:20.229883 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:28:20.230917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:20.230917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:28:20.230917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:28:20.230917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:28:20.230917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:20.230917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:28:20.230917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:20.230917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:28:20.230917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:20.230917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:28:20.230917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:20.230917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:28:20.230917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:20.230917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:28:20.230917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:28:20.231582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:20.231582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:28:20.231582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:20.231582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:28:20.231582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:28:20.231582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:20.231582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:28:20.231582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:20.231582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:28:20.232049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:20.232049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:28:20.232049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:20.232049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:28:20.232049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:28:20.232049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:28:20.232049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:28:20.232049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:28:20.232049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:28:20.232049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:20.232049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:20.232604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:20.232604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:28:20.232604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:20.232604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:20.232604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:20.232604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:28:20.232604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:20.232604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:28:20.232604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:28:20.232604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:20.232604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:28:20.232604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:20.232604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:28:20.232604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:28:20.233227 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:20.233227 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:28:20.233227 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:20.233227 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:28:20.233227 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:20.233227 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:28:20.233227 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:28:20.233227 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:28:20.233227 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:28:20.233227 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:28:20.233227 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:20.233227 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:28:20.233227 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:28:20.233227 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:28:20.233963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:28:20.233963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:28:20.233963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:20.233963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:20.233963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:28:20.233963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:28:20.233963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:28:20.233963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:28:20.233963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:28:20.233963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:28:20.233963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:20.233963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:20.233963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:20.233963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:28:20.233963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:20.233963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:20.233963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:20.233963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:28:20.233963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:20.234874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:28:20.234874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:28:20.234874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:20.234874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:28:20.234874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:20.234874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:28:20.234874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:28:20.234874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:20.234874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:28:20.234874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:20.234874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:28:20.235624 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:28:20.235810 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:28:20.235906 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:28:20.236082 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:28:20.236546 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:28:20.236868 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:28:20.237081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:20.237799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:28:20.237799 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:20.237799 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:28:20.237799 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:28:20.237799 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:20.237799 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:28:20.237799 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:20.237799 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:28:20.238646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:20.238646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:28:20.238646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:20.238646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:28:20.238646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:28:20.238646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:28:20.238646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:28:20.238646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:20.238646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:20.238646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:20.238646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:28:20.238646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:20.239038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:20.239038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:20.239038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:28:20.239038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:20.239038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:28:20.239038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:28:20.239038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:20.239038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:28:20.239038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:20.239038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:28:20.239038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:28:20.239038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:20.239038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:28:20.239434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:20.239434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:28:20.239664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:20.239664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:28:20.239664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:28:20.239664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:28:20.239664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:28:20.239664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:28:20.239664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:20.239664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:28:20.239664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:28:20.239664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:28:20.239664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:28:20.239664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:28:20.239664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:20.239664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:20.239664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:28:20.239664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:28:20.240123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:28:20.240123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:28:20.240123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:20.240123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:20.240123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:20.240123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:28:20.240123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:20.240123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:20.240123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:20.240123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:28:20.240123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:20.240123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:28:20.240123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:28:20.240123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:20.240123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:28:20.240123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:20.240123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:28:20.240123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:28:20.240123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:20.240714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:28:20.240714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:20.240714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:28:24.982235 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:28:24.982890 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:28:24.982890 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:28:24.983491 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:28:24.984843 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG dbcounter [-] [105581] Writing DB stats neutron:SELECT=240 {{(pid=105581) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 14:28:25.020147 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:28:25.020581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:25.021502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:28:25.021502 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:25.021502 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:28:25.021502 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:28:25.021502 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:25.021502 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:28:25.021502 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:25.021502 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:28:25.022510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:25.022510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:28:25.022510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:25.022510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:28:25.022510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:28:25.022510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:28:25.022510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:28:25.022510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:25.022510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:25.022510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:25.022510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:28:25.022510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:25.022943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:25.022943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:25.022943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:28:25.022943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:25.022943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:28:25.022943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:28:25.022943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:25.022943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:28:25.022943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:25.022943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:28:25.022943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:28:25.022943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:25.022943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:28:25.023311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:25.023311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:28:25.023674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:25.023674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:28:25.023674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:28:25.023674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:28:25.023674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:28:25.023674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:28:25.023674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:25.023674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:28:25.023674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:28:25.023674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:28:25.023674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:28:25.023674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:28:25.023674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:25.023674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:25.023674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:28:25.023674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:28:25.024647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:28:25.024647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:28:25.024647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:25.024647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:25.024647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:25.024647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:28:25.024647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:25.024647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:25.024647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:25.024647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:28:25.024647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:25.024647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:28:25.024647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:28:25.024647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:25.024647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:28:25.024647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:25.024647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:28:25.024647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:28:25.024647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:25.025792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:28:25.025792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:25.025792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:28:25.192481 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:28:25.192841 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:28:25.193012 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:28:25.193872 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:28:25.194214 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:28:25.194525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:25.195435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:28:25.195435 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:25.195435 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:28:25.195435 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:28:25.195435 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:25.195435 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:28:25.195435 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:25.195435 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:28:25.196188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:25.196188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:28:25.196188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:25.196188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:28:25.196188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:28:25.196188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:28:25.196188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:28:25.196188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:25.196188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:25.196188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:25.196188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:28:25.196188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:25.196761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:25.196761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:25.196761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:28:25.196761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:25.196761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:28:25.196761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:28:25.196761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:25.196761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:28:25.196761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:25.196761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:28:25.196761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:28:25.196761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:25.196761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:28:25.197379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:25.197379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:28:25.197379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:25.197379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:28:25.197379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:28:25.197379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:28:25.197379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:28:25.197379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:28:25.197379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:25.197379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:28:25.197379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:28:25.197379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:28:25.197379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:28:25.197379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:28:25.197379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:25.198161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:25.198161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:28:25.198161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:28:25.198161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:28:25.198161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:28:25.198161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:25.198161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:25.198161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:25.198161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:28:25.198161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:25.198161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:25.198161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:25.198161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:28:25.198161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:25.198161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:28:25.198161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:28:25.198161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:25.198161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:28:25.198161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:25.199229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:28:25.199229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:28:25.199229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:25.199229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:28:25.199229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:25.199229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:28:25.199711 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:28:25.199954 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:28:25.200041 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:28:25.200155 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:28:25.200535 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:28:25.200689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:25.201167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:28:25.201167 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:25.201167 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:28:25.201167 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:28:25.201167 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:25.201167 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:28:25.201167 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:25.201167 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:28:25.201916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:25.201916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:28:25.201916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:28:25.201916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:28:25.201916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:25.201916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:28:25.201916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:25.201916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:28:25.201916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:25.201916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:28:25.201916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:25.201916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:28:25.201916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:25.201916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:28:25.201916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:28:25.202500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:25.202500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:28:25.202500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:25.202500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:28:25.202500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:28:25.202500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:25.202500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:28:25.202500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:25.202500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:28:25.202881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:25.202881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:28:25.202881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:25.202881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:28:25.202881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:28:25.202881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:28:25.202881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:28:25.202881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:28:25.202881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:28:25.202881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:28:25.202881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:25.202881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:25.203343 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:25.203343 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:28:25.203343 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:25.203343 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:25.203343 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:25.203343 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:28:25.203343 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:25.203343 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:28:25.203343 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:28:25.203343 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:25.203343 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:28:25.203343 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:25.203343 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:28:25.203343 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:28:25.203785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:25.203785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:28:25.203785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:25.203785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:28:25.203785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:25.203785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:28:25.203785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:28:25.203785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:28:25.203785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:28:25.203785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:28:25.203785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:25.203785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:28:25.203785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:28:25.203785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:28:25.204298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:28:25.204298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:28:25.204298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:25.204298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:25.204298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:28:25.204298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:28:25.204298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:28:25.204298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:28:25.204298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:28:25.204298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:28:25.204298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:28:25.204298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:25.204298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:25.204298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:25.204298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:28:25.204298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:25.204298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:25.204298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:25.204298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:28:25.204900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:25.204900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:28:25.204900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:28:25.204900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:25.204900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:28:25.204900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:25.204900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:28:25.204900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:28:25.204900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:25.204900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:28:25.204900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:25.204900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:28:25.227862 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:28:25.228179 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:28:25.228351 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:28:25.228567 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:28:25.229049 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:28:25.229423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:25.230134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:28:25.230134 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:25.230134 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:28:25.230134 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:28:25.230134 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:25.230134 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:28:25.230134 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:25.230134 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:28:25.231318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:25.231318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:28:25.231318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:28:25.231318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:28:25.231318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:25.231318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:28:25.231318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:25.231318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:28:25.231318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:25.231318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:28:25.231318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:25.231318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:28:25.231318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:25.231318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:28:25.231318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:28:25.232120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:25.232120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:28:25.232120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:25.232120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:28:25.232120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:28:25.232120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:25.232120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:28:25.232120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:25.232120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:28:25.232690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:25.232690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:28:25.232690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:25.232690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:28:25.232690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:28:25.232690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:28:25.232690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:28:25.232690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:28:25.232690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:28:25.232690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:25.232690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:25.233210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:25.233210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:28:25.233210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:25.233210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:25.233210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:25.233210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:28:25.233210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:25.233210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:28:25.233210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:28:25.233210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:25.233210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:28:25.233210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:25.233210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:28:25.233210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:28:25.233824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:25.233824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:28:25.233824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:25.233824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:28:25.233824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:25.233824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:28:25.233824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:28:25.233824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:28:25.233824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:28:25.233824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:28:25.233824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:25.233824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:28:25.233824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:28:25.233824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:28:25.234478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:28:25.234478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:28:25.234478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:25.234478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:25.234478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:28:25.234478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:28:25.234478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:28:25.234478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:28:25.234478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:28:25.234478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:28:25.234478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:25.234478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:25.234478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:25.234478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:28:25.234478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:25.234478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:25.234478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:25.234478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:28:25.234478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:25.235313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:28:25.235313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:28:25.235313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:25.235313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:28:25.235313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:25.235313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:28:25.235313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:28:25.235313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:25.235313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:28:25.235313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:25.235313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:28:25.236016 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:28:25.236328 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:28:25.236487 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:28:25.236684 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:28:25.237125 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:28:25.237531 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:28:25.237756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:25.238518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:28:25.238518 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:25.238518 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:28:25.238518 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:28:25.238518 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:25.238518 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:28:25.238518 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:25.238518 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:28:25.239501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:25.239501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:28:25.239501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:25.239501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:28:25.239501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:28:25.239501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:28:25.239501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:28:25.239501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:25.239501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:25.239501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:25.239501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:28:25.239501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:25.239842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:25.239842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:25.239842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:28:25.239842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:25.239842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:28:25.239842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:28:25.239842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:25.239842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:28:25.239842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:25.239842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:28:25.239842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:28:25.239842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:25.239842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:28:25.240162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:25.240162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:28:25.240492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:25.240492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:28:25.240492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:28:25.240492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:28:25.240492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:28:25.240492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:28:25.240492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:25.240492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:28:25.240492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:28:25.240492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:28:25.240492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:28:25.240492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:28:25.240492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:25.240492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:25.240492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:28:25.240492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:28:25.240919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:28:25.240919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:28:25.240919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:25.240919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:25.240919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:25.240919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:28:25.240919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:25.240919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:25.240919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:25.240919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:28:25.240919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:25.240919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:28:25.240919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:28:25.240919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:25.240919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:28:25.240919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:25.240919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:28:25.240919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:28:25.240919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:25.241388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:28:25.241388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:25.241388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:28:29.982826 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:28:29.983771 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:28:29.983771 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:28:29.984866 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:28:30.020809 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:28:30.021186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:30.022022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:28:30.022022 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:30.022022 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:28:30.022022 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:28:30.022022 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:30.022022 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:28:30.022022 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:30.022022 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:28:30.022770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:30.022770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:28:30.022770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:30.022770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:28:30.022770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:28:30.022770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:28:30.022770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:28:30.022770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:30.022770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:30.022770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:30.022770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:28:30.022770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:30.023528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:30.023528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:30.023528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:28:30.023528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:30.023528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:28:30.023528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:28:30.023528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:30.023528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:28:30.023528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:30.023528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:28:30.023528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:28:30.023528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:30.023528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:28:30.024136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:30.024136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:28:30.024136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:30.024136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:28:30.024136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:28:30.024136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:28:30.024136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:28:30.024136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:28:30.024136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:30.024136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:28:30.024136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:28:30.024136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:28:30.024136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:28:30.024136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:28:30.024136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:30.025057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:30.025057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:28:30.025057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:28:30.025057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:28:30.025057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:28:30.025057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:30.025057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:30.025057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:30.025057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:28:30.025057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:30.025057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:30.025057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:30.025057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:28:30.025057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:30.025057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:28:30.025057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:28:30.025057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:30.025057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:28:30.025057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:30.025981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:28:30.025981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:28:30.025981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:30.025981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:28:30.025981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:30.025981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:28:30.193160 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:28:30.193723 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:28:30.193957 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:28:30.194997 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:28:30.196105 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:28:30.196248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:30.197082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:28:30.197082 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:30.197082 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:28:30.197082 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:28:30.197082 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:30.197082 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:28:30.197082 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:30.197082 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:28:30.198742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:30.198742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:28:30.198742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:30.198742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:28:30.198742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:28:30.198742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:28:30.198742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:28:30.198742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:30.198742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:30.198742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:30.198742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:28:30.198742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:30.199332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:30.199332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:30.199332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:28:30.199332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:30.199332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:28:30.199332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:28:30.199332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:30.199332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:28:30.199332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:30.199332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:28:30.199332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:28:30.199332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:30.199332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:28:30.199896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:30.199896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:28:30.200446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:30.200446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:28:30.200446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:28:30.200446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:28:30.200446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:28:30.200446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:28:30.200446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:30.200446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:28:30.200446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:28:30.200446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:28:30.200446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:28:30.200446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:28:30.200446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:30.200446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:30.200446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:28:30.200446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:28:30.201176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:28:30.201176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:28:30.201176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:30.201176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:30.201176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:30.201176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:28:30.201176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:30.201176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:30.201176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:30.201176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:28:30.201176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:30.201176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:28:30.201176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:28:30.201176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:30.201176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:28:30.201176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:30.201176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:28:30.201176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:28:30.201176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:30.201993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:28:30.201993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:30.201993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:28:30.201993 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:28:30.201993 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:28:30.201993 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:28:30.202191 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:28:30.202753 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:28:30.202753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:30.203146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:28:30.203146 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:30.203146 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:28:30.203146 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:28:30.203146 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:30.203146 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:28:30.203146 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:30.203146 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:28:30.203948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:30.203948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:28:30.203948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:28:30.203948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:28:30.203948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:30.203948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:28:30.203948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:30.203948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:28:30.203948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:30.203948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:28:30.203948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:30.203948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:28:30.203948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:30.203948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:28:30.203948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:28:30.204842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:30.204842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:28:30.204842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:30.204842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:28:30.204842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:28:30.204842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:30.204842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:28:30.204842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:30.204842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:28:30.204842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:30.204842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:28:30.204842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:30.204842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:28:30.204842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:28:30.206649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:28:30.206649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:28:30.206649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:28:30.206649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:28:30.206649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:28:30.206649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:30.206649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:30.206649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:30.206649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:28:30.206649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:30.206649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:30.206649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:30.206649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:28:30.207540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:30.207540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:28:30.207540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:28:30.207540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:30.207540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:28:30.207540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:30.207540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:28:30.207540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:28:30.207540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:30.207540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:28:30.207540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:30.207540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:28:30.208185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:30.208185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:28:30.208185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:28:30.208185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:28:30.208185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:28:30.208185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:28:30.208185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:30.208185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:28:30.208185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:28:30.208185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:28:30.208185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:28:30.208185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:28:30.208185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:30.208185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:30.208185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:28:30.208185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:28:30.208683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:28:30.208683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:28:30.208683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:28:30.208683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:28:30.208683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:28:30.208683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:30.208683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:30.208683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:30.208683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:28:30.208683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:30.208683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:30.208683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:30.208683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:28:30.208683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:30.208683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:28:30.208683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:28:30.208683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:30.208683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:28:30.209139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:30.209139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:28:30.209139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:28:30.209139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:30.209139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:28:30.209139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:30.209139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:28:30.228276 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:28:30.228614 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:28:30.228736 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:28:30.228940 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:28:30.229338 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:28:30.229493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:30.230025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:28:30.230025 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:30.230025 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:28:30.230025 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:28:30.230025 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:30.230025 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:28:30.230025 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:30.230025 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:28:30.231275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:30.231275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:28:30.231275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:28:30.231275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:28:30.231275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:30.231275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:28:30.231275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:30.231275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:28:30.231275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:30.231275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:28:30.231275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:30.231275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:28:30.231275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:30.231275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:28:30.231275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:28:30.231806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:30.231806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:28:30.231806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:30.231806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:28:30.231806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:28:30.231806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:30.231806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:28:30.231806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:30.231806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:28:30.232683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:30.232683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:28:30.232683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:30.232683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:28:30.232683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:28:30.232683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:28:30.232683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:28:30.232683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:28:30.232683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:28:30.232683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:30.232683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:30.233185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:30.233185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:28:30.233185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:30.233185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:30.233185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:30.233185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:28:30.233185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:30.233185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:28:30.233185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:28:30.233185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:30.233185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:28:30.233185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:30.233185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:28:30.233185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:28:30.233758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:30.233758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:28:30.233758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:30.233758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:28:30.234781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:30.234781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:28:30.234781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:28:30.234781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:28:30.234781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:28:30.234781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:28:30.234781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:30.234781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:28:30.234781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:28:30.234781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:28:30.234781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:28:30.234781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:28:30.234781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:30.234781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:30.234781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:28:30.234781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:28:30.234781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:28:30.235302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:28:30.235302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:28:30.235302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:28:30.235302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:30.235302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:30.235302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:30.235302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:28:30.235302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:30.235302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:30.235302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:30.235302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:28:30.235302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:30.235302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:28:30.235302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:28:30.235302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:30.235302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:28:30.235302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:30.235302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:28:30.235302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:28:30.235896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:30.235896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:28:30.235896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:30.235896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:28:30.236477 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:28:30.236837 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:28:30.237036 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:28:30.237301 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:28:30.237805 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:28:30.238186 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:28:30.238417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:30.239103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:28:30.239103 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:30.239103 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:28:30.239103 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:28:30.239103 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:30.239103 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:28:30.239103 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:30.239103 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:28:30.239942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:30.239942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:28:30.239942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:30.239942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:28:30.239942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:28:30.239942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:28:30.239942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:28:30.239942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:30.239942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:30.239942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:30.239942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:28:30.239942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:30.240319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:30.240319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:30.240319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:28:30.240319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:30.240319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:28:30.240319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:28:30.240319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:30.240319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:28:30.240319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:30.240319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:28:30.240319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:28:30.240319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:30.240319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:28:30.240699 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:30.240699 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:28:30.240984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:30.240984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:28:30.240984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:28:30.240984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:28:30.240984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:28:30.240984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:28:30.240984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:30.240984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:28:30.240984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:28:30.240984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:28:30.240984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:28:30.240984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:28:30.240984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:30.240984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:30.240984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:28:30.240984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:28:30.241464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:28:30.241464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:28:30.241464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:30.241464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:30.241464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:30.241464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:28:30.241464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:30.241464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:30.241464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:30.241464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:28:30.241464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:30.241464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:28:30.241464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:28:30.241464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:30.241464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:28:30.241464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:30.241464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:28:30.241464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:28:30.241464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:30.241988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:28:30.241988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:30.241988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:28:34.983674 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:28:34.985091 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:28:34.985213 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:28:34.986069 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:28:35.026025 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:28:35.026490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:35.027002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:28:35.027002 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:35.027002 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:28:35.027002 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:28:35.027002 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:35.027002 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:28:35.027002 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:35.027002 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:28:35.028044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:35.028044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:28:35.028044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:35.028044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:28:35.028044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:28:35.028044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:28:35.028044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:28:35.028044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:35.028044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:35.028044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:35.028044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:28:35.028044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:35.028721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:35.028721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:35.028721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:28:35.028721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:35.028721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:28:35.028721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:28:35.028721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:35.028721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:28:35.028721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:35.028721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:28:35.028721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:28:35.028721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:35.028721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:28:35.029449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:35.029449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:28:35.029449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:35.029449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:28:35.029449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:28:35.029449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:28:35.029449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:28:35.029449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:28:35.029449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:35.029449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:28:35.029449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:28:35.029449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:28:35.029449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:28:35.029449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:28:35.029449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:35.030188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:35.030188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:28:35.030188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:28:35.030188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:28:35.030188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:28:35.030188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:35.030188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:35.030188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:35.030188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:28:35.030188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:35.030188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:35.030188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:35.030188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:28:35.030188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:35.030188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:28:35.030188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:28:35.030188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:35.030188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:28:35.030188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:35.031198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:28:35.031198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:28:35.031198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:35.031198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:28:35.031198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:35.031198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:28:35.194083 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:28:35.194783 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:28:35.194923 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:28:35.196248 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:28:35.196632 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:28:35.196854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:35.197732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:28:35.197732 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:35.197732 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:28:35.197732 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:28:35.197732 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:35.197732 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:28:35.197732 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:35.197732 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:28:35.199276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:35.199276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:28:35.199276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:35.199276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:28:35.199276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:28:35.199276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:28:35.199276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:28:35.199276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:35.199276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:35.199276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:35.199276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:28:35.199276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:35.200078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:35.200078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:35.200078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:28:35.200078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:35.200078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:28:35.200078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:28:35.200078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:35.200078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:28:35.200078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:35.200078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:28:35.200078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:28:35.200078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:35.200078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:28:35.201019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:35.201019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:28:35.201174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:35.201174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:28:35.201174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:28:35.201174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:28:35.201174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:28:35.201174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:28:35.201174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:35.201174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:28:35.201174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:28:35.201174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:28:35.201174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:28:35.201174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:28:35.201174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:35.201174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:35.201174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:28:35.201174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:28:35.202066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:28:35.202066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:28:35.202066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:35.202066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:35.202066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:35.202066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:28:35.202066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:35.202066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:35.202066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:35.202066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:28:35.202066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:35.202066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:28:35.202066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:28:35.202066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:35.202066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:28:35.202066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:35.202066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:28:35.202066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:28:35.202066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:35.203274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:28:35.203274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:35.203274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:28:35.203274 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:28:35.203274 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:28:35.203274 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:28:35.203274 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:28:35.203274 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:28:35.203953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:35.203953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:28:35.203953 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:35.203953 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:28:35.203953 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:28:35.203953 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:35.203953 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:28:35.203953 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:35.203953 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:28:35.204618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:35.204618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:28:35.204618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:28:35.204618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:28:35.204618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:35.204618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:28:35.204618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:35.204618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:28:35.204618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:35.204618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:28:35.204618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:35.204618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:28:35.204618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:35.204618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:28:35.204618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:28:35.205290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:35.205290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:28:35.205290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:35.205290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:28:35.205290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:28:35.205290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:35.205290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:28:35.205290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:35.205290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:28:35.205290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:35.205290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:28:35.205290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:35.205290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:28:35.205290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:28:35.205888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:28:35.205888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:28:35.205888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:28:35.205888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:28:35.205888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:28:35.205888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:35.205888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:35.205888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:35.205888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:28:35.205888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:35.205888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:35.205888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:35.205888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:28:35.206364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:35.206364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:28:35.206364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:28:35.206364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:35.206364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:28:35.206364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:35.206364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:28:35.206364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:28:35.206364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:35.206364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:28:35.206364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:35.206364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:28:35.206872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:35.206872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:28:35.206872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:28:35.206872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:28:35.206872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:28:35.206872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:28:35.206872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:35.206872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:28:35.206872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:28:35.206872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:28:35.206872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:28:35.206872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:28:35.206872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:35.206872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:35.206872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:28:35.206872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:28:35.207513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:28:35.207513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:28:35.207513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:28:35.207513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:28:35.207513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:28:35.207513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:35.207513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:35.207513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:35.207513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:28:35.207513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:35.207513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:35.207513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:35.207513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:28:35.207513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:35.207513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:28:35.207513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:28:35.207513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:35.207513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:28:35.208207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:35.208207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:28:35.208207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:28:35.208207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:35.208207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:28:35.208207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:35.208207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:28:35.228803 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:28:35.229425 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:28:35.229637 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:28:35.229851 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:28:35.230208 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:28:35.230429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:35.230991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:28:35.230991 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:35.230991 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:28:35.230991 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:28:35.230991 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:35.230991 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:28:35.230991 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:35.230991 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:28:35.232199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:35.232199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:28:35.232199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:28:35.232199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:28:35.232199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:35.232199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:28:35.232199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:35.232199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:28:35.232199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:35.232199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:28:35.232199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:35.232199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:28:35.232199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:35.232199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:28:35.232199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:28:35.232628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:35.232628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:28:35.232628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:35.232628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:28:35.232628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:28:35.232628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:35.232628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:28:35.232628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:35.232628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:28:35.233798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:35.233798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:28:35.233798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:35.233798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:28:35.233798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:28:35.233798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:28:35.233798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:28:35.233798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:28:35.233798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:28:35.233798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:35.233798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:35.234166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:35.234166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:28:35.234166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:35.234166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:35.234166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:35.234166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:28:35.234166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:35.234166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:28:35.234166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:28:35.234166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:35.234166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:28:35.234166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:35.234166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:28:35.234166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:28:35.234556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:35.234556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:28:35.234556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:35.234556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:28:35.235038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:35.235038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:28:35.235038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:28:35.235038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:28:35.235038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:28:35.235038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:28:35.235038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:35.235038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:28:35.235038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:28:35.235038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:28:35.235038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:28:35.235038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:28:35.235038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:35.235038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:35.235038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:28:35.235038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:28:35.235038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:28:35.235933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:28:35.235933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:28:35.235933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:28:35.235933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:35.235933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:35.235933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:35.235933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:28:35.235933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:35.235933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:35.235933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:35.235933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:28:35.235933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:35.235933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:28:35.235933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:28:35.235933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:35.235933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:28:35.235933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:35.235933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:28:35.235933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:28:35.236788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:35.236788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:28:35.236788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:35.236788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:28:35.237004 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:28:35.237315 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:28:35.237418 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:28:35.237581 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:28:35.237881 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:28:35.238094 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:28:35.238250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:35.238704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:28:35.238704 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:35.238704 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:28:35.238704 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:28:35.238704 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:35.238704 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:28:35.238704 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:35.238704 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:28:35.239506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:35.239506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:28:35.239506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:35.239506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:28:35.239506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:28:35.239506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:28:35.239506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:28:35.239506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:35.239506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:35.239506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:35.239506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:28:35.239506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:35.239960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:35.239960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:35.239960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:28:35.239960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:35.239960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:28:35.239960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:28:35.239960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:35.239960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:28:35.239960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:35.239960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:28:35.239960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:28:35.239960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:35.239960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:28:35.240509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:35.240509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:28:35.240509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:35.240509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:28:35.240509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:28:35.240509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:28:35.240509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:28:35.240509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:28:35.240509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:35.240509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:28:35.240509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:28:35.240509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:28:35.240509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:28:35.240509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:28:35.240509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:35.241090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:35.241090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:28:35.241090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:28:35.241090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:28:35.241090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:28:35.241090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:35.241090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:35.241090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:35.241090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:28:35.241090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:35.241090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:35.241090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:35.241090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:28:35.241090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:35.241090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:28:35.241090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:28:35.241090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:35.241090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:28:35.241090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:35.241880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:28:35.241880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:28:35.241880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:35.241880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:28:35.241880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:35.241880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:28:39.985193 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:28:39.986275 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:28:39.986275 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:28:39.987598 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:28:40.035092 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:28:40.035559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:40.036611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:28:40.036611 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:40.036611 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:28:40.036611 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:28:40.036611 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:40.036611 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:28:40.036611 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:40.036611 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:28:40.037587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.050 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:40.037587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:28:40.037587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:40.037587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:28:40.037587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:28:40.037587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:28:40.037587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:28:40.037587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:40.037587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:40.037587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:40.037587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:28:40.037587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:40.038226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:40.038226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:40.038226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:28:40.038226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:40.038226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:28:40.038226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:28:40.038226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:40.038226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:28:40.038226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:40.038226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:28:40.038226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:28:40.038226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:40.038226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:28:40.038883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:40.038883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:28:40.038883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:40.038883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:28:40.038883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:28:40.038883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:28:40.038883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:28:40.038883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:28:40.038883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:40.038883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:28:40.038883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:28:40.038883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:28:40.038883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:28:40.038883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:28:40.038883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:40.039638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:40.039638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:28:40.039638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:28:40.039638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:28:40.039638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:28:40.039638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:40.039638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:40.039638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:40.039638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:28:40.039638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:40.039638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:40.039638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:40.039638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:28:40.039638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:40.039638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:28:40.039638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:28:40.039638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:40.039638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:28:40.039638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:40.041448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:28:40.041448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:28:40.041448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:40.041448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:28:40.041448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:40.041448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:28:40.194933 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:28:40.195597 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:28:40.195597 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:28:40.196733 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:28:40.197105 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:28:40.197391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:40.198383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:28:40.198383 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:40.198383 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:28:40.198383 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:28:40.198383 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:40.198383 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:28:40.198383 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:40.198383 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:28:40.199983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:40.199983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:28:40.199983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:40.199983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:28:40.199983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:28:40.199983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:28:40.199983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:28:40.199983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:40.199983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:40.199983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:40.199983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:28:40.199983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:40.200741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:40.200741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:40.200741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:28:40.200741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:40.200741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:28:40.200741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:28:40.200741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:40.200741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:28:40.200741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:40.200741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:28:40.200741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:28:40.200741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:40.200741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:28:40.201612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:40.201612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:28:40.201794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:40.201794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:28:40.201794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:28:40.201794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:28:40.201794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:28:40.201794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:28:40.201794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:40.201794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:28:40.201794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:28:40.201794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:28:40.201794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:28:40.201794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:28:40.201794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:40.201794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:40.201794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:28:40.201794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:28:40.202808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:28:40.202808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:28:40.202808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:40.202808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:40.202808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:40.202808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:28:40.202808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:40.202808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:40.202808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:40.202808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:28:40.202808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:40.202808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:28:40.202808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:28:40.202808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:40.202808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:28:40.202808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:40.202808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:28:40.202808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:28:40.202808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:40.203726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:28:40.203726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:40.203726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:28:40.203726 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:28:40.203726 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:28:40.203726 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:28:40.203726 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:28:40.203726 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:28:40.203988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:40.203988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:28:40.203988 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:40.203988 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:28:40.203988 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:28:40.203988 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:40.203988 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:28:40.203988 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:40.203988 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:28:40.204666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:40.204666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:28:40.204666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:28:40.204666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:28:40.204666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:40.204666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:28:40.204666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:40.204666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:28:40.204666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:40.204666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:28:40.204666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:40.204666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:28:40.204666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:40.204666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:28:40.204666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:28:40.205140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:40.205140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:28:40.205140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:40.205140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:28:40.205140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:28:40.205140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:40.205140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:28:40.205140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:40.205140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:28:40.205518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:40.205518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:28:40.205518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:40.205518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:28:40.205518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:28:40.205518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:28:40.205518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:28:40.205518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:28:40.205518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:28:40.205518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:28:40.205518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:40.205518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:40.205862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:40.205862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:28:40.205862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:40.205862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:40.205862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:40.205862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:28:40.205862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:40.205862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:28:40.205862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:28:40.205862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:40.205862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:28:40.205862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:40.205862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:28:40.205862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:28:40.206305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:40.206305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:28:40.206305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:40.206305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:28:40.206498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:40.206498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:28:40.206498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:28:40.206498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:28:40.206498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:28:40.206498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:28:40.206498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:40.206498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:28:40.206498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:28:40.206498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:28:40.206498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:28:40.206498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:28:40.206498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:40.206498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:40.206498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:28:40.206498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:28:40.207080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:28:40.207080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:28:40.207080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:28:40.207080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:28:40.207080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:28:40.207080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:40.207080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:40.207080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:40.207080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:28:40.207080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:40.207080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:40.207080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:40.207080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:28:40.207080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:40.207080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:28:40.207080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:28:40.207080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:40.207080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:28:40.207696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:40.207696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:28:40.207696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:28:40.207696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:40.207696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:28:40.207696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:40.207696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:28:40.229751 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:28:40.230027 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:28:40.230192 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:28:40.230376 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:28:40.230898 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:28:40.231280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:40.232104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:28:40.232104 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:40.232104 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:28:40.232104 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:28:40.232104 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:40.232104 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:28:40.232104 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:40.232104 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:28:40.233254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:40.233254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:28:40.233254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:28:40.233254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:28:40.233254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:40.233254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:28:40.233254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:40.233254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:28:40.233254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:40.233254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:28:40.233254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:40.233254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:28:40.233254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:40.233254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:28:40.233254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:28:40.234304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:40.234304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:28:40.234304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:40.234304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:28:40.234304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:28:40.234304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:40.234304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:28:40.234304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:40.234304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:28:40.234962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:40.234962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:28:40.234962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:40.234962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:28:40.234962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:28:40.234962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:28:40.234962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:28:40.234962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:28:40.234962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:28:40.234962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:40.234962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:40.235741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:40.235741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:28:40.235741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:40.235741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:40.235741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:40.235741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:28:40.235741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:40.235741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:28:40.235741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:28:40.235741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:40.235741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:28:40.235741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:40.235741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:28:40.235741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:28:40.236311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:40.236311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:28:40.236311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:40.236311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:28:40.236311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:40.236311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:28:40.236311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:28:40.236311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:28:40.236311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:28:40.236311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:28:40.236311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:40.236311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:28:40.236311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:28:40.236311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:28:40.236731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:28:40.236731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:28:40.236731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:40.236731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:40.236731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:28:40.236731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:28:40.236731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:28:40.236731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:28:40.236731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:28:40.236731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:28:40.236731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:40.236731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:40.236731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:40.236731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:28:40.236731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:40.236731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:40.236731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:40.236731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:28:40.236731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:40.237227 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:28:40.237227 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:28:40.237227 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:40.237227 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:28:40.237227 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:40.237227 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:28:40.237227 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:28:40.237227 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:40.237227 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:28:40.237227 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:40.237227 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:28:40.237549 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:28:40.237741 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:28:40.237914 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:28:40.238148 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:28:40.238631 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:28:40.238985 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:28:40.239220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:40.240033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:28:40.240033 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:40.240033 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:28:40.240033 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:28:40.240033 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:40.240033 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:28:40.240033 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:40.240033 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:28:40.240877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:40.240877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:28:40.240877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:40.240877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:28:40.240877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:28:40.240877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:28:40.240877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:28:40.240877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:40.240877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:40.240877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:40.240877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:28:40.240877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:40.241253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:40.241253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:40.241253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:28:40.241253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:40.241253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:28:40.241253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:28:40.241253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:40.241253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:28:40.241253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:40.241253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:28:40.241253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:28:40.241253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:40.241253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:28:40.241634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:40.241634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:28:40.241943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:40.241943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:28:40.241943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:28:40.241943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:28:40.241943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:28:40.241943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:28:40.241943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:40.241943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:28:40.241943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:28:40.241943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:28:40.241943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:28:40.241943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:28:40.241943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:40.241943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:40.241943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:28:40.241943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:28:40.242481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:28:40.242481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:28:40.242481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:40.242481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:40.242481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:40.242481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:28:40.242481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:40.242481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:40.242481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:40.242481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:28:40.242481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:40.242481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:28:40.242481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:28:40.242481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:40.242481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:28:40.242481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:40.242481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:28:40.242481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:28:40.242481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:40.242942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:28:40.242942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:40.242942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:28:44.986255 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:28:44.987272 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:28:44.987272 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:28:44.987981 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:28:45.024020 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:28:45.024434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:45.025262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:28:45.025262 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:45.025262 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:28:45.025262 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:28:45.025262 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:45.025262 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:28:45.025262 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:45.025262 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:28:45.026297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:45.026297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:28:45.026297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:45.026297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:28:45.026297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:28:45.026297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:28:45.026297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:28:45.026297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:45.026297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:45.026297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:45.026297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:28:45.026297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:45.027117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:45.027117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:45.027117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:28:45.027117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:45.027117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:28:45.027117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:28:45.027117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:45.027117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:28:45.027117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:45.027117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:28:45.027117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:28:45.027117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:45.027117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:28:45.027840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:45.027840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:28:45.027840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:45.027840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:28:45.027840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:28:45.027840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:28:45.027840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:28:45.027840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:28:45.027840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:45.027840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:28:45.027840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:28:45.027840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:28:45.027840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:28:45.027840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:28:45.027840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:45.028633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:45.028633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:28:45.028633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:28:45.028633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:28:45.028633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:28:45.028633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:45.028633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:45.028633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:45.028633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:28:45.028633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:45.028633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:45.028633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:45.028633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:28:45.028633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:45.028633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:28:45.028633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:28:45.028633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:45.028633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:28:45.028633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:45.029665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:28:45.029665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:28:45.029665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:45.029665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:28:45.029665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:45.029665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:28:45.195642 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:28:45.195866 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:28:45.196062 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:28:45.197159 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:28:45.197544 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:28:45.197792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:45.198671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:28:45.198671 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:45.198671 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:28:45.198671 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:28:45.198671 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:45.198671 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:28:45.198671 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:45.198671 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:28:45.200127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:45.200127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:28:45.200127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:45.200127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:28:45.200127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:28:45.200127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:28:45.200127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:28:45.200127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:45.200127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:45.200127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:45.200127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:28:45.200127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:45.200729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:45.200729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:45.200729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:28:45.200729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:45.200729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:28:45.200729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:28:45.200729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:45.200729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:28:45.200729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:45.200729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:28:45.200729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:28:45.200729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:45.200729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:28:45.201297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:45.201297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:28:45.201710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:45.201710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:28:45.201710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:28:45.201710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:28:45.201710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:28:45.201710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:28:45.201710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:45.201710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:28:45.201710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:28:45.201710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:28:45.201710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:28:45.201710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:28:45.201710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:45.201710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:45.201710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:28:45.201710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:28:45.202389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:28:45.202389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:28:45.202389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:45.202389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:45.202389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:45.202389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:28:45.202389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:45.202389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:45.202389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:45.202389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:28:45.202389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:45.202389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:28:45.202389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:28:45.202389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:45.202389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:28:45.202389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:45.202389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:28:45.202389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:28:45.202389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:45.203194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:28:45.203194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:45.203194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:28:45.203194 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:28:45.203194 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:28:45.203194 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:28:45.203194 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:28:45.203612 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:28:45.203770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:45.204613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:28:45.204613 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:45.204613 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:28:45.204613 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:28:45.204613 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:45.204613 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:28:45.204613 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:45.204613 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:28:45.205926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:45.205926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:28:45.205926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:28:45.205926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:28:45.205926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:45.205926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:28:45.205926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:45.205926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:28:45.205926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:45.205926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:28:45.205926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:45.205926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:28:45.205926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:45.205926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:28:45.205926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:28:45.206696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:45.206696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:28:45.206696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:45.206696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:28:45.206696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:28:45.206696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:45.206696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:28:45.206696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:45.206696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:28:45.207339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:45.207339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:28:45.207339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:45.207339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:28:45.207339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:28:45.207339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:28:45.207339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:28:45.207339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:28:45.207339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:28:45.207339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:28:45.207339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:45.207339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:45.207951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:45.207951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:28:45.207951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:45.207951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:45.207951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:45.207951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:28:45.207951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:45.207951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:28:45.207951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:28:45.207951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:45.207951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:28:45.207951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:45.207951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:28:45.207951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:28:45.208609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:45.208609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:28:45.208609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:45.208609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:28:45.209063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:45.209063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:28:45.209063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:28:45.209063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:28:45.209063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:28:45.209063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:28:45.209063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:45.209063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:28:45.209063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:28:45.209063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:28:45.209063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:28:45.209063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:28:45.209063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:45.209063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:45.209063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:28:45.209063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:28:45.209613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:28:45.209613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:28:45.209613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:28:45.209613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:28:45.209613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:28:45.209613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:45.209613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:45.209613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:45.209613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:28:45.209613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:45.209613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:45.209613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:45.209613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:28:45.209613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:45.209613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:28:45.209613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:28:45.209613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:45.209613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:28:45.210128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:45.210128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:28:45.210128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:28:45.210128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:45.210128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:28:45.210128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:45.210128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:28:45.230187 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:28:45.230548 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:28:45.230731 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:28:45.230930 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:28:45.231420 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:28:45.231810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:45.232634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:28:45.232634 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:45.232634 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:28:45.232634 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:28:45.232634 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:45.232634 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:28:45.232634 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:45.232634 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:28:45.233923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:45.233923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:28:45.233923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:28:45.233923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:28:45.233923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:45.233923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:28:45.233923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:45.233923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:28:45.233923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:45.233923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:28:45.233923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:45.233923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:28:45.233923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:45.233923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:28:45.233923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:28:45.234597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:45.234597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:28:45.234597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:45.234597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:28:45.234597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:28:45.234597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:45.234597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:28:45.234597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:45.234597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:28:45.235394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:45.235394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:28:45.235394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:45.235394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:28:45.235394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:28:45.235394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:28:45.235394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:28:45.235394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:28:45.235394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:28:45.235394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:45.235394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:45.235962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:45.235962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:28:45.235962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:45.235962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:45.235962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:45.235962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:28:45.235962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:45.235962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:28:45.235962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:28:45.235962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:45.235962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:28:45.235962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:45.235962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:28:45.235962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:28:45.236718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:45.236718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:28:45.236718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:45.236718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:28:45.236929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:45.236929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:28:45.236929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:28:45.236929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:28:45.236929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:28:45.236929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:28:45.236929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:45.236929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:28:45.236929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:28:45.236929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:28:45.236929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:28:45.236929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:28:45.236929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:45.236929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:45.236929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:28:45.236929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:28:45.236929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:28:45.237788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:28:45.237788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:28:45.237788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:28:45.237788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:45.237788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:45.237788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:45.237788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:28:45.237788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:45.237788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:45.237788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:45.237788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:28:45.237788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:45.237788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:28:45.237788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:28:45.237788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:45.237788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:28:45.237788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:45.237788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:28:45.237788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:28:45.238692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:45.238692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:28:45.238692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:45.238692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:28:45.238692 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:28:45.238692 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:28:45.238692 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:28:45.238692 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:28:45.239129 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:28:45.239129 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:28:45.239129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:45.239883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:28:45.239883 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:45.239883 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:28:45.239883 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:28:45.239883 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:45.239883 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:28:45.239883 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:45.239883 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:28:45.240686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:45.240686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:28:45.240686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:45.240686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:28:45.240686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:28:45.240686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:28:45.240686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:28:45.240686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:45.240686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:45.240686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:45.240686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:28:45.240686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:45.241038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:45.241038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:45.241038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:28:45.241038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:45.241038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:28:45.241038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:28:45.241038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:45.241038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:28:45.241038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:45.241038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:28:45.241038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:28:45.241038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:45.241038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:28:45.241393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:45.241393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:28:45.241614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:45.241614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:28:45.241614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:28:45.241614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:28:45.241614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:28:45.241614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:28:45.241614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:45.241614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:28:45.241614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:28:45.241614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:28:45.241614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:28:45.241614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:28:45.241614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:45.241614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:45.241614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:28:45.241614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:28:45.242026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:28:45.242026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:28:45.242026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:45.242026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:45.242026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:45.242026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:28:45.242026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:45.242026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:45.242026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:45.242026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:28:45.242026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:45.242026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:28:45.242026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:28:45.242026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:45.242026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:28:45.242026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:45.242026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:28:45.242026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:28:45.242026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:45.242535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:28:45.242535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:45.242535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:28:49.986940 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:28:49.988121 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:28:49.988121 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:28:49.988715 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:28:50.025526 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:28:50.025942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:50.026817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:28:50.026817 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:50.026817 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:28:50.026817 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:28:50.026817 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:50.026817 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:28:50.026817 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:50.026817 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:28:50.027825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:50.027825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:28:50.027825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:50.027825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:28:50.027825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:28:50.027825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:28:50.027825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:28:50.027825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:50.027825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:50.027825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:50.027825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:28:50.027825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:50.028271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:50.028271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:50.028271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:28:50.028271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:50.028271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:28:50.028271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:28:50.028271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:50.028271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:28:50.028271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:50.028271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:28:50.028271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:28:50.028271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:50.028271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:28:50.028702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:50.028702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:28:50.028811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:50.028811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:28:50.028811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:28:50.028811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:28:50.028811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:28:50.028811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:28:50.028811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:50.028811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:28:50.028811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:28:50.028811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:28:50.028811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:28:50.028811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:28:50.028811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:50.028811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:50.028811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:28:50.028811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:28:50.029307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:28:50.029307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:28:50.029307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:50.029307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:50.029307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:50.029307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:28:50.029307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:50.029307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:50.029307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:50.029307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:28:50.029307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:50.029307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:28:50.029307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:28:50.029307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:50.029307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:28:50.029307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:50.029307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:28:50.029307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:28:50.029307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:50.029902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:28:50.029902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:50.029902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:28:50.196196 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:28:50.196653 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:28:50.196818 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:28:50.198059 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:28:50.198448 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:28:50.198733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:50.199675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:28:50.199675 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:50.199675 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:28:50.199675 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:28:50.199675 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:50.199675 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:28:50.199675 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:50.199675 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:28:50.201128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:50.201128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:28:50.201128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:50.201128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:28:50.201128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:28:50.201128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:28:50.201128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:28:50.201128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:50.201128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:50.201128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:50.201128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:28:50.201128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:50.201749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:50.201749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:50.201749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:28:50.201749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:50.201749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:28:50.201749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:28:50.201749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:50.201749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:28:50.201749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:50.201749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:28:50.201749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:28:50.201749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:50.201749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:28:50.202285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:50.202285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:28:50.202748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:50.202748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:28:50.202748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:28:50.202748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:28:50.202748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:28:50.202748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:28:50.202748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:50.202748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:28:50.202748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:28:50.202748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:28:50.202748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:28:50.202748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:28:50.202748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:50.202748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:50.202748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:28:50.202748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:28:50.203473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:28:50.203473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:28:50.203473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:50.203473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:50.203473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:50.203473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:28:50.203473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:50.203473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:50.203473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:50.203473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:28:50.203473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:50.203473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:28:50.203473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:28:50.203473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:50.203473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:28:50.203473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:50.203473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:28:50.203473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:28:50.203473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:50.204484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:28:50.204484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:50.204484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:28:50.204484 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:28:50.204484 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:28:50.204484 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:28:50.204484 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:28:50.204484 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:28:50.204886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:50.205038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:28:50.205038 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:50.205038 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:28:50.205038 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:28:50.205038 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:50.205038 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:28:50.205038 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:50.205038 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:28:50.205905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:50.205905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:28:50.205905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:28:50.205905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:28:50.205905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:50.205905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:28:50.205905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:50.205905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:28:50.205905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:50.205905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:28:50.205905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:50.205905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:28:50.205905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:50.205905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:28:50.205905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:28:50.206362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:50.206362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:28:50.206362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:50.206362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:28:50.206362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:28:50.206362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:50.206362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:28:50.206362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:50.206362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:28:50.206719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:50.206719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:28:50.206719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:50.206719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:28:50.206719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:28:50.206719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:28:50.206719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:28:50.206719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:28:50.206719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:28:50.206719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:28:50.206719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:50.206719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:50.207078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:50.207078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:28:50.207078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:50.207078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:50.207078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:50.207078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:28:50.207078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:50.207078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:28:50.207078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:28:50.207078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:50.207078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:28:50.207078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:50.207078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:28:50.207078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:28:50.207463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:50.207463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:28:50.207463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:50.207463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:28:50.207673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:50.207673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:28:50.207673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:28:50.207673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:28:50.207673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:28:50.207673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:28:50.207673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:50.207673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:28:50.207673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:28:50.207673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:28:50.207673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:28:50.207673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:28:50.207673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:50.207673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:50.207673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:28:50.207673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:28:50.208114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:28:50.208114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:28:50.208114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:28:50.208114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:28:50.208114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:28:50.208114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:50.208114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:50.208114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:50.208114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:28:50.208114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:50.208114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:50.208114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:50.208114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:28:50.208114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:50.208114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:28:50.208114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:28:50.208114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:50.208114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:28:50.208670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:50.208670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:28:50.208670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:28:50.208670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:50.208670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:28:50.208670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:50.208670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:28:50.230640 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:28:50.230927 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:28:50.231089 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:28:50.231275 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:28:50.232815 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:28:50.232985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:50.233475 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:28:50.233475 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:50.233475 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:28:50.233475 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:28:50.233475 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:50.233475 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:28:50.233475 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:50.233475 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:28:50.234629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:50.234629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:28:50.234629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:28:50.234629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:28:50.234629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:50.234629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:28:50.234629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:50.234629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:28:50.234629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:50.234629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:28:50.234629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:50.234629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:28:50.234629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:50.234629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:28:50.234629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:28:50.235278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:50.235278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:28:50.235278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:50.235278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:28:50.235278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:28:50.235278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:50.235278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:28:50.235278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:50.235278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:28:50.236005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:50.236005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:28:50.236005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:50.236005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:28:50.236005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:28:50.236005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:28:50.236005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:28:50.236005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:28:50.236005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:28:50.236005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:50.236005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:50.236578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:50.236578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:28:50.236578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:50.236578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:50.236578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:50.236578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:28:50.236578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:50.236578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:28:50.236578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:28:50.236578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:50.236578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:28:50.236578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:50.236578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:28:50.236578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:28:50.237212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:50.237212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:28:50.237212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:50.237212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:28:50.237424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:50.237424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:28:50.237424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:28:50.237424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:28:50.237424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:28:50.237424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:28:50.237424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:50.237424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:28:50.237424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:28:50.237424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:28:50.237424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:28:50.237424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:28:50.237424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:50.237424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:50.237424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:28:50.237424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:28:50.237424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:28:50.238154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:28:50.238154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:28:50.238154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:28:50.238154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:50.238154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:50.238154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:50.238154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:28:50.238154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:50.238154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:50.238154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:50.238154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:28:50.238154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:50.238154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:28:50.238154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:28:50.238154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:50.238154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:28:50.238154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:50.238154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:28:50.238154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:28:50.238969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:50.238969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:28:50.238969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:50.238969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:28:50.238969 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:28:50.238969 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:28:50.238969 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:28:50.238969 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:28:50.239343 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:28:50.239550 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:28:50.239785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:50.240366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:28:50.240366 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:50.240366 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:28:50.240366 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:28:50.240366 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:50.240366 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:28:50.240366 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:50.240366 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:28:50.241202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:50.241202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:28:50.241202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:50.241202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:28:50.241202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:28:50.241202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:28:50.241202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:28:50.241202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:50.241202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:50.241202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:50.241202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:28:50.241202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:50.241620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:50.241620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:50.241620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:28:50.241620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:50.241620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:28:50.241620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:28:50.241620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:50.241620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:28:50.241620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:50.241620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:28:50.241620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:28:50.241620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:50.241620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:28:50.241979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:50.241979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:28:50.242203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:50.242203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:28:50.242203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:28:50.242203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:28:50.242203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:28:50.242203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:28:50.242203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:50.242203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:28:50.242203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:28:50.242203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:28:50.242203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:28:50.242203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:28:50.242203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:50.242203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:50.242203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:28:50.242203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:28:50.242715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:28:50.242715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:28:50.242715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:50.242715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:50.242715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:50.242715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:28:50.242715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:50.242715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:50.242715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:50.242715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:28:50.242715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:50.242715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:28:50.242715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:28:50.242715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:50.242715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:28:50.242715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:50.242715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:28:50.242715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:28:50.242715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:50.243268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:28:50.243268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:50.243268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:28:54.987444 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:28:54.988646 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:28:54.988646 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:28:54.989302 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:28:55.017614 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:28:55.018009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:55.018977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:28:55.018977 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:55.018977 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:28:55.018977 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:28:55.018977 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:55.018977 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:28:55.018977 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:55.018977 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:28:55.019723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.031 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:55.019723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:28:55.019723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:55.019723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:28:55.019723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:28:55.019723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:28:55.019723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:28:55.019723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:55.019723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:55.019723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:55.019723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:28:55.019723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:55.021247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:55.021247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:55.021247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:28:55.021247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:55.021247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:28:55.021247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:28:55.021247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:55.021247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:28:55.021247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:55.021247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:28:55.021247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:28:55.021247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:55.021247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:28:55.022148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:55.022148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:28:55.022148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:55.022148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:28:55.022148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:28:55.022148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:28:55.022148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:28:55.022148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:28:55.022148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:55.022148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:28:55.022148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:28:55.022148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:28:55.022148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:28:55.022148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:28:55.022148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:55.023015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:55.023015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:28:55.023015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:28:55.023015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:28:55.023015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:28:55.023015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:55.023015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:55.023015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:55.023015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:28:55.023015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:55.023015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:55.023015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:55.023015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:28:55.023015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:55.023015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:28:55.023015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:28:55.023015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:55.023015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:28:55.023015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:55.024118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:28:55.024118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:28:55.024118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:55.024118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:28:55.024118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:55.024118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:28:55.196882 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:28:55.197500 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:28:55.197691 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:28:55.198681 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:28:55.199018 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:28:55.199275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:55.200107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:28:55.200107 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:55.200107 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:28:55.200107 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:28:55.200107 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:55.200107 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:28:55.200107 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:55.200107 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:28:55.201656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:55.201656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:28:55.201656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:55.201656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:28:55.201656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:28:55.201656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:28:55.201656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:28:55.201656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:55.201656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:55.201656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:55.201656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:28:55.201656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:55.202246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:55.202246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:55.202246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:28:55.202246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:55.202246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:28:55.202246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:28:55.202246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:55.202246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:28:55.202246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:55.202246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:28:55.202246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:28:55.202246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:55.202246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:28:55.202828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:55.202828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:28:55.203239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:55.203239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:28:55.203239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:28:55.203239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:28:55.203239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:28:55.203239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:28:55.203239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:55.203239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:28:55.203239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:28:55.203239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:28:55.203239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:28:55.203239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:28:55.203239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:55.203239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:55.203239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:28:55.203239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:28:55.203905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:28:55.203905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:28:55.203905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:55.203905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:55.203905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:55.203905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:28:55.203905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:55.203905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:55.203905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:55.203905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:28:55.203905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:55.203905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:28:55.203905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:28:55.203905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:55.203905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:28:55.203905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:55.203905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:28:55.203905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:28:55.203905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:55.204457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:28:55.204457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:55.204457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:28:55.204457 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:28:55.204457 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:28:55.204457 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:28:55.204457 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:28:55.204706 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:28:55.204870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:55.205582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:28:55.205582 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:55.205582 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:28:55.205582 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:28:55.205582 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:55.205582 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:28:55.205582 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:55.205582 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:28:55.206297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:55.206297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:28:55.206297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:28:55.206297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:28:55.206297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:55.206297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:28:55.206297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:55.206297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:28:55.206297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:55.206297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:28:55.206297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:55.206297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:28:55.206297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:55.206297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:28:55.206297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:28:55.206799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:55.206799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:28:55.206799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:55.206799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:28:55.206799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:28:55.206799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:55.206799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:28:55.206799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:55.206799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:28:55.207078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:55.207078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:28:55.207078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:55.207078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:28:55.207078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:28:55.207078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:28:55.207078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:28:55.207078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:28:55.207078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:28:55.207078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:28:55.207078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:55.207078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:55.207472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:55.207472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:28:55.207472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:55.207472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:55.207472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:55.207472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:28:55.207472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:55.207472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:28:55.207472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:28:55.207472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:55.207472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:28:55.207472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:55.207472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:28:55.207472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:28:55.207862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:55.207862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:28:55.207862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:55.207862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:28:55.208116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:55.208116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:28:55.208116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:28:55.208116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:28:55.208116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:28:55.208116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:28:55.208116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:55.208116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:28:55.208116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:28:55.208116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:28:55.208116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:28:55.208116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:28:55.208116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:55.208116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:55.208116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:28:55.208116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:28:55.208605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:28:55.208605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:28:55.208605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:28:55.208605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:28:55.208605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:28:55.208605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:55.208605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:55.208605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:55.208605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:28:55.208605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:55.208605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:55.208605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:55.208605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:28:55.208605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:55.208605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:28:55.208605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:28:55.208605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:55.208605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:28:55.209110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:55.209110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:28:55.209110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:28:55.209110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:55.209110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:28:55.209110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:55.209110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:28:55.231038 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:28:55.231443 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:28:55.231628 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:28:55.231812 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:28:55.232275 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:28:55.232662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:55.233498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:28:55.233498 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:55.233498 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:28:55.233498 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:28:55.233498 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:55.233498 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:28:55.233498 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:55.233498 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:28:55.234438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:55.234438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:28:55.234438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:28:55.234438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:28:55.234438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:55.234438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:28:55.234438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:55.234438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:28:55.234438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:55.234438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:28:55.234438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:55.234438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:28:55.234438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:55.234438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:28:55.234438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:28:55.234922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:55.234922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:28:55.234922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:55.234922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:28:55.234922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:28:55.234922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:55.234922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:28:55.234922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:55.234922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:28:55.235949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:55.235949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:28:55.235949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:55.235949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:28:55.235949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:28:55.235949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:28:55.235949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:28:55.235949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:28:55.235949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:28:55.235949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:55.235949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:55.236327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:55.236327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:28:55.236327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:55.236327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:55.236327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:55.236327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:28:55.236327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:55.236327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:28:55.236327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:28:55.236327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:55.236327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:28:55.236327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:55.236327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:28:55.236327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:28:55.236726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:55.236726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:28:55.236726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:55.236726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:28:55.237556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:55.237556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:28:55.237556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:28:55.237556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:28:55.237556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:28:55.237556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:28:55.237556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:55.237556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:28:55.237556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:28:55.237556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:28:55.237556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:28:55.237556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:28:55.237556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:55.237556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:55.237556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:28:55.237556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:28:55.237556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:28:55.238059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:28:55.238059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:28:55.238059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:28:55.238059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:55.238059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:55.238059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:55.238059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:28:55.238059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:55.238059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:55.238059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:55.238059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:28:55.238059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:55.238059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:28:55.238059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:28:55.238059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:55.238059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:28:55.238059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:55.238059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:28:55.238059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:28:55.238604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:55.238604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:28:55.238604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:55.238604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:28:55.238604 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:28:55.238769 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:28:55.238834 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:28:55.239122 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:28:55.239562 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:28:55.239902 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:28:55.240115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:55.240793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:28:55.240793 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:55.240793 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:28:55.240793 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:28:55.240793 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:55.240793 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:28:55.240793 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:55.240793 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:28:55.241633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:55.241633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:28:55.241633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:55.241633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:28:55.241633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:28:55.241633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:28:55.241633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:28:55.241633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:55.241633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:55.241633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:55.241633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:28:55.241633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:55.242003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:28:55.242003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:55.242003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:28:55.242003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:55.242003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:28:55.242003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:28:55.242003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:55.242003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:28:55.242003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:55.242003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:28:55.242003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:28:55.242003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:55.242003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:28:55.242360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:55.242360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:28:55.242612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:55.242612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:28:55.242612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:28:55.242612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:28:55.242612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:28:55.242612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:28:55.242612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:55.242612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:28:55.242612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:28:55.242612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:28:55.242612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:28:55.242612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:28:55.242612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:28:55.242612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:28:55.242612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:28:55.242612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:28:55.243078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:28:55.243078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:28:55.243078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:55.243078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:55.243078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:28:55.243078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:28:55.243078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:28:55.243078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:28:55.243078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:28:55.243078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:28:55.243078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:28:55.243078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:28:55.243078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:28:55.243078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:28:55.243078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:28:55.243078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:28:55.243078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:28:55.243078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:28:55.243078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:28:55.243653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:28:55.243653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:28:55.243653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:28:59.988168 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:28:59.989042 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:28:59.989042 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:28:59.990041 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:29:00.025020 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:00.025547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:00.026748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:29:00.026748 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:00.026748 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:29:00.026748 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:29:00.026748 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:00.026748 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:29:00.026748 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:00.026748 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:29:00.027837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:00.027837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:29:00.027837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:00.027837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:29:00.027837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:29:00.027837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:29:00.027837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:29:00.027837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:00.027837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:00.027837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:00.027837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:29:00.027837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:00.028322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:00.028322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:00.028322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:29:00.028322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:00.028322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:29:00.028322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:29:00.028322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:00.028322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:29:00.028322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:00.028322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:29:00.028322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:29:00.028322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:00.028322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:29:00.028791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:00.028791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:29:00.029009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:00.029009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:29:00.029009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:29:00.029009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:29:00.029009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:29:00.029009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:29:00.029009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:00.029009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:29:00.029009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:29:00.029009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:29:00.029009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:29:00.029009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:29:00.029009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:00.029009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:00.029009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:29:00.029009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:29:00.029559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:29:00.029559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:29:00.029559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:00.029559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:00.029559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:00.029559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:29:00.029559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:00.029559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:00.029559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:00.029559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:29:00.029559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:00.029559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:29:00.029559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:29:00.029559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:00.029559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:29:00.029559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:00.029559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:29:00.029559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:29:00.029559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:00.030090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:29:00.030090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:00.030090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:29:00.197578 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:29:00.198582 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:29:00.198582 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:29:00.199500 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:00.199838 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:00.200080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:00.201055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:29:00.201055 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:00.201055 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:29:00.201055 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:29:00.201055 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:00.201055 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:29:00.201055 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:00.201055 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:29:00.202024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:00.202024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:29:00.202024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:00.202024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:29:00.202024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:29:00.202024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:29:00.202024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:29:00.202024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:00.202024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:00.202024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:00.202024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:29:00.202024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:00.202661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:00.202661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:00.202661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:29:00.202661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:00.202661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:29:00.202661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:29:00.202661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:00.202661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:29:00.202661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:00.202661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:29:00.202661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:29:00.202661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:00.202661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:29:00.203229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:00.203229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:29:00.203229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:00.203229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:29:00.203229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:29:00.203229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:29:00.203229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:29:00.203229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:29:00.203229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:00.203229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:29:00.203229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:29:00.203229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:29:00.203229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:29:00.203229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:29:00.203229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:00.203977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:00.203977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:29:00.203977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:29:00.203977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:29:00.203977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:29:00.203977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:00.203977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:00.203977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:00.203977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:29:00.203977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:00.203977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:00.203977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:00.203977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:29:00.203977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:00.203977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:29:00.203977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:29:00.203977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:00.203977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:29:00.203977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:00.205018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:29:00.205018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:29:00.205018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:00.205018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:29:00.205018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:00.205018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:29:00.205018 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:29:00.205018 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:29:00.205018 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:29:00.205018 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:29:00.205479 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:00.205536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:00.206238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:29:00.206238 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:00.206238 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:29:00.206238 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:29:00.206238 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:00.206238 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:29:00.206238 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:00.206238 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:29:00.206990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:00.206990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:29:00.206990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:29:00.206990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:29:00.206990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:00.206990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:29:00.206990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:00.206990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:29:00.206990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:00.206990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:29:00.206990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:00.206990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:29:00.206990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:00.206990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:29:00.206990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:29:00.207488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:00.207488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:29:00.207488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:00.207488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:29:00.207488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:29:00.207488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:00.207488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:29:00.207488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:00.207488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:29:00.207992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:00.207992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:29:00.207992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:00.207992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:29:00.207992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:29:00.207992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:29:00.207992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:29:00.207992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:29:00.207992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:29:00.207992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:29:00.207992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:00.207992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:00.208370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:00.208370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:29:00.208370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:00.208370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:00.208370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:00.208370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:29:00.208370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:00.208370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:29:00.208370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:29:00.208370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:00.208370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:29:00.208370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:00.208370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:29:00.208370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:29:00.208834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:00.208834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:29:00.208834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:00.208834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:29:00.209081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:00.209081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:29:00.209081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:29:00.209081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:29:00.209081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:29:00.209081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:29:00.209081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:00.209081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:29:00.209081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:29:00.209081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:29:00.209081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:29:00.209081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:29:00.209081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:00.209081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:00.209081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:29:00.209081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:29:00.209602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:29:00.209602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:29:00.209602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:29:00.209602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:29:00.209602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:29:00.209602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:00.209602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:00.209602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:00.209602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:29:00.209602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:00.209602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:00.209602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:00.209602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:29:00.209602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:00.209602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:29:00.209602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:29:00.209602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:00.209602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:29:00.210098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:00.210098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:29:00.210098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:29:00.210098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:00.210098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:29:00.210098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:00.210098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:29:00.231681 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:29:00.232005 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:29:00.232117 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:29:00.232270 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:29:00.232596 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:00.232800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:00.233859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:29:00.233859 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:00.233859 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:29:00.233859 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:29:00.233859 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:00.233859 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:29:00.233859 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:00.233859 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:29:00.234633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:00.234633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:29:00.234633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:29:00.234633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:29:00.234633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:00.234633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:29:00.234633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:00.234633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:29:00.234633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:00.234633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:29:00.234633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:00.234633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:29:00.234633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:00.234633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:29:00.234633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:29:00.235304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:00.235304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:29:00.235304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:00.235304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:29:00.235304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:29:00.235304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:00.235304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:29:00.235304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:00.235304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:29:00.235710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:00.235710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:29:00.235710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:00.235710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:29:00.235710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:29:00.235710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:29:00.235710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:29:00.235710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:29:00.235710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:29:00.235710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:00.235710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:00.236234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:00.236234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:29:00.236234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:00.236234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:00.236234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:00.236234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:29:00.236234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:00.236234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:29:00.236234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:29:00.236234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:00.236234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:29:00.236234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:00.236234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:29:00.236234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:29:00.236903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:00.236903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:29:00.236903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:00.236903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:29:00.237113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:00.237113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:29:00.237113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:29:00.237113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:29:00.237113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:29:00.237113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:29:00.237113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:00.237113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:29:00.237113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:29:00.237113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:29:00.237113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:29:00.237113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:29:00.237113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:00.237113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:00.237113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:29:00.237113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:29:00.237113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:29:00.237948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:29:00.237948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:29:00.237948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:29:00.237948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:00.237948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:00.237948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:00.237948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:29:00.237948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:00.237948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:00.237948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:00.237948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:29:00.237948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:00.237948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:29:00.237948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:29:00.237948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:00.237948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:29:00.237948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:00.237948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:29:00.237948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:29:00.238975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:00.238975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:29:00.238975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:00.238975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:29:00.238975 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:29:00.239223 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:29:00.239223 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:29:00.239525 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:29:00.240009 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:00.240422 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:00.240677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:00.241499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:29:00.241499 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:00.241499 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:29:00.241499 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:29:00.241499 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:00.241499 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:29:00.241499 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:00.241499 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:29:00.242296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:00.242296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:29:00.242296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:00.242296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:29:00.242296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:29:00.242296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:29:00.242296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:29:00.242296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:00.242296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:00.242296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:00.242296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:29:00.242296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:00.242705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:00.242705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:00.242705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:29:00.242705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:00.242705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:29:00.242705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:29:00.242705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:00.242705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:29:00.242705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:00.242705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:29:00.242705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:29:00.242705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:00.242705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:29:00.243062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:00.243062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:29:00.243242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:00.243242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:29:00.243242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:29:00.243242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:29:00.243242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:29:00.243242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:29:00.243242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:00.243242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:29:00.243242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:29:00.243242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:29:00.243242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:29:00.243242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:29:00.243242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:00.243242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:00.243242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:29:00.243242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:29:00.243773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:29:00.243773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:29:00.243773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:00.243773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:00.243773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:00.243773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:29:00.243773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:00.243773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:00.243773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:00.243773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:29:00.243773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:00.243773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:29:00.243773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:29:00.243773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:00.243773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:29:00.243773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:00.243773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:29:00.243773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:29:00.243773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:00.244287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:29:00.244287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:00.244287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:29:04.988856 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:29:04.989851 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:29:04.989851 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:29:04.991797 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:29:05.025568 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:05.025993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:05.026943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:29:05.026943 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:05.026943 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:29:05.026943 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:29:05.026943 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:05.026943 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:29:05.026943 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:05.026943 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:29:05.027695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:05.027695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:29:05.027695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:05.027695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:29:05.027695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:29:05.027695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:29:05.027695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:29:05.027695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:05.027695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:05.027695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:05.027695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:29:05.027695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:05.028306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:05.028306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:05.028306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:29:05.028306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:05.028306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:29:05.028306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:29:05.028306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:05.028306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:29:05.028306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:05.028306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:29:05.028306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:29:05.028306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:05.028306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:29:05.028964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:05.028964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:29:05.028964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:05.028964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:29:05.028964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:29:05.028964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:29:05.028964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:29:05.028964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:29:05.028964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:05.028964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:29:05.028964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:29:05.028964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:29:05.028964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:29:05.028964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:29:05.028964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:05.029757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:05.029757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:29:05.029757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:29:05.029757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:29:05.029757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:29:05.029757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:05.029757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:05.029757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:05.029757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:29:05.029757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:05.029757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:05.029757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:05.029757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:29:05.029757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:05.029757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:29:05.029757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:29:05.029757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:05.029757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:29:05.029757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:05.030731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:29:05.030731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:29:05.030731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:05.030731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:29:05.030731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:05.030731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:29:05.198242 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:29:05.198887 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:29:05.199017 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:29:05.200219 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:05.200806 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:05.200806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:05.201671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:29:05.201671 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:05.201671 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:29:05.201671 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:29:05.201671 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:05.201671 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:29:05.201671 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:05.201671 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:29:05.203695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:05.203695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:29:05.203695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:05.203695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:29:05.203695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:29:05.203695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:29:05.203695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:29:05.203695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:05.203695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:05.203695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:05.203695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:29:05.203695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:05.204348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:05.204348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:05.204348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:29:05.204348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:05.204348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:29:05.204348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:29:05.204348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:05.204348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:29:05.204348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:05.204348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:29:05.204348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:29:05.204348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:05.204348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:29:05.204918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:05.204918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:29:05.205207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:05.205207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:29:05.205207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:29:05.205207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:29:05.205207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:29:05.205207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:29:05.205207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:05.205207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:29:05.205207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:29:05.205207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:29:05.205207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:29:05.205207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:29:05.205207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:05.205207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:05.205207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:29:05.205207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:29:05.205968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:29:05.205968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:29:05.205968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:05.205968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:05.205968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:05.205968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:29:05.205968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:05.205968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:05.205968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:05.205968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:29:05.205968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:05.205968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:29:05.205968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:29:05.205968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:05.205968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:29:05.205968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:05.205968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:29:05.205968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:29:05.205968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:05.206800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:29:05.206800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:05.206800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:29:05.206800 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:29:05.206800 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:29:05.206800 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:29:05.206800 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:29:05.206800 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:05.207264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:05.207815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:29:05.207815 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:05.207815 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:29:05.207815 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:29:05.207815 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:05.207815 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:29:05.207815 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:05.207815 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:29:05.209085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:05.209085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:29:05.209085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:29:05.209085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:29:05.209085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:05.209085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:29:05.209085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:05.209085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:29:05.209085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:05.209085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:29:05.209085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:05.209085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:29:05.209085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:05.209085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:29:05.209085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:29:05.209437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:05.209437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:29:05.209437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:05.209437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:29:05.209437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:29:05.209437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:05.209437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:29:05.209437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:05.209437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:29:05.210485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:05.210485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:29:05.210485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:05.210485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:29:05.210485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:29:05.210485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:29:05.210485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:29:05.210485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:29:05.210485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:29:05.210485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:29:05.210485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:05.210485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:05.210982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:05.210982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:29:05.210982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:05.210982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:05.210982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:05.210982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:29:05.210982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:05.210982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:29:05.210982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:29:05.210982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:05.210982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:29:05.210982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:05.210982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:29:05.210982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:29:05.211251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:05.211251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:29:05.211251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:05.211251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:29:05.212076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:05.212076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:29:05.212076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:29:05.212076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:29:05.212076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:29:05.212076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:29:05.212076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:05.212076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:29:05.212076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:29:05.212076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:29:05.212076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:29:05.212076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:29:05.212076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:05.212076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:05.212076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:29:05.212076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:29:05.212437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:29:05.212437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:29:05.212437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:29:05.212437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:29:05.212437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:29:05.212437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:05.212437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:05.212437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:05.212437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:29:05.212437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:05.212437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:05.212437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:05.212437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:29:05.212437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:05.212437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:29:05.212437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:29:05.212437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:05.212437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:29:05.212867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:05.212867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:29:05.212867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:29:05.212867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:05.212867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:29:05.212867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:05.212867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:29:05.232271 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:29:05.232640 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:29:05.232821 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:29:05.233145 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:29:05.233553 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:05.233914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:05.234665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:29:05.234665 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:05.234665 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:29:05.234665 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:29:05.234665 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:05.234665 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:29:05.234665 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:05.234665 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:29:05.235928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:05.235928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:29:05.235928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:29:05.235928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:29:05.235928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:05.235928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:29:05.235928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:05.235928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:29:05.235928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:05.235928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:29:05.235928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:05.235928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:29:05.235928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:05.235928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:29:05.235928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:29:05.236631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:05.236631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:29:05.236631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:05.236631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:29:05.236631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:29:05.236631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:05.236631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:29:05.236631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:05.236631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:29:05.237376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:05.237376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:29:05.237376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:05.237376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:29:05.237376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:29:05.237376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:29:05.237376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:29:05.237376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:29:05.237376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:29:05.237376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:05.237376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:05.237981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:05.237981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:29:05.237981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:05.237981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:05.237981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:05.237981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:29:05.237981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:05.237981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:29:05.237981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:29:05.237981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:05.237981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:29:05.237981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:05.237981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:29:05.237981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:29:05.238604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:05.238604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:29:05.238604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:05.238604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:29:05.239850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:05.239850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:29:05.239850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:29:05.239850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:29:05.239850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:29:05.239850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:29:05.239850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:05.239850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:29:05.239850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:29:05.239850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:29:05.239850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:29:05.239850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:29:05.239850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:05.239850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:05.239850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:29:05.239850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:29:05.239850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:29:05.240692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:29:05.240692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:29:05.240692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:29:05.240692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:05.240692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:05.240692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:05.240692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:29:05.240692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:05.240692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:05.240692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:05.240692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:29:05.240692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:05.240692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:29:05.240692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:29:05.240692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:05.240692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:29:05.240692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:05.240692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:29:05.240692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:29:05.241518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:05.241518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:29:05.241518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:05.241518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:29:05.241518 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:29:05.241518 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:29:05.241518 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:29:05.241518 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:29:05.241948 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:05.241948 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:05.241948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:05.242119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:29:05.242119 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:05.242119 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:29:05.242119 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:29:05.242119 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:05.242119 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:29:05.242119 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:05.242119 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:29:05.242815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:05.242815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:29:05.242815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:05.242815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:29:05.242815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:29:05.242815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:29:05.242815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:29:05.242815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:05.242815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:05.242815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:05.242815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:29:05.242815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:05.243238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:05.243238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:05.243238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:29:05.243238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:05.243238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:29:05.243238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:29:05.243238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:05.243238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:29:05.243238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:05.243238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:29:05.243238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:29:05.243238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:05.243238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:29:05.243584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:05.243584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:29:05.243757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:05.243757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:29:05.243757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:29:05.243757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:29:05.243757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:29:05.243757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:29:05.243757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:05.243757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:29:05.243757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:29:05.243757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:29:05.243757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:29:05.243757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:29:05.243757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:05.243757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:05.243757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:29:05.243757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:29:05.244205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:29:05.244205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:29:05.244205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:05.244205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:05.244205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:05.244205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:29:05.244205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:05.244205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:05.244205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:05.244205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:29:05.244205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:05.244205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:29:05.244205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:29:05.244205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:05.244205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:29:05.244205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:05.244205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:29:05.244205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:29:05.244205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:05.244748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:29:05.244748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:05.244748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:29:09.989647 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:29:09.990646 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:29:09.990646 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:29:09.991752 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:29:10.029559 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:10.029723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:10.030186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:29:10.030186 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:10.030186 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:29:10.030186 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:29:10.030186 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:10.030186 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:29:10.030186 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:10.030186 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:29:10.030882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:10.030882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:29:10.030882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:10.030882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:29:10.030882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:29:10.030882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:29:10.030882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:29:10.030882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:10.030882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:10.030882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:10.030882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:29:10.030882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:10.031578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:10.031578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:10.031578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:29:10.031578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:10.031578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:29:10.031578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:29:10.031578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:10.031578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:29:10.031578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:10.031578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:29:10.031578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:29:10.031578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:10.031578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:29:10.032239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:10.032239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:29:10.032239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:10.032239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:29:10.032239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:29:10.032239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:29:10.032239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:29:10.032239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:29:10.032239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:10.032239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:29:10.032239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:29:10.032239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:29:10.032239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:29:10.032239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:29:10.032239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:10.033005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:10.033005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:29:10.033005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:29:10.033005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:29:10.033005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:29:10.033005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:10.033005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:10.033005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:10.033005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:29:10.033005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:10.033005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:10.033005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:10.033005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:29:10.033005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:10.033005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:29:10.033005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:29:10.033005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:10.033005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:29:10.033005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:10.033936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:29:10.033936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:29:10.033936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:10.033936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:29:10.033936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:10.033936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:29:10.198852 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:29:10.199677 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:29:10.199889 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:29:10.200972 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:10.201251 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:10.201507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:10.202400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:29:10.202400 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:10.202400 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:29:10.202400 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:29:10.202400 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:10.202400 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:29:10.202400 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:10.202400 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:29:10.203845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:10.203845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:29:10.203845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:10.203845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:29:10.203845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:29:10.203845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:29:10.203845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:29:10.203845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:10.203845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:10.203845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:10.203845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:29:10.203845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:10.204393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:10.204393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:10.204393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:29:10.204393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:10.204393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:29:10.204393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:29:10.204393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:10.204393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:29:10.204393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:10.204393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:29:10.204393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:29:10.204393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:10.204393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:29:10.205017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:10.205017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:29:10.205639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:10.205639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:29:10.205639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:29:10.205639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:29:10.205639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:29:10.205639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:29:10.205639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:10.205639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:29:10.205639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:29:10.205639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:29:10.205639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:29:10.205639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:29:10.205639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:10.205639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:10.205639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:29:10.205639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:29:10.206381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:29:10.206381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:29:10.206381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:10.206381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:10.206381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:10.206381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:29:10.206381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:10.206381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:10.206381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:10.206381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:29:10.206381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:10.206381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:29:10.206381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:29:10.206381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:10.206381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:29:10.206381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:10.206381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:29:10.206381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:29:10.206381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:10.207570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:29:10.207570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:10.207570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:29:10.207570 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:29:10.207570 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:29:10.207570 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:29:10.207570 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:29:10.207570 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:10.208000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:10.208064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:29:10.208064 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:10.208064 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:29:10.208064 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:29:10.208064 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:10.208064 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:29:10.208064 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:10.208064 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:29:10.209193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:10.209193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:29:10.209193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:29:10.209193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:29:10.209193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:10.209193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:29:10.209193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:10.209193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:29:10.209193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:10.209193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:29:10.209193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:10.209193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:29:10.209193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:10.209193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:29:10.209193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:29:10.209963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:10.209963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:29:10.209963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:10.209963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:29:10.209963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:29:10.209963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:10.209963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:29:10.209963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:10.209963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:29:10.210578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:10.210578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:29:10.210578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:10.210578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:29:10.210578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:29:10.210578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:29:10.210578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:29:10.210578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:29:10.210578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:29:10.210578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:29:10.210578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:10.210578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:10.211167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:10.211167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:29:10.211167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:10.211167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:10.211167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:10.211167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:29:10.211167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:10.211167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:29:10.211167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:29:10.211167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:10.211167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:29:10.211167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:10.211167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:29:10.211167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:29:10.211782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:10.211782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:29:10.211782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:10.211782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:29:10.212079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:10.212079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:29:10.212079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:29:10.212079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:29:10.212079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:29:10.212079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:29:10.212079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:10.212079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:29:10.212079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:29:10.212079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:29:10.212079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:29:10.212079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:29:10.212079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:10.212079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:10.212079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:29:10.212079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:29:10.212935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:29:10.212935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:29:10.212935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:29:10.212935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:29:10.212935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:29:10.212935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:10.212935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:10.212935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:10.212935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:29:10.212935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:10.212935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:10.212935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:10.212935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:29:10.212935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:10.212935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:29:10.212935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:29:10.212935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:10.212935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:29:10.213862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:10.213862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:29:10.213862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:29:10.213862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:10.213862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:29:10.213862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:10.213862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:29:10.232812 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:29:10.233266 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:29:10.233502 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:29:10.233714 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:29:10.234055 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:10.234279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:10.234828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:29:10.234828 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:10.234828 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:29:10.234828 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:29:10.234828 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:10.234828 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:29:10.234828 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:10.234828 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:29:10.236120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:10.236120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:29:10.236120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:29:10.236120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:29:10.236120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:10.236120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:29:10.236120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:10.236120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:29:10.236120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:10.236120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:29:10.236120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:10.236120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:29:10.236120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:10.236120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:29:10.236120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:29:10.236870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:10.236870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:29:10.236870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:10.236870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:29:10.236870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:29:10.236870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:10.236870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:29:10.236870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:10.236870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:29:10.237738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:10.237738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:29:10.237738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:10.237738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:29:10.237738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:29:10.237738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:29:10.237738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:29:10.237738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:29:10.237738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:29:10.237738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:10.237738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:10.238135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:10.238135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:29:10.238135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:10.238135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:10.238135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:10.238135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:29:10.238135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:10.238135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:29:10.238135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:29:10.238135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:10.238135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:29:10.238135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:10.238135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:29:10.238135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:29:10.238580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:10.238580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:29:10.238580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:10.238580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:29:10.239639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:10.239639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:29:10.239639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:29:10.239639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:29:10.239639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:29:10.239639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:29:10.239639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:10.239639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:29:10.239639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:29:10.239639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:29:10.239639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:29:10.239639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:29:10.239639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:10.239639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:10.239639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:29:10.239639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:29:10.239639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:29:10.240162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:29:10.240162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:29:10.240162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:29:10.240162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:10.240162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:10.240162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:10.240162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:29:10.240162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:10.240162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:10.240162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:10.240162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:29:10.240162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:10.240162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:29:10.240162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:29:10.240162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:10.240162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:29:10.240162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:10.240162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:29:10.240162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:29:10.240732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:10.240732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:29:10.240732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:10.240732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:29:10.240732 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:29:10.240732 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:29:10.240732 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:29:10.240957 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:29:10.241240 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:10.241628 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:10.241853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:10.242628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:29:10.242628 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:10.242628 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:29:10.242628 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:29:10.242628 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:10.242628 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:29:10.242628 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:10.242628 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:29:10.243648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:10.243648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:29:10.243648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:10.243648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:29:10.243648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:29:10.243648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:29:10.243648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:29:10.243648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:10.243648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:10.243648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:10.243648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:29:10.243648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:10.244003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:10.244003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:10.244003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:29:10.244003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:10.244003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:29:10.244003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:29:10.244003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:10.244003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:29:10.244003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:10.244003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:29:10.244003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:29:10.244003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:10.244003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:29:10.244353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:10.244353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:29:10.244597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:10.244597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:29:10.244597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:29:10.244597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:29:10.244597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:29:10.244597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:29:10.244597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:10.244597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:29:10.244597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:29:10.244597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:29:10.244597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:29:10.244597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:29:10.244597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:10.244597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:10.244597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:29:10.244597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:29:10.245054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:29:10.245054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:29:10.245054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:10.245054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:10.245054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:10.245054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:29:10.245054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:10.245054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:10.245054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:10.245054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:29:10.245054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:10.245054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:29:10.245054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:29:10.245054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:10.245054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:29:10.245054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:10.245054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:29:10.245054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:29:10.245054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:10.245582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:29:10.245582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:10.245582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:29:14.990443 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:29:14.991457 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:29:14.991457 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:29:14.991951 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:29:15.027607 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:15.027980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:15.028879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:29:15.028879 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:15.028879 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:29:15.028879 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:29:15.028879 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:15.028879 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:29:15.028879 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:15.028879 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:29:15.029899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:15.029899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:29:15.029899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:15.029899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:29:15.029899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:29:15.029899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:29:15.029899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:29:15.029899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:15.029899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:15.029899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:15.029899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:29:15.029899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:15.030764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:15.030764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:15.030764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:29:15.030764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:15.030764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:29:15.030764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:29:15.030764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:15.030764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:29:15.030764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:15.030764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:29:15.030764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:29:15.030764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:15.030764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:29:15.031484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:15.031484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:29:15.031484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:15.031484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:29:15.031484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:29:15.031484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:29:15.031484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:29:15.031484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:29:15.031484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:15.031484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:29:15.031484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:29:15.031484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:29:15.031484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:29:15.031484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:29:15.031484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:15.032378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:15.032378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:29:15.032378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:29:15.032378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:29:15.032378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:29:15.032378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:15.032378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:15.032378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:15.032378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:29:15.032378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:15.032378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:15.032378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:15.032378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:29:15.032378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:15.032378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:29:15.032378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:29:15.032378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:15.032378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:29:15.032378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:15.033355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:29:15.033355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:29:15.033355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:15.033355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:29:15.033355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:15.033355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:29:15.199625 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:29:15.200271 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:29:15.200514 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:29:15.201553 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:15.201905 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:15.202235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:15.203172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:29:15.203172 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:15.203172 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:29:15.203172 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:29:15.203172 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:15.203172 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:29:15.203172 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:15.203172 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:29:15.204794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:15.204794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:29:15.204794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:15.204794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:29:15.204794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:29:15.204794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:29:15.204794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:29:15.204794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:15.204794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:15.204794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:15.204794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:29:15.204794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:15.206177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:15.206177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:15.206177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:29:15.206177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:15.206177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:29:15.206177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:29:15.206177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:15.206177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:29:15.206177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:15.206177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:29:15.206177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:29:15.206177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:15.206177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:29:15.206917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:15.206917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:29:15.206917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:15.206917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:29:15.206917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:29:15.206917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:29:15.206917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:29:15.206917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:29:15.206917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:15.206917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:29:15.206917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:29:15.206917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:29:15.206917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:29:15.206917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:29:15.206917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:15.207694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:15.207694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:29:15.207694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:29:15.207694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:29:15.207694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:29:15.207694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:15.207694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:15.207694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:15.207694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:29:15.207694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:15.207694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:15.207694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:15.207694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:29:15.207694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:15.207694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:29:15.207694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:29:15.207694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:15.207694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:29:15.207694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:15.208715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:29:15.208715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:29:15.208715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:15.208715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:29:15.208715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:15.208715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:29:15.208715 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:29:15.208715 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:29:15.208715 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:29:15.208715 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:29:15.209213 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:15.209213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:15.209213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:29:15.209213 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:15.209213 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:29:15.209213 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:29:15.209213 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:15.209213 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:29:15.209213 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:15.209213 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:29:15.209842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:15.209842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:29:15.209842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:29:15.209842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:29:15.209842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:15.209842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:29:15.209842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:15.209842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:29:15.209842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:15.209842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:29:15.209842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:15.209842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:29:15.209842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:15.209842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:29:15.209842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:29:15.210335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:15.210335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:29:15.210335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:15.210335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:29:15.210335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:29:15.210335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:15.210335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:29:15.210335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:15.210335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:29:15.210634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:15.210634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:29:15.210634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:15.210634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:29:15.210634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:29:15.210634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:29:15.210634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:29:15.210634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:29:15.210634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:29:15.210634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:29:15.210634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:15.210634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:15.210997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:15.210997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:29:15.210997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:15.210997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:15.210997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:15.210997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:29:15.210997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:15.210997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:29:15.210997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:29:15.210997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:15.210997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:29:15.210997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:15.210997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:29:15.210997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:29:15.211358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:15.211358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:29:15.211358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:15.211358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:29:15.211619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:15.211619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:29:15.211619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:29:15.211619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:29:15.211619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:29:15.211619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:29:15.211619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:15.211619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:29:15.211619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:29:15.211619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:29:15.211619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:29:15.211619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:29:15.211619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:15.211619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:15.211619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:29:15.211619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:29:15.212180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:29:15.212180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:29:15.212180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:29:15.212180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:29:15.212180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:29:15.212180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:15.212180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:15.212180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:15.212180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:29:15.212180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:15.212180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:15.212180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:15.212180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:29:15.212180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:15.212180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:29:15.212180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:29:15.212180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:15.212180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:29:15.212815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:15.212815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:29:15.212815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:29:15.212815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:15.212815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:29:15.212815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:15.212815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:29:15.233776 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:29:15.234141 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:29:15.234309 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:29:15.234547 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:29:15.234979 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:15.235257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:15.235890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:29:15.235890 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:15.235890 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:29:15.235890 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:29:15.235890 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:15.235890 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:29:15.235890 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:15.235890 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:29:15.237002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:15.237002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:29:15.237002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:29:15.237002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:29:15.237002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:15.237002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:29:15.237002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:15.237002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:29:15.237002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:15.237002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:29:15.237002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:15.237002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:29:15.237002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:15.237002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:29:15.237002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:29:15.237469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:15.237469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:29:15.237469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:15.237469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:29:15.237469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:29:15.237469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:15.237469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:29:15.237469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:15.237469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:29:15.238342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:15.238342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:29:15.238342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:15.238342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:29:15.238342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:29:15.238342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:29:15.238342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:29:15.238342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:29:15.238342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:29:15.238342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:15.238342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:15.238714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:15.238714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:29:15.238714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:15.238714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:15.238714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:15.238714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:29:15.238714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:15.238714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:29:15.238714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:29:15.238714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:15.238714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:29:15.238714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:15.238714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:29:15.238714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:29:15.239083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:15.239083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:29:15.239083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:15.239083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:29:15.240066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:15.240066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:29:15.240066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:29:15.240066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:29:15.240066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:29:15.240066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:29:15.240066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:15.240066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:29:15.240066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:29:15.240066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:29:15.240066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:29:15.240066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:29:15.240066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:15.240066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:15.240066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:29:15.240066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:29:15.240066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:29:15.240755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:29:15.240755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:29:15.240755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:29:15.240755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:15.240755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:15.240755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:15.240755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:29:15.240755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:15.240755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:15.240755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:15.240755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:29:15.240755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:15.240755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:29:15.240755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:29:15.240755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:15.240755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:29:15.240755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:15.240755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:29:15.240755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:29:15.241244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:15.241244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:29:15.241244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:15.241244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:29:15.241244 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:29:15.241244 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:29:15.241244 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:29:15.241244 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:29:15.241522 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:15.241766 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:15.241958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:15.242613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:29:15.242613 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:15.242613 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:29:15.242613 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:29:15.242613 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:15.242613 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:29:15.242613 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:15.242613 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:29:15.243892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:15.243892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:29:15.243892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:15.243892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:29:15.243892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:29:15.243892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:29:15.243892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:29:15.243892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:15.243892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:15.243892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:15.243892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:29:15.243892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:15.244262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:15.244262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:15.244262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:29:15.244262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:15.244262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:29:15.244262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:29:15.244262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:15.244262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:29:15.244262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:15.244262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:29:15.244262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:29:15.244262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:15.244262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:29:15.244695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:15.244695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:29:15.244890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:15.244890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:29:15.244890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:29:15.244890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:29:15.244890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:29:15.244890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:29:15.244890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:15.244890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:29:15.244890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:29:15.244890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:29:15.244890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:29:15.244890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:29:15.244890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:15.244890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:15.244890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:29:15.244890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:29:15.245393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:29:15.245393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:29:15.245393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:15.245393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:15.245393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:15.245393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:29:15.245393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:15.245393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:15.245393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:15.245393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:29:15.245393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:15.245393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:29:15.245393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:29:15.245393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:15.245393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:29:15.245393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:15.245393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:29:15.245393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:29:15.245393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:15.245983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:29:15.245983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:15.245983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:29:19.991204 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:29:19.991986 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:29:19.991986 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:29:19.992644 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:29:20.023952 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:20.023952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:20.024180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:29:20.024180 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:20.024180 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:29:20.024180 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:29:20.024180 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:20.024180 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:29:20.024180 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:20.024180 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:29:20.025234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:20.025234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:29:20.025234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:20.025234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:29:20.025234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:29:20.025234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:29:20.025234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:29:20.025234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:20.025234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:20.025234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:20.025234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:29:20.025234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:20.026498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:20.026498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:20.026498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:29:20.026498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:20.026498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:29:20.026498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:29:20.026498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:20.026498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:29:20.026498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:20.026498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:29:20.026498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:29:20.026498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:20.026498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:29:20.027519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:20.027519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:29:20.027519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:20.027519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:29:20.027519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:29:20.027519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:29:20.027519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:29:20.027519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:29:20.027519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:20.027519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:29:20.027519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:29:20.027519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:29:20.027519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:29:20.027519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:29:20.027519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:20.028213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:20.028213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:29:20.028213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:29:20.028213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:29:20.028213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:29:20.028213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:20.028213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:20.028213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:20.028213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:29:20.028213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:20.028213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:20.028213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:20.028213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:29:20.028213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:20.028213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:29:20.028213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:29:20.028213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:20.028213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:29:20.028213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:20.029069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:29:20.029069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:29:20.029069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:20.029069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:29:20.029069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:20.029069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:29:20.200453 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:29:20.201250 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:29:20.201385 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:29:20.203498 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:20.203498 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:20.203498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:20.203795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:29:20.203795 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:20.203795 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:29:20.203795 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:29:20.203795 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:20.203795 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:29:20.203795 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:20.203795 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:29:20.205374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:20.205374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:29:20.205374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:20.205374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:29:20.205374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:29:20.205374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:29:20.205374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:29:20.205374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:20.205374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:20.205374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:20.205374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:29:20.205374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:20.205961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:20.205961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:20.205961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:29:20.205961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:20.205961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:29:20.205961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:29:20.205961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:20.205961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:29:20.205961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:20.205961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:29:20.205961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:29:20.205961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:20.205961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:29:20.206587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:20.206587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:29:20.207128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:20.207128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:29:20.207128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:29:20.207128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:29:20.207128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:29:20.207128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:29:20.207128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:20.207128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:29:20.207128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:29:20.207128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:29:20.207128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:29:20.207128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:29:20.207128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:20.207128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:20.207128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:29:20.207128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:29:20.207590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:29:20.207590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:29:20.207590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:20.207590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:20.207590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:20.207590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:29:20.207590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:20.207590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:20.207590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:20.207590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:29:20.207590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:20.207590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:29:20.207590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:29:20.207590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:20.207590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:29:20.207590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:20.207590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:29:20.207590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:29:20.207590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:20.208095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:29:20.208095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:20.208095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:29:20.208095 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:29:20.208095 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:29:20.208095 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:29:20.208095 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:29:20.209085 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:20.209085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:20.209269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:29:20.209269 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:20.209269 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:29:20.209269 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:29:20.209269 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:20.209269 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:29:20.209269 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:20.209269 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:29:20.209991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:20.209991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:29:20.209991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:29:20.209991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:29:20.209991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:20.209991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:29:20.209991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:20.209991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:29:20.209991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:20.209991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:29:20.209991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:20.209991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:29:20.209991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:20.209991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:29:20.209991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:29:20.210422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:20.210422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:29:20.210422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:20.210422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:29:20.210422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:29:20.210422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:20.210422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:29:20.210422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:20.210422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:29:20.210785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:20.210785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:29:20.210785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:20.210785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:29:20.210785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:29:20.210785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:29:20.210785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:29:20.210785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:29:20.210785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:29:20.210785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:29:20.210785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:20.210785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:20.211145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:20.211145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:29:20.211145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:20.211145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:20.211145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:20.211145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:29:20.211145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:20.211145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:29:20.211145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:29:20.211145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:20.211145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:29:20.211145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:20.211145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:29:20.211145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:29:20.211539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:20.211539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:29:20.211539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:20.211539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:29:20.211742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:20.211742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:29:20.211742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:29:20.211742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:29:20.211742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:29:20.211742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:29:20.211742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:20.211742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:29:20.211742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:29:20.211742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:29:20.211742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:29:20.211742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:29:20.211742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:20.211742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:20.211742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:29:20.211742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:29:20.212175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:29:20.212175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:29:20.212175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:29:20.212175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:29:20.212175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:29:20.212175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:20.212175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:20.212175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:20.212175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:29:20.212175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:20.212175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:20.212175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:20.212175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:29:20.212175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:20.212175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:29:20.212175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:29:20.212175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:20.212175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:29:20.212685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:20.212685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:29:20.212685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:29:20.212685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:20.212685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:29:20.212685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:20.212685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:29:20.234983 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:29:20.235532 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:29:20.235716 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:29:20.235925 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:29:20.236576 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:20.236823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:20.237547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:29:20.237547 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:20.237547 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:29:20.237547 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:29:20.237547 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:20.237547 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:29:20.237547 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:20.237547 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:29:20.238800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:20.238800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:29:20.238800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:29:20.238800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:29:20.238800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:20.238800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:29:20.238800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:20.238800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:29:20.238800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:20.238800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:29:20.238800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:20.238800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:29:20.238800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:20.238800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:29:20.238800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:29:20.239235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:20.239235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:29:20.239235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:20.239235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:29:20.239235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:29:20.239235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:20.239235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:29:20.239235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:20.239235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:29:20.240294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:20.240294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:29:20.240294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:20.240294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:29:20.240294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:29:20.240294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:29:20.240294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:29:20.240294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:29:20.240294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:29:20.240294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:20.240294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:20.240650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:20.240650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:29:20.240650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:20.240650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:20.240650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:20.240650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:29:20.240650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:20.240650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:29:20.240650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:29:20.240650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:20.240650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:29:20.240650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:20.240650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:29:20.240650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:29:20.241048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:20.241048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:29:20.241048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:20.241048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:29:20.242540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:20.242540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:29:20.242540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:29:20.242540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:29:20.242540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:29:20.242540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:29:20.242540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:20.242540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:29:20.242540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:29:20.242540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:29:20.242540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:29:20.242540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:29:20.242540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:20.242540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:20.242540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:29:20.242540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:29:20.242540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:29:20.243052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:29:20.243052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:29:20.243052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:29:20.243052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:20.243052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:20.243052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:20.243052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:29:20.243052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:20.243052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:20.243052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:20.243052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:29:20.243052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:20.243052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:29:20.243052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:29:20.243052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:20.243052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:29:20.243052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:20.243052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:29:20.243052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:29:20.243594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:20.243594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:29:20.243594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:20.243594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:29:20.243594 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:29:20.243594 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:29:20.243594 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:29:20.243809 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:29:20.244154 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:20.244510 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:20.244762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:20.245419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:29:20.245419 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:20.245419 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:29:20.245419 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:29:20.245419 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:20.245419 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:29:20.245419 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:20.245419 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:29:20.246236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:20.246236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:29:20.246236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:20.246236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:29:20.246236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:29:20.246236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:29:20.246236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:29:20.246236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:20.246236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:20.246236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:20.246236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:29:20.246236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:20.246582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:20.246582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:20.246582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:29:20.246582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:20.246582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:29:20.246582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:29:20.246582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:20.246582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:29:20.246582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:20.246582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:29:20.246582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:29:20.246582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:20.246582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:29:20.247149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:20.247149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:29:20.247218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:20.247218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:29:20.247218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:29:20.247218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:29:20.247218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:29:20.247218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:29:20.247218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:20.247218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:29:20.247218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:29:20.247218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:29:20.247218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:29:20.247218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:29:20.247218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:20.247218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:20.247218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:29:20.247218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:29:20.247658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:29:20.247658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:29:20.247658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:20.247658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:20.247658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:20.247658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:29:20.247658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:20.247658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:20.247658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:20.247658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:29:20.247658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:20.247658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:29:20.247658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:29:20.247658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:20.247658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:29:20.247658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:20.247658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:29:20.247658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:29:20.247658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:20.248184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:29:20.248184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:20.248184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:29:24.991810 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:29:24.994530 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:29:24.994530 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:29:24.995672 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:29:24.999684 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG dbcounter [-] [105581] Writing DB stats neutron:SELECT=240 {{(pid=105581) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 14:29:25.033870 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:25.034065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:25.034527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:29:25.034527 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:25.034527 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:29:25.034527 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:29:25.034527 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:25.034527 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:29:25.034527 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:25.034527 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:29:25.035192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:25.035192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:29:25.035192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:25.035192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:29:25.035192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:29:25.035192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:29:25.035192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:29:25.035192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:25.035192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:25.035192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:25.035192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:29:25.035192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:25.035942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:25.035942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:25.035942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:29:25.035942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:25.035942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:29:25.035942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:29:25.035942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:25.035942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:29:25.035942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:25.035942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:29:25.035942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:29:25.035942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:25.035942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:29:25.036613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:25.036613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:29:25.036613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:25.036613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:29:25.036613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:29:25.036613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:29:25.036613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:29:25.036613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:29:25.036613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:25.036613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:29:25.036613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:29:25.036613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:29:25.036613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:29:25.036613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:29:25.036613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:25.037432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:25.037432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:29:25.037432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:29:25.037432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:29:25.037432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:29:25.037432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:25.037432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:25.037432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:25.037432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:29:25.037432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:25.037432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:25.037432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:25.037432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:29:25.037432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:25.037432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:29:25.037432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:29:25.037432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:25.037432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:29:25.037432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:25.039119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:29:25.039119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:29:25.039119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:25.039119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:29:25.039119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:25.039119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:29:25.201644 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:29:25.202662 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:29:25.202876 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:29:25.203997 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:25.204481 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:25.204657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:25.205611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:29:25.205611 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:25.205611 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:29:25.205611 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:29:25.205611 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:25.205611 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:29:25.205611 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:25.205611 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:29:25.207001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:25.207001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:29:25.207001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:25.207001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:29:25.207001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:29:25.207001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:29:25.207001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:29:25.207001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:25.207001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:25.207001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:25.207001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:29:25.207001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:25.207449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:25.207449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:25.207449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:29:25.207449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:25.207449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:29:25.207449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:29:25.207449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:25.207449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:29:25.207449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:25.207449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:29:25.207449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:29:25.207449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:25.207449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:29:25.207915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:25.207915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:29:25.208140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:25.208140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:29:25.208140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:29:25.208140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:29:25.208140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:29:25.208140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:29:25.208140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:25.208140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:29:25.208140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:29:25.208140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:29:25.208140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:29:25.208140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:29:25.208140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:25.208140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:25.208140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:29:25.208140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:29:25.208663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:29:25.208663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:29:25.208663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:25.208663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:25.208663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:25.208663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:29:25.208663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:25.208663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:25.208663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:25.208663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:29:25.208663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:25.208663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:29:25.208663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:29:25.208663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:25.208663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:29:25.208663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:25.208663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:29:25.208663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:29:25.208663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:25.209171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:29:25.209171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:25.209171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:29:25.209171 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:29:25.209171 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:29:25.209171 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:29:25.209171 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:29:25.210490 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:25.210490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:25.210490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:29:25.210490 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:25.210490 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:29:25.210490 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:29:25.210490 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:25.210490 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:29:25.210490 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:25.210490 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:29:25.211161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:25.211161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:29:25.211161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:29:25.211161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:29:25.211161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:25.211161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:29:25.211161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:25.211161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:29:25.211161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:25.211161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:29:25.211161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:25.211161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:29:25.211161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:25.211161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:29:25.211161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:29:25.211801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:25.211801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:29:25.211801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:25.211801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:29:25.211801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:29:25.211801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:25.211801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:29:25.211801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:25.211801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:29:25.212060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:25.212060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:29:25.212060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:25.212060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:29:25.212060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:29:25.212060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:29:25.212060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:29:25.212060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:29:25.212060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:29:25.212060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:29:25.212060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:25.212060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:25.212459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:25.212459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:29:25.212459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:25.212459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:25.212459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:25.212459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:29:25.212459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:25.212459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:29:25.212459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:29:25.212459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:25.212459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:29:25.212459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:25.212459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:29:25.212459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:29:25.213054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:25.213054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:29:25.213054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:25.213054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:29:25.213178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:25.213178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:29:25.213178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:29:25.213178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:29:25.213178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:29:25.213178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:29:25.213178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:25.213178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:29:25.213178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:29:25.213178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:29:25.213178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:29:25.213178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:29:25.213178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:25.213178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:25.213178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:29:25.213178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:29:25.213731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:29:25.213731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:29:25.213731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:29:25.213731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:29:25.213731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:29:25.213731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:25.213731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:25.213731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:25.213731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:29:25.213731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:25.213731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:25.213731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:25.213731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:29:25.213731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:25.213731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:29:25.213731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:29:25.213731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:25.213731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:29:25.214236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:25.214236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:29:25.214236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:29:25.214236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:25.214236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:29:25.214236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:25.214236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:29:25.235801 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:29:25.236600 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:29:25.236854 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:29:25.237163 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:29:25.237800 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:25.238516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:25.239234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:29:25.239234 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:25.239234 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:29:25.239234 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:29:25.239234 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:25.239234 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:29:25.239234 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:25.239234 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:29:25.240541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:25.240541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:29:25.240541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:29:25.240541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:29:25.240541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:25.240541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:29:25.240541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:25.240541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:29:25.240541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:25.240541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:29:25.240541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:25.240541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:29:25.240541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:25.240541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:29:25.240541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:29:25.241205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:25.241205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:29:25.241205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:25.241205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:29:25.241205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:29:25.241205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:25.241205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:29:25.241205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:25.241205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:29:25.242152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:25.242152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:29:25.242152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:25.242152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:29:25.242152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:29:25.242152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:29:25.242152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:29:25.242152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:29:25.242152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:29:25.242152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:25.242152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:25.242712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:25.242712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:29:25.242712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:25.242712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:25.242712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:25.242712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:29:25.242712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:25.242712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:29:25.242712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:29:25.242712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:25.242712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:29:25.242712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:25.242712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:29:25.242712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:29:25.243288 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:25.243288 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:29:25.243288 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:25.243288 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:29:25.243496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:25.243496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:29:25.243496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:29:25.243496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:29:25.243496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:29:25.243496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:29:25.243496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:25.243496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:29:25.243496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:29:25.243496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:29:25.243496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:29:25.243496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:29:25.243496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:25.243496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:25.243496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:29:25.243496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:29:25.243496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:29:25.244219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:29:25.244219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:29:25.244219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:29:25.244219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:25.244219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:25.244219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:25.244219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:29:25.244219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:25.244219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:25.244219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:25.244219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:29:25.244219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:25.244219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:29:25.244219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:29:25.244219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:25.244219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:29:25.244219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:25.244219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:29:25.244219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:29:25.245244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:25.245244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:29:25.245244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:25.245244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:29:25.245244 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:29:25.245244 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:29:25.245244 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:29:25.245244 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:29:25.245856 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:25.245856 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:25.245856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:25.246185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:29:25.246185 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:25.246185 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:29:25.246185 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:29:25.246185 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:25.246185 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:29:25.246185 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:25.246185 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:29:25.246938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:25.246938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:29:25.246938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:25.246938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:29:25.246938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:29:25.246938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:29:25.246938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:29:25.246938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:25.246938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:25.246938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:25.246938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:29:25.246938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:25.247272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:25.247272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:25.247272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:29:25.247272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:25.247272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:29:25.247272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:29:25.247272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:25.247272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:29:25.247272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:25.247272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:29:25.247272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:29:25.247272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:25.247272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:29:25.247599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:25.247599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:29:25.247877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:25.247877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:29:25.247877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:29:25.247877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:29:25.247877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:29:25.247877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:29:25.247877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:25.247877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:29:25.247877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:29:25.247877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:29:25.247877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:29:25.247877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:29:25.247877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:25.247877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:25.247877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:29:25.247877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:29:25.248276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:29:25.248276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:29:25.248276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:25.248276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:25.248276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:25.248276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:29:25.248276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:25.248276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:25.248276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:25.248276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:29:25.248276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:25.248276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:29:25.248276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:29:25.248276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:25.248276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:29:25.248276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:25.248276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:29:25.248276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:29:25.248276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:25.248825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:29:25.248825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:25.248825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:29:29.993639 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:29:29.994359 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:29:29.994359 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:29:29.994962 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:29:30.027814 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:30.028094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:30.028726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:29:30.028726 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:30.028726 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:29:30.028726 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:29:30.028726 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:30.028726 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:29:30.028726 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:30.028726 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:29:30.029396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:30.029396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:29:30.029396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:30.029396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:29:30.029396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:29:30.029396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:29:30.029396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:29:30.029396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:30.029396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:30.029396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:30.029396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:29:30.029396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:30.030061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:30.030061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:30.030061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:29:30.030061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:30.030061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:29:30.030061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:29:30.030061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:30.030061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:29:30.030061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:30.030061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:29:30.030061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:29:30.030061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:30.030061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:29:30.030749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:30.030749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:29:30.030749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:30.030749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:29:30.030749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:29:30.030749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:29:30.030749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:29:30.030749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:29:30.030749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:30.030749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:29:30.030749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:29:30.030749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:29:30.030749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:29:30.030749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:29:30.030749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:30.031557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:30.031557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:29:30.031557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:29:30.031557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:29:30.031557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:29:30.031557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:30.031557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:30.031557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:30.031557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:29:30.031557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:30.031557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:30.031557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:30.031557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:29:30.031557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:30.031557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:29:30.031557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:29:30.031557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:30.031557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:29:30.031557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:30.032489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:29:30.032489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:29:30.032489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:30.032489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:29:30.032489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:30.032489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:29:30.203080 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:29:30.203903 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:29:30.204155 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:29:30.205492 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:30.205974 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:30.206247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:30.207816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:29:30.207816 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:30.207816 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:29:30.207816 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:29:30.207816 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:30.207816 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:29:30.207816 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:30.207816 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:29:30.209386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:30.209386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:29:30.209386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:30.209386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:29:30.209386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:29:30.209386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:29:30.209386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:29:30.209386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:30.209386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:30.209386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:30.209386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:29:30.209386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:30.209995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:30.209995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:30.209995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:29:30.209995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:30.209995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:29:30.209995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:29:30.209995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:30.209995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:29:30.209995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:30.209995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:29:30.209995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:29:30.209995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:30.209995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:29:30.210583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:30.210583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:29:30.211866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:30.211866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:29:30.211866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:29:30.211866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:29:30.211866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:29:30.211866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:29:30.211866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:30.211866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:29:30.211866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:29:30.211866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:29:30.211866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:29:30.211866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:29:30.211866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:30.211866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:30.211866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:29:30.211866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:29:30.212812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:29:30.212812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:29:30.212812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:30.212812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:30.212812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:30.212812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:29:30.212812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:30.212812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:30.212812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:30.212812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:29:30.212812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:30.212812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:29:30.212812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:29:30.212812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:30.212812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:29:30.212812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:30.212812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:29:30.212812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:29:30.212812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:30.213658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:29:30.213658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:30.213658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:29:30.213658 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:29:30.213658 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:29:30.213658 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:29:30.213658 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:29:30.214045 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:30.214186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:30.214818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:29:30.214818 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:30.214818 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:29:30.214818 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:29:30.214818 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:30.214818 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:29:30.214818 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:30.214818 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:29:30.215564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:30.215564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:29:30.215564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:29:30.215564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:29:30.215564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:30.215564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:29:30.215564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:30.215564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:29:30.215564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:30.215564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:29:30.215564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:30.215564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:29:30.215564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:30.215564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:29:30.215564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:29:30.216304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:30.216304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:29:30.216304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:30.216304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:29:30.216304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:29:30.216304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:30.216304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:29:30.216304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:30.216304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:29:30.216304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:30.216304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:29:30.216304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:30.216304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:29:30.216304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:29:30.217026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:29:30.217026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:29:30.217026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:29:30.217026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:29:30.217026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:29:30.217026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:30.217026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:30.217026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:30.217026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:29:30.217026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:30.217026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:30.217026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:30.217026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:29:30.217631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:30.217631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:29:30.217631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:29:30.217631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:30.217631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:29:30.217631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:30.217631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:29:30.217631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:29:30.217631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:30.217631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:29:30.217631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:30.217631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:29:30.218116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:30.218116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:29:30.218116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:29:30.218116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:29:30.218116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:29:30.218116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:29:30.218116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:30.218116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:29:30.218116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:29:30.218116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:29:30.218116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:29:30.218116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:29:30.218116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:30.218116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:30.218116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:29:30.218116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:29:30.218832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:29:30.218832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:29:30.218832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:29:30.218832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:29:30.218832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:29:30.218832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:30.218832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:30.218832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:30.218832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:29:30.218832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:30.218832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:30.218832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:30.218832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:29:30.218832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:30.218832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:29:30.218832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:29:30.218832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:30.218832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:29:30.219652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:30.219652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:29:30.219652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:29:30.219652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:30.219652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:29:30.219652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:30.219652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:29:30.236368 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:29:30.236900 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:29:30.237072 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:29:30.237316 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:29:30.237896 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:30.238330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:30.239137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:29:30.239137 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:30.239137 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:29:30.239137 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:29:30.239137 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:30.239137 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:29:30.239137 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:30.239137 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:29:30.240264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:30.240264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:29:30.240264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:29:30.240264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:29:30.240264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:30.240264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:29:30.240264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:30.240264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:29:30.240264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:30.240264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:29:30.240264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:30.240264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:29:30.240264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:30.240264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:29:30.240264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:29:30.240985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:30.240985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:29:30.240985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:30.240985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:29:30.240985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:29:30.240985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:30.240985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:29:30.240985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:30.240985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:29:30.241659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:30.241659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:29:30.241659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:30.241659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:29:30.241659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:29:30.241659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:29:30.241659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:29:30.241659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:29:30.241659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:29:30.241659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:30.241659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:30.242205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:30.242205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:29:30.242205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:30.242205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:30.242205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:30.242205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:29:30.242205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:30.242205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:29:30.242205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:29:30.242205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:30.242205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:29:30.242205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:30.242205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:29:30.242205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:29:30.242908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:30.242908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:29:30.242908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:30.242908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:29:30.243271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:30.243271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:29:30.243271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:29:30.243271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:29:30.243271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:29:30.243271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:29:30.243271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:30.243271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:29:30.243271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:29:30.243271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:29:30.243271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:29:30.243271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:29:30.243271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:30.243271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:30.243271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:29:30.243271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:29:30.243271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:29:30.244098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:29:30.244098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:29:30.244098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:29:30.244098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:30.244098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:30.244098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:30.244098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:29:30.244098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:30.244098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:30.244098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:30.244098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:29:30.244098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:30.244098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:29:30.244098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:29:30.244098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:30.244098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:29:30.244098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:30.244098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:29:30.244098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:29:30.245034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:30.245034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:29:30.245034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:30.245034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:29:30.245034 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:29:30.245034 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:29:30.245034 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:29:30.245034 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:29:30.245523 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:30.247146 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:30.247297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:30.248033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:29:30.248033 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:30.248033 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:29:30.248033 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:29:30.248033 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:30.248033 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:29:30.248033 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:30.248033 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:29:30.248819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:30.248819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:29:30.248819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:30.248819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:29:30.248819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:29:30.248819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:29:30.248819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:29:30.248819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:30.248819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:30.248819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:30.248819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:29:30.248819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:30.249165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:30.249165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:30.249165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:29:30.249165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:30.249165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:29:30.249165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:29:30.249165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:30.249165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:29:30.249165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:30.249165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:29:30.249165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:29:30.249165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:30.249165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:29:30.249528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:30.249528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:29:30.249749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:30.249749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:29:30.249749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:29:30.249749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:29:30.249749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:29:30.249749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:29:30.249749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:30.249749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:29:30.249749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:29:30.249749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:29:30.249749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:29:30.249749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:29:30.249749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:30.249749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:30.249749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:29:30.249749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:29:30.250218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:29:30.250218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:29:30.250218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:30.250218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:30.250218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:30.250218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:29:30.250218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:30.250218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:30.250218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:30.250218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:29:30.250218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:30.250218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:29:30.250218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:29:30.250218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:30.250218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:29:30.250218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:30.250218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:29:30.250218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:29:30.250218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:30.250724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:29:30.250724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:30.250724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:29:34.272039 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:29:34.273079 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_for_inconsistencies {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:29:34.273079 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Checking Neutron and OVN revision consistency. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:29:34.286264 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Synchronizing Neutron and OVN databases started {{(pid=105581) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:440}}
May 09 14:29:34.286426 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Number of inconsistencies found at create/update: networks=4, routers=1, router_ports=1 {{(pid=105581) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:419}}
May 09 14:29:34.286544 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Fixing resource 0b4a5724-2518-4e33-a6d3-0d92c749f303 (type: networks) at create/update {{(pid=105581) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 14:29:34.320425 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-0b4a5724-2518-4e33-a6d3-0d92c749f303, resource={'id': '0b4a5724-2518-4e33-a6d3-0d92c749f303', 'name': 'private', 'tenant_id': '5796380176014a32af8ff613b146faed', 'project_id': '5796380176014a32af8ff613b146faed', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['23eb2e2e-ec2f-4f98-bc47-7125a2f92952', 'db2636e9-0880-488d-9aa3-e100ce5eeb9f'], 'standard_attr_id': 12, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-09T14:04:20Z', 'updated_at': '2026-05-09T14:04:23Z', 'revision_number': 3, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 52278}, resource_type=networks, if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:34.320740 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-0b4a5724-2518-4e33-a6d3-0d92c749f303, col_values=(('external_ids', {'neutron:network_name': 'private', 'neutron:mtu': '1372', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:34.321109 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-f71b8cb1-fe70-438c-84f9-bbb7008a596d, external_ids_update=None, columns={'tag': 52278}, if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:34.321248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:34.321996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:29:34.321996 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:34.321996 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:29:34.321996 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:29:34.321996 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:34.321996 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:29:34.321996 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:34.321996 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:29:34.323269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Failed to fix resource 0b4a5724-2518-4e33-a6d3-0d92c749f303 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:34.323269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:29:34.323269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 14:29:34.323269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 14:29:34.323269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 14:29:34.323269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 14:29:34.323269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 09 14:29:34.323269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:29:34.323269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:34.323269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:34.323269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:34.323949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:29:34.323949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:34.323949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:34.323949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:34.323949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:29:34.323949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:34.323949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:29:34.323949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:29:34.323949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:34.323949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:29:34.323949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:34.323949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:29:34.323949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:29:34.323949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:34.324756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:29:34.324756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:34.324756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:29:34.324756 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Fixing resource a7d094cd-23f3-4e0c-9e88-24be6ee17b2c (type: networks) at create/update {{(pid=105581) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 14:29:34.357928 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-a7d094cd-23f3-4e0c-9e88-24be6ee17b2c, resource={'id': 'a7d094cd-23f3-4e0c-9e88-24be6ee17b2c', 'name': 'public', 'tenant_id': '5796380176014a32af8ff613b146faed', 'project_id': '5796380176014a32af8ff613b146faed', 'admin_state_up': True, 'mtu': 1430, 'status': 'ACTIVE', 'subnets': ['2f0821e3-c6a2-49d6-80ee-3480a17ba8d2', '644269b7-6b84-4d8a-ad28-d59a7c6b7c5e'], 'standard_attr_id': 18, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': True, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'is_default': True, 'tags': [], 'created_at': '2026-05-09T14:04:26Z', 'updated_at': '2026-05-09T14:04:34Z', 'revision_number': 3, 'provider:network_type': 'flat', 'provider:physical_network': 'public', 'provider:segmentation_id': None}, resource_type=networks, if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:34.358347 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-a7d094cd-23f3-4e0c-9e88-24be6ee17b2c, col_values=(('external_ids', {'neutron:network_name': 'public', 'neutron:mtu': '1430', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'flat', 'neutron:provnet-physical-network': 'public'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false', 'fdb_age_threshold': '300', 'broadcast-arps-to-all-routers': 'true'})), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:34.359133 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-08fea115-9b05-4667-a0d0-487c303084a5, external_ids_update=None, columns={'tag': []}, if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:34.359553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:34.360341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:29:34.360341 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:34.360341 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:29:34.360341 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:29:34.360341 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:34.360341 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:29:34.360341 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:34.360341 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:29:34.361897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Failed to fix resource a7d094cd-23f3-4e0c-9e88-24be6ee17b2c (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:34.361897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:29:34.361897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 14:29:34.361897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 14:29:34.361897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 14:29:34.361897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 14:29:34.361897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 09 14:29:34.361897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:29:34.361897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:34.361897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:34.361897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:34.362510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:29:34.362510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:34.362510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:34.362510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:34.362510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:29:34.362510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:34.362510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:29:34.362510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:29:34.362510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:34.362510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:29:34.362510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:34.362510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:29:34.362510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:29:34.362510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:34.363323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:29:34.363323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:34.363323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:29:34.363323 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Fixing resource 3e80d551-6de5-4370-806b-36d1db5f6fda (type: networks) at create/update {{(pid=105581) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 14:29:34.391766 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-3e80d551-6de5-4370-806b-36d1db5f6fda, resource={'id': '3e80d551-6de5-4370-806b-36d1db5f6fda', 'name': 'admin_net', 'tenant_id': '590c644b03ae4bed862ecf492c6b2eb2', 'project_id': '590c644b03ae4bed862ecf492c6b2eb2', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['453849e5-22a7-43ca-911e-9401ed2b0d74'], 'standard_attr_id': 42, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-09T14:07:41Z', 'updated_at': '2026-05-09T14:07:42Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 35952}, resource_type=networks, if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:34.392350 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-3e80d551-6de5-4370-806b-36d1db5f6fda, col_values=(('external_ids', {'neutron:network_name': 'admin_net', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:34.392956 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-f84050fb-ff65-46ce-b744-016e1ff15440, external_ids_update=None, columns={'tag': 35952}, if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:34.393160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:34.394074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:29:34.394074 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:34.394074 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:29:34.394074 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:29:34.394074 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:34.394074 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:29:34.394074 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:34.394074 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:29:34.394955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Failed to fix resource 3e80d551-6de5-4370-806b-36d1db5f6fda (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:34.394955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:29:34.394955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 14:29:34.394955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 14:29:34.394955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 14:29:34.394955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 14:29:34.394955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 09 14:29:34.394955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:29:34.394955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:34.394955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:34.394955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:34.395252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:29:34.395252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:34.395252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:34.395252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:34.395252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:29:34.395252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:34.395252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:29:34.395252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:29:34.395252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:34.395252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:29:34.395252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:34.395252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:29:34.395252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:29:34.395252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:34.395622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:29:34.395622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:34.395622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:29:34.395622 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Fixing resource 632a2ddb-5634-4bb3-aad0-f05d97abd45e (type: networks) at create/update {{(pid=105581) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 14:29:34.422340 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-632a2ddb-5634-4bb3-aad0-f05d97abd45e, resource={'id': '632a2ddb-5634-4bb3-aad0-f05d97abd45e', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-5urlz', 'tenant_id': '1f17f4a2a7ac4543a1253522e61993d2', 'project_id': '1f17f4a2a7ac4543a1253522e61993d2', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['32b523fe-135f-4f9e-a98b-fc0537fcdb35'], 'standard_attr_id': 52, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-09T14:08:15Z', 'updated_at': '2026-05-09T14:08:16Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 39245}, resource_type=networks, if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:34.422799 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-632a2ddb-5634-4bb3-aad0-f05d97abd45e, col_values=(('external_ids', {'neutron:network_name': 'k8s-clusterapi-cluster-magnum-system-kube-5urlz', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:34.423429 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-bca44f95-fe96-432e-84b8-1edc05e10ce0, external_ids_update=None, columns={'tag': 39245}, if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:34.423615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:34.424543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:29:34.424543 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:34.424543 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:29:34.424543 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:29:34.424543 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:34.424543 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:29:34.424543 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:34.424543 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:29:34.425695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Failed to fix resource 632a2ddb-5634-4bb3-aad0-f05d97abd45e (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:34.425695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:29:34.425695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 14:29:34.425695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 14:29:34.425695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 14:29:34.425695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 14:29:34.425695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 09 14:29:34.425695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:29:34.425695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:34.425695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:34.425695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:34.426247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:29:34.426247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:34.426247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:34.426247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:34.426247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:29:34.426247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:34.426247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:29:34.426247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:29:34.426247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:34.426247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:29:34.426247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:34.426247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:29:34.426247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:29:34.426247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:34.426868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:29:34.426868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:34.426868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:29:34.426868 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Fixing resource 09e042cf-39a9-42c8-b683-b29ffc42b049 (type: routers) at create/update {{(pid=105581) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 14:29:34.502098 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, resource={'id': '09e042cf-39a9-42c8-b683-b29ffc42b049', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-5urlz', 'project_id': '1f17f4a2a7ac4543a1253522e61993d2', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': 'a7d094cd-23f3-4e0c-9e88-24be6ee17b2c', 'external_fixed_ips': [{'subnet_id': '2f0821e3-c6a2-49d6-80ee-3480a17ba8d2', 'ip_address': '172.24.5.175'}, {'subnet_id': '644269b7-6b84-4d8a-ad28-d59a7c6b7c5e', 'ip_address': '2001:db8::d8'}], 'enable_snat': True}, 'gw_port_id': '71ded037-e812-40c0-8d8f-70c141c74323', 'description': 'Created by cluster-api-provider-openstack cluster magnum-system-kube-5urlz', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': 'a7d094cd-23f3-4e0c-9e88-24be6ee17b2c', 'external_fixed_ips': [{'ip_address': '172.24.5.175', 'subnet_id': '2f0821e3-c6a2-49d6-80ee-3480a17ba8d2'}, {'ip_address': '2001:db8::d8', 'subnet_id': '644269b7-6b84-4d8a-ad28-d59a7c6b7c5e'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2026-05-09T14:08:17Z', 'updated_at': '2026-05-09T14:08:19Z', 'revision_number': 3, 'tenant_id': '1f17f4a2a7ac4543a1253522e61993d2'}, resource_type=routers, if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:34.502389 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, columns={'external_ids': {'neutron:router_name': 'k8s-clusterapi-cluster-magnum-system-kube-5urlz', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:34.502695 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-a7d094cd-23f3-4e0c-9e88-24be6ee17b2c, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "71ded037-e812-40c0-8d8f-70c141c74323"')], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:34.502846 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-a7d094cd-23f3-4e0c-9e88-24be6ee17b2c, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "71ded037-e812-40c0-8d8f-70c141c74323"')], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:34.502943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:34.503673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:29:34.503673 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:34.503673 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:29:34.503673 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:29:34.503673 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:34.503673 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:29:34.503673 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:34.503673 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:29:34.504114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Unable to update router 09e042cf-39a9-42c8-b683-b29ffc42b049. Error: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:34.505466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Failed to fix resource 09e042cf-39a9-42c8-b683-b29ffc42b049 (type: routers): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:34.505466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:29:34.505466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 14:29:34.505466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 14:29:34.505466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 14:29:34.505466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 14:29:34.505466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1635, in update_router
May 09 14:29:34.505466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with excutils.save_and_reraise_exception():
May 09 14:29:34.505466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 271, in __exit__
May 09 14:29:34.505466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.force_reraise()
May 09 14:29:34.505466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 233, in force_reraise
May 09 14:29:34.506070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise self.value
May 09 14:29:34.506070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1569, in update_router
May 09 14:29:34.506070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:29:34.506070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:34.506070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:34.506070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:34.506070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:29:34.506070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:34.506070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:34.506070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:34.506070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:29:34.506070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:34.506070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:29:34.506070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:29:34.506753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:34.506753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:29:34.506753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:34.506753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:29:34.506753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:29:34.506753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:34.506753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:29:34.506753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:34.506753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:29:34.506753 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Fixing resource 71ded037-e812-40c0-8d8f-70c141c74323 (type: router_ports) at create/update {{(pid=105581) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 14:29:34.608527 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-71ded037-e812-40c0-8d8f-70c141c74323, resource={'id': '71ded037-e812-40c0-8d8f-70c141c74323', 'name': '', 'network_id': 'a7d094cd-23f3-4e0c-9e88-24be6ee17b2c', 'project_id': '', 'mac_address': 'fa:16:3e:c7:86:37', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '09e042cf-39a9-42c8-b683-b29ffc42b049', 'device_owner': 'network:router_gateway', 'standard_attr_id': 57, 'fixed_ips': [{'subnet_id': '2f0821e3-c6a2-49d6-80ee-3480a17ba8d2', 'ip_address': '172.24.5.175'}, {'subnet_id': '644269b7-6b84-4d8a-ad28-d59a7c6b7c5e', 'ip_address': '2001:db8::d8'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0000173183', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'ovs_create_tap': False, 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2026-05-09T14:08:17Z', 'updated_at': '2026-05-09T14:08:19Z', 'revision_number': 5, 'tenant_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:34.608759 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-71ded037-e812-40c0-8d8f-70c141c74323, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': '2f0821e3-c6a2-49d6-80ee-3480a17ba8d2 644269b7-6b84-4d8a-ad28-d59a7c6b7c5e', 'neutron:network_name': 'neutron-a7d094cd-23f3-4e0c-9e88-24be6ee17b2c', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-09e042cf-39a9-42c8-b683-b29ffc42b049'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['2001:db8::d8/64', '172.24.5.175/24'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:34.609016 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=71ded037-e812-40c0-8d8f-70c141c74323, lrouter_port=lrp-71ded037-e812-40c0-8d8f-70c141c74323, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:34.609240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:34.609940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:29:34.609940 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:34.609940 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:29:34.609940 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:29:34.609940 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:34.609940 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:29:34.609940 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:34.609940 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:29:34.610829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Failed to fix resource 71ded037-e812-40c0-8d8f-70c141c74323 (type: router_ports): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:34.610829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:29:34.610829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 14:29:34.610829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 14:29:34.610829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 14:29:34.610829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 14:29:34.610829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1976, in update_router_port
May 09 14:29:34.610829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:29:34.610829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:34.610829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:34.610829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:34.611401 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:29:34.611401 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:34.611401 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:34.611401 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:34.611401 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:29:34.611401 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:34.611401 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:29:34.611401 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:29:34.611401 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:34.611401 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:29:34.611401 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:34.611401 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:29:34.611401 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:29:34.611401 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:34.612061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:29:34.612061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:34.612061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:29:34.612061 np0000173183 neutron-ovn-maintenance-worker[105581]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_for_inconsistencies finished in 0.338 seconds
May 09 14:29:34.994229 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:29:34.994885 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:29:34.995104 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:29:34.996939 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:29:35.027824 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:35.028214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:35.028640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:29:35.028640 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:35.028640 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:29:35.028640 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:29:35.028640 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:35.028640 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:29:35.028640 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:35.028640 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:29:35.029540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:35.029540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:29:35.029540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:35.029540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:29:35.029540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:29:35.029540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:29:35.029540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:29:35.029540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:35.029540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:35.029540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:35.029540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:29:35.029540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:35.030309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:35.030309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:35.030309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:29:35.030309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:35.030309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:29:35.030309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:29:35.030309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:35.030309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:29:35.030309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:35.030309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:29:35.030309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:29:35.030309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:35.030309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:29:35.030928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:35.030928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:29:35.030928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:35.030928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:29:35.030928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:29:35.030928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:29:35.030928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:29:35.030928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:29:35.030928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:35.030928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:29:35.030928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:29:35.030928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:29:35.030928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:29:35.030928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:29:35.030928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:35.031957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:35.031957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:29:35.031957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:29:35.031957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:29:35.031957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:29:35.031957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:35.031957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:35.031957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:35.031957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:29:35.031957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:35.031957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:35.031957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:35.031957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:29:35.031957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:35.031957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:29:35.031957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:29:35.031957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:35.031957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:29:35.031957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:35.033048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:29:35.033048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:29:35.033048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:35.033048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:29:35.033048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:35.033048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:29:35.203834 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:29:35.204812 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:29:35.204812 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:29:35.205766 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:35.206052 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:35.206289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:35.207180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:29:35.207180 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:35.207180 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:29:35.207180 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:29:35.207180 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:35.207180 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:29:35.207180 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:35.207180 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:29:35.208817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:35.208817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:29:35.208817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:35.208817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:29:35.208817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:29:35.208817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:29:35.208817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:29:35.208817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:35.208817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:35.208817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:35.208817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:29:35.208817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:35.209466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:35.209466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:35.209466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:29:35.209466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:35.209466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:29:35.209466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:29:35.209466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:35.209466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:29:35.209466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:35.209466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:29:35.209466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:29:35.209466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:35.209466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:29:35.210537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:35.210537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:29:35.210537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:35.210537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:29:35.210537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:29:35.210537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:29:35.210537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:29:35.210537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:29:35.210537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:35.210537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:29:35.210537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:29:35.210537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:29:35.210537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:29:35.210537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:29:35.210537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:35.211584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:35.211584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:29:35.211584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:29:35.211584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:29:35.211584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:29:35.211584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:35.211584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:35.211584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:35.211584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:29:35.211584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:35.211584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:35.211584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:35.211584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:29:35.211584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:35.211584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:29:35.211584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:29:35.211584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:35.211584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:29:35.211584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:35.212701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:29:35.212701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:29:35.212701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:35.212701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:29:35.212701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:35.212701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:29:35.213024 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:29:35.213156 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:29:35.213274 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:29:35.213427 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:29:35.213700 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:35.213865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:35.214327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:29:35.214327 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:35.214327 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:29:35.214327 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:29:35.214327 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:35.214327 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:29:35.214327 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:35.214327 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:29:35.215023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:35.215023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:29:35.215023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:29:35.215023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:29:35.215023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:35.215023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:29:35.215023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:35.215023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:29:35.215023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:35.215023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:29:35.215023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:35.215023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:29:35.215023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:35.215023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:29:35.215023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:29:35.215551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:35.215551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:29:35.215551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:35.215551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:29:35.215551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:29:35.215551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:35.215551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:29:35.215551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:35.215551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:29:35.215857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:35.215857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:29:35.215857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:35.215857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:29:35.215857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:29:35.215857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:29:35.215857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:29:35.215857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:29:35.215857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:29:35.215857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:29:35.215857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:35.215857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:35.216244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:35.216244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:29:35.216244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:35.216244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:35.216244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:35.216244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:29:35.216244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:35.216244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:29:35.216244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:29:35.216244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:35.216244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:29:35.216244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:35.216244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:29:35.216244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:29:35.216721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:35.216721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:29:35.216721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:35.216721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:29:35.216902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:35.216902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:29:35.216902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:29:35.216902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:29:35.216902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:29:35.216902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:29:35.216902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:35.216902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:29:35.216902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:29:35.216902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:29:35.216902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:29:35.216902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:29:35.216902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:35.216902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:35.216902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:29:35.216902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:29:35.217371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:29:35.217371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:29:35.217371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:29:35.217371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:29:35.217371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:29:35.217371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:35.217371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:35.217371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:35.217371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:29:35.217371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:35.217371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:35.217371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:35.217371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:29:35.217371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:35.217371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:29:35.217371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:29:35.217371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:35.217371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:29:35.217965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:35.217965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:29:35.217965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:29:35.217965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:35.217965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:29:35.217965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:35.217965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:29:35.237082 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:29:35.237625 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:29:35.237925 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:29:35.237925 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:29:35.238285 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:35.238668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:35.239321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:29:35.239321 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:35.239321 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:29:35.239321 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:29:35.239321 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:35.239321 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:29:35.239321 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:35.239321 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:29:35.240531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:35.240531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:29:35.240531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:29:35.240531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:29:35.240531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:35.240531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:29:35.240531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:35.240531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:29:35.240531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:35.240531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:29:35.240531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:35.240531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:29:35.240531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:35.240531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:29:35.240531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:29:35.241598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:35.241598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:29:35.241598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:35.241598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:29:35.241598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:29:35.241598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:35.241598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:29:35.241598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:35.241598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:29:35.242019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:35.242019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:29:35.242019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:35.242019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:29:35.242019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:29:35.242019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:29:35.242019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:29:35.242019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:29:35.242019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:29:35.242019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:35.242019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:35.242581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:35.242581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:29:35.242581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:35.242581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:35.242581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:35.242581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:29:35.242581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:35.242581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:29:35.242581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:29:35.242581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:35.242581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:29:35.242581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:35.242581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:29:35.242581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:29:35.243199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:35.243199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:29:35.243199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:35.243199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:29:35.243724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:35.243724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:29:35.243724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:29:35.243724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:29:35.243724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:29:35.243724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:29:35.243724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:35.243724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:29:35.243724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:29:35.243724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:29:35.243724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:29:35.243724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:29:35.243724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:35.243724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:35.243724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:29:35.243724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:29:35.243724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:29:35.244654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:29:35.244654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:29:35.244654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:29:35.244654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:35.244654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:35.244654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:35.244654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:29:35.244654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:35.244654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:35.244654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:35.244654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:29:35.244654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:35.244654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:29:35.244654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:29:35.244654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:35.244654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:29:35.244654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:35.244654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:29:35.244654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:29:35.245477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:35.245477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:29:35.245477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:35.245477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:29:35.245477 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:29:35.245477 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:29:35.245477 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:29:35.245477 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:29:35.246033 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:35.246130 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:35.246354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:35.247060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:29:35.247060 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:35.247060 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:29:35.247060 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:29:35.247060 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:35.247060 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:29:35.247060 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:35.247060 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:29:35.248415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:35.248415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:29:35.248415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:35.248415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:29:35.248415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:29:35.248415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:29:35.248415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:29:35.248415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:35.248415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:35.248415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:35.248415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:29:35.248415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:35.248775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:35.248775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:35.248775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:29:35.248775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:35.248775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:29:35.248775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:29:35.248775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:35.248775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:29:35.248775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:35.248775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:29:35.248775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:29:35.248775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:35.248775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:29:35.249109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:35.249109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:29:35.250135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:35.250135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:29:35.250135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:29:35.250135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:29:35.250135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:29:35.250135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:29:35.250135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:35.250135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:29:35.250135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:29:35.250135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:29:35.250135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:29:35.250135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:29:35.250135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:35.250135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:35.250135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:29:35.250135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:29:35.250622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:29:35.250622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:29:35.250622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:35.250622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:35.250622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:35.250622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:29:35.250622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:35.250622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:35.250622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:35.250622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:29:35.250622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:35.250622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:29:35.250622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:29:35.250622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:35.250622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:29:35.250622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:35.250622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:29:35.250622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:29:35.250622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:35.251135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:29:35.251135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:35.251135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:29:39.994803 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:29:39.995675 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:29:39.995900 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:29:39.997203 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:29:40.033506 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:40.033671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:40.034096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:29:40.034096 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:40.034096 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:29:40.034096 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:29:40.034096 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:40.034096 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:29:40.034096 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:40.034096 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:29:40.034767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:40.034767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:29:40.034767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:40.034767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:29:40.034767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:29:40.034767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:29:40.034767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:29:40.034767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:40.034767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:40.034767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:40.034767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:29:40.034767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:40.035386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:40.035386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:40.035386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:29:40.035386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:40.035386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:29:40.035386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:29:40.035386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:40.035386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:29:40.035386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:40.035386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:29:40.035386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:29:40.035386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:40.035386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:29:40.035984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:40.035984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:29:40.035984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:40.035984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:29:40.035984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:29:40.035984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:29:40.035984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:29:40.035984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:29:40.035984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:40.035984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:29:40.035984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:29:40.035984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:29:40.035984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:29:40.035984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:29:40.035984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:40.036668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:40.036668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:29:40.036668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:29:40.036668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:29:40.036668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:29:40.036668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:40.036668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:40.036668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:40.036668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:29:40.036668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:40.036668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:40.036668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:40.036668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:29:40.036668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:40.036668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:29:40.036668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:29:40.036668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:40.036668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:29:40.036668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:40.037488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:29:40.037488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:29:40.037488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:40.037488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:29:40.037488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:40.037488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:29:40.204715 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:29:40.205325 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:29:40.205631 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:29:40.206699 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:40.207051 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:40.207365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:40.208335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:29:40.208335 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:40.208335 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:29:40.208335 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:29:40.208335 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:40.208335 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:29:40.208335 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:40.208335 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:29:40.210106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:40.210106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:29:40.210106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:40.210106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:29:40.210106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:29:40.210106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:29:40.210106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:29:40.210106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:40.210106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:40.210106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:40.210106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:29:40.210106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:40.210968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:40.210968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:40.210968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:29:40.210968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:40.210968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:29:40.210968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:29:40.210968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:40.210968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:29:40.210968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:40.210968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:29:40.210968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:29:40.210968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:40.210968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:29:40.211534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:40.211534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:29:40.212102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:40.212102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:29:40.212102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:29:40.212102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:29:40.212102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:29:40.212102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:29:40.212102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:40.212102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:29:40.212102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:29:40.212102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:29:40.212102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:29:40.212102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:29:40.212102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:40.212102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:40.212102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:29:40.212102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:29:40.212838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:29:40.212838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:29:40.212838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:40.212838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:40.212838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:40.212838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:29:40.212838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:40.212838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:40.212838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:40.212838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:29:40.212838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:40.212838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:29:40.212838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:29:40.212838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:40.212838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:29:40.212838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:40.212838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:29:40.212838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:29:40.212838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:40.213703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:29:40.213703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:40.213703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:29:40.213703 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:29:40.213703 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:29:40.213946 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:29:40.213946 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:29:40.214252 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:40.214493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:40.214973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:29:40.214973 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:40.214973 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:29:40.214973 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:29:40.214973 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:40.214973 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:29:40.214973 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:40.214973 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:29:40.215862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:40.215862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:29:40.215862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:29:40.215862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:29:40.215862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:40.215862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:29:40.215862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:40.215862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:29:40.215862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:40.215862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:29:40.215862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:40.215862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:29:40.215862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:40.215862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:29:40.215862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:29:40.216614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:40.216614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:29:40.216614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:40.216614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:29:40.216614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:29:40.216614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:40.216614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:29:40.216614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:40.216614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:29:40.217030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:40.217030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:29:40.217030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:40.217030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:29:40.217030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:29:40.217030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:29:40.217030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:29:40.217030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:29:40.217030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:29:40.217030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:29:40.217030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:40.217030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:40.217546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:40.217546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:29:40.217546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:40.217546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:40.217546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:40.217546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:29:40.217546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:40.217546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:29:40.217546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:29:40.217546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:40.217546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:29:40.217546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:40.217546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:29:40.217546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:29:40.218099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:40.218099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:29:40.218099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:40.218099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:29:40.219923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:40.219923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:29:40.219923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:29:40.219923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:29:40.219923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:29:40.219923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:29:40.219923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:40.219923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:29:40.219923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:29:40.219923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:29:40.219923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:29:40.219923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:29:40.219923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:40.219923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:40.219923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:29:40.219923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:29:40.220719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:29:40.220719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:29:40.220719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:29:40.220719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:29:40.220719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:29:40.220719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:40.220719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:40.220719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:40.220719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:29:40.220719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:40.220719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:40.220719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:40.220719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:29:40.220719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:40.220719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:29:40.220719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:29:40.220719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:40.220719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:29:40.221211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:40.221211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:29:40.221211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:29:40.221211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:40.221211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:29:40.221211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:40.221211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:29:40.237754 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:29:40.238182 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:29:40.238422 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:29:40.238746 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:29:40.239187 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:40.239714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:40.240734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:29:40.240734 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:40.240734 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:29:40.240734 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:29:40.240734 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:40.240734 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:29:40.240734 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:40.240734 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:29:40.241753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:40.241753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:29:40.241753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:29:40.241753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:29:40.241753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:40.241753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:29:40.241753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:40.241753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:29:40.241753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:40.241753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:29:40.241753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:40.241753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:29:40.241753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:40.241753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:29:40.241753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:29:40.242158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:40.242158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:29:40.242158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:40.242158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:29:40.242158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:29:40.242158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:40.242158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:29:40.242158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:40.242158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:29:40.242950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:40.242950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:29:40.242950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:40.242950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:29:40.242950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:29:40.242950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:29:40.242950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:29:40.242950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:29:40.242950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:29:40.242950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:40.242950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:40.243277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:40.243277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:29:40.243277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:40.243277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:40.243277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:40.243277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:29:40.243277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:40.243277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:29:40.243277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:29:40.243277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:40.243277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:29:40.243277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:40.243277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:29:40.243277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:29:40.243685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:40.243685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:29:40.243685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:40.243685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:29:40.244078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:40.244078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:29:40.244078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:29:40.244078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:29:40.244078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:29:40.244078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:29:40.244078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:40.244078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:29:40.244078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:29:40.244078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:29:40.244078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:29:40.244078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:29:40.244078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:40.244078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:40.244078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:29:40.244078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:29:40.244078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:29:40.244590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:29:40.244590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:29:40.244590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:29:40.244590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:40.244590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:40.244590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:40.244590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:29:40.244590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:40.244590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:40.244590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:40.244590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:29:40.244590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:40.244590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:29:40.244590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:29:40.244590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:40.244590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:29:40.244590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:40.244590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:29:40.244590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:29:40.245068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:40.245068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:29:40.245068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:40.245068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:29:40.245174 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:29:40.245426 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:29:40.245555 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:29:40.245722 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:29:40.246232 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:40.246528 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:40.246747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:40.247218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:29:40.247218 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:40.247218 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:29:40.247218 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:29:40.247218 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:40.247218 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:29:40.247218 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:40.247218 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:29:40.248034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:40.248034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:29:40.248034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:40.248034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:29:40.248034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:29:40.248034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:29:40.248034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:29:40.248034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:40.248034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:40.248034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:40.248034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:29:40.248034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:40.248387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:40.248387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:40.248387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:29:40.248387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:40.248387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:29:40.248387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:29:40.248387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:40.248387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:29:40.248387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:40.248387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:29:40.248387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:29:40.248387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:40.248387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:29:40.248767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:40.248767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:29:40.248962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:40.248962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:29:40.248962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:29:40.248962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:29:40.248962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:29:40.248962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:29:40.248962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:40.248962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:29:40.248962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:29:40.248962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:29:40.248962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:29:40.248962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:29:40.248962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:40.248962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:40.248962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:29:40.248962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:29:40.249435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:29:40.249435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:29:40.249435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:40.249435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:40.249435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:40.249435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:29:40.249435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:40.249435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:40.249435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:40.249435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:29:40.249435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:40.249435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:29:40.249435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:29:40.249435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:40.249435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:29:40.249435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:40.249435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:29:40.249435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:29:40.249435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:40.249972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:29:40.249972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:40.249972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:29:44.995604 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:29:44.997488 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:29:44.997488 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:29:44.998474 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:29:45.030105 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:45.030264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:45.030994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:29:45.030994 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:45.030994 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:29:45.030994 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:29:45.030994 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:45.030994 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:29:45.030994 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:45.030994 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:29:45.031741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:45.031741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:29:45.031741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:45.031741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:29:45.031741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:29:45.031741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:29:45.031741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:29:45.031741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:45.031741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:45.031741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:45.031741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:29:45.031741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:45.032371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:45.032371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:45.032371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:29:45.032371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:45.032371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:29:45.032371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:29:45.032371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:45.032371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:29:45.032371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:45.032371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:29:45.032371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:29:45.032371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:45.032371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:29:45.033066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:45.033066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:29:45.033066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:45.033066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:29:45.033066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:29:45.033066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:29:45.033066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:29:45.033066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:29:45.033066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:45.033066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:29:45.033066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:29:45.033066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:29:45.033066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:29:45.033066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:29:45.033066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:45.033867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:45.033867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:29:45.033867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:29:45.033867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:29:45.033867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:29:45.033867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:45.033867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:45.033867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:45.033867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:29:45.033867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:45.033867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:45.033867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:45.033867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:29:45.033867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:45.033867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:29:45.033867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:29:45.033867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:45.033867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:29:45.033867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:45.034801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:29:45.034801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:29:45.034801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:45.034801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:29:45.034801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:45.034801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:29:45.205730 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:29:45.206218 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:29:45.206302 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:29:45.207158 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:45.207431 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:45.207589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:45.208195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:29:45.208195 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:45.208195 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:29:45.208195 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:29:45.208195 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:45.208195 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:29:45.208195 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:45.208195 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:29:45.208992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:45.208992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:29:45.208992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:45.208992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:29:45.208992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:29:45.208992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:29:45.208992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:29:45.208992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:45.208992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:45.208992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:45.208992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:29:45.208992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:45.209632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:45.209632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:45.209632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:29:45.209632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:45.209632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:29:45.209632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:29:45.209632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:45.209632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:29:45.209632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:45.209632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:29:45.209632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:29:45.209632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:45.209632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:29:45.210212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:45.210212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:29:45.210212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:45.210212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:29:45.210212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:29:45.210212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:29:45.210212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:29:45.210212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:29:45.210212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:45.210212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:29:45.210212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:29:45.210212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:29:45.210212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:29:45.210212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:29:45.210212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:45.211299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:45.211299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:29:45.211299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:29:45.211299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:29:45.211299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:29:45.211299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:45.211299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:45.211299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:45.211299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:29:45.211299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:45.211299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:45.211299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:45.211299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:29:45.211299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:45.211299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:29:45.211299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:29:45.211299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:45.211299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:29:45.211299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:45.212183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:29:45.212183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:29:45.212183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:45.212183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:29:45.212183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:45.212183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:29:45.213724 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:29:45.214105 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:29:45.214280 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:29:45.214495 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:29:45.214975 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:45.215266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:45.216062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:29:45.216062 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:45.216062 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:29:45.216062 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:29:45.216062 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:45.216062 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:29:45.216062 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:45.216062 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:29:45.217206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:45.217206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:29:45.217206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:29:45.217206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:29:45.217206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:45.217206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:29:45.217206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:45.217206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:29:45.217206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:45.217206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:29:45.217206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:45.217206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:29:45.217206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:45.217206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:29:45.217206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:29:45.217741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:45.217741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:29:45.217741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:45.217741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:29:45.217741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:29:45.217741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:45.217741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:29:45.217741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:45.217741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:29:45.219192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:45.219192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:29:45.219192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:45.219192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:29:45.219192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:29:45.219192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:29:45.219192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:29:45.219192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:29:45.219192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:29:45.219192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:29:45.219192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:45.219192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:45.220380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:45.220380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:29:45.220380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:45.220380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:45.220380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:45.220380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:29:45.220380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:45.220380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:29:45.220380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:29:45.220380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:45.220380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:29:45.220380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:45.220380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:29:45.220380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:29:45.220743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:45.220743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:29:45.220743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:45.220743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:29:45.220743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:45.220743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:29:45.220743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:29:45.220743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:29:45.220743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:29:45.220743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:29:45.220743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:45.220743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:29:45.220743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:29:45.220743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:29:45.221140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:29:45.221140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:29:45.221140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:45.221140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:45.221140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:29:45.221140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:29:45.221140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:29:45.221140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:29:45.221140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:29:45.221140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:29:45.221140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:29:45.221140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:45.221140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:45.221140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:45.221140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:29:45.221140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:45.221140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:45.221140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:45.221140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:29:45.221836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:45.221836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:29:45.221836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:29:45.221836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:45.221836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:29:45.221836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:45.221836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:29:45.221836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:29:45.221836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:45.221836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:29:45.221836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:45.221836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:29:45.238553 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:29:45.238884 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:29:45.238981 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:29:45.239115 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:29:45.239717 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:45.240099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:45.241121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:29:45.241121 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:45.241121 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:29:45.241121 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:29:45.241121 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:45.241121 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:29:45.241121 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:45.241121 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:29:45.242496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:45.242496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:29:45.242496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:29:45.242496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:29:45.242496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:45.242496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:29:45.242496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:45.242496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:29:45.242496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:45.242496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:29:45.242496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:45.242496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:29:45.242496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:45.242496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:29:45.242496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:29:45.243138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:45.243138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:29:45.243138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:45.243138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:29:45.243138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:29:45.243138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:45.243138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:29:45.243138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:45.243138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:29:45.243936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:45.243936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:29:45.243936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:45.243936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:29:45.243936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:29:45.243936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:29:45.243936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:29:45.243936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:29:45.243936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:29:45.243936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:45.243936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:45.244483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:45.244483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:29:45.244483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:45.244483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:45.244483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:45.244483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:29:45.244483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:45.244483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:29:45.244483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:29:45.244483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:45.244483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:29:45.244483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:45.244483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:29:45.244483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:29:45.245069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:45.245069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:29:45.245069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:45.245069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:29:45.245708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:45.245708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:29:45.245708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:29:45.245708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:29:45.245708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:29:45.245708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:29:45.245708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:45.245708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:29:45.245708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:29:45.245708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:29:45.245708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:29:45.245708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:29:45.245708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:45.245708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:45.245708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:29:45.245708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:29:45.245708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:29:45.246530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:29:45.246530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:29:45.246530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:29:45.246530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:45.246530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:45.246530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:45.246530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:29:45.246530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:45.246530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:45.246530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:45.246530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:29:45.246530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:45.246530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:29:45.246530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:29:45.246530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:45.246530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:29:45.246530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:45.246530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:29:45.246530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:29:45.247334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:45.247334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:29:45.247334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:45.247334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:29:45.247334 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:29:45.247334 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:29:45.247334 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:29:45.247334 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:29:45.247759 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:45.247759 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:45.247877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:45.248521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:29:45.248521 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:45.248521 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:29:45.248521 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:29:45.248521 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:45.248521 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:29:45.248521 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:45.248521 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:29:45.249758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:45.249758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:29:45.249758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:45.249758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:29:45.249758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:29:45.249758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:29:45.249758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:29:45.249758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:45.249758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:45.249758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:45.249758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:29:45.249758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:45.250382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:45.250382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:45.250382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:29:45.250382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:45.250382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:29:45.250382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:29:45.250382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:45.250382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:29:45.250382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:45.250382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:29:45.250382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:29:45.250382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:45.250382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:29:45.250756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:45.250756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:29:45.251352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:45.251352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:29:45.251352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:29:45.251352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:29:45.251352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:29:45.251352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:29:45.251352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:45.251352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:29:45.251352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:29:45.251352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:29:45.251352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:29:45.251352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:29:45.251352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:45.251352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:45.251352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:29:45.251352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:29:45.251816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:29:45.251816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:29:45.251816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:45.251816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:45.251816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:45.251816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:29:45.251816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:45.251816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:45.251816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:45.251816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:29:45.251816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:45.251816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:29:45.251816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:29:45.251816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:45.251816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:29:45.251816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:45.251816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:29:45.251816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:29:45.251816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:45.252333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:29:45.252333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:45.252333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:29:49.997064 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:29:49.997724 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:29:49.997891 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:29:49.999393 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:29:50.034131 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:50.034589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:50.035506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:29:50.035506 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:50.035506 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:29:50.035506 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:29:50.035506 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:50.035506 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:29:50.035506 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:50.035506 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:29:50.036268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:50.036268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:29:50.036268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:50.036268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:29:50.036268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:29:50.036268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:29:50.036268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:29:50.036268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:50.036268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:50.036268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:50.036268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:29:50.036268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:50.036716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:50.036716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:50.036716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:29:50.036716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:50.036716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:29:50.036716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:29:50.036716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:50.036716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:29:50.036716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:50.036716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:29:50.036716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:29:50.036716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:50.036716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:29:50.037168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:50.037168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:29:50.037168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:50.037168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:29:50.037168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:29:50.037168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:29:50.037168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:29:50.037168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:29:50.037168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:50.037168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:29:50.037168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:29:50.037168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:29:50.037168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:29:50.037168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:29:50.037168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:50.037763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:50.037763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:29:50.037763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:29:50.037763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:29:50.037763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:29:50.037763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:50.037763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:50.037763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:50.037763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:29:50.037763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:50.037763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:50.037763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:50.037763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:29:50.037763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:50.037763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:29:50.037763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:29:50.037763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:50.037763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:29:50.037763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:50.038330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:29:50.038330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:29:50.038330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:50.038330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:29:50.038330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:50.038330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:29:50.206583 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:29:50.207186 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:29:50.207373 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:29:50.208397 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:50.208662 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:50.208816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:50.209500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:29:50.209500 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:50.209500 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:29:50.209500 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:29:50.209500 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:50.209500 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:29:50.209500 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:50.209500 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:29:50.211062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:50.211062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:29:50.211062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:50.211062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:29:50.211062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:29:50.211062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:29:50.211062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:29:50.211062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:50.211062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:50.211062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:50.211062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:29:50.211062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:50.211662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:50.211662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:50.211662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:29:50.211662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:50.211662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:29:50.211662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:29:50.211662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:50.211662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:29:50.211662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:50.211662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:29:50.211662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:29:50.211662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:50.211662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:29:50.212223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:50.212223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:29:50.212795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:50.212795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:29:50.212795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:29:50.212795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:29:50.212795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:29:50.212795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:29:50.212795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:50.212795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:29:50.212795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:29:50.212795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:29:50.212795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:29:50.212795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:29:50.212795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:50.212795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:50.212795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:29:50.212795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:29:50.213545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:29:50.213545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:29:50.213545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:50.213545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:50.213545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:50.213545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:29:50.213545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:50.213545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:50.213545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:50.213545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:29:50.213545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:50.213545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:29:50.213545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:29:50.213545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:50.213545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:29:50.213545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:50.213545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:29:50.213545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:29:50.213545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:50.214319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:29:50.214319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:50.214319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:29:50.214319 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:29:50.214634 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:29:50.214808 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:29:50.214995 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:29:50.215507 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:50.215801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:50.216594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:29:50.216594 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:50.216594 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:29:50.216594 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:29:50.216594 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:50.216594 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:29:50.216594 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:50.216594 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:29:50.217761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:50.217761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:29:50.217761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:29:50.217761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:29:50.217761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:50.217761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:29:50.217761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:50.217761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:29:50.217761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:50.217761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:29:50.217761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:50.217761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:29:50.217761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:50.217761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:29:50.217761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:29:50.218266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:50.218266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:29:50.218266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:50.218266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:29:50.218266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:29:50.218266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:50.218266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:29:50.218266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:50.218266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:29:50.219095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:50.219095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:29:50.219095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:50.219095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:29:50.219095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:29:50.219095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:29:50.219095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:29:50.219095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:29:50.219095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:29:50.219095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:29:50.219095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:50.219095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:50.219493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:50.219493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:29:50.219493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:50.219493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:50.219493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:50.219493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:29:50.219493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:50.219493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:29:50.219493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:29:50.219493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:50.219493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:29:50.219493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:50.219493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:29:50.219493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:29:50.219879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:50.219879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:29:50.219879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:50.219879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:29:50.220319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:50.220319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:29:50.220319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:29:50.220319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:29:50.220319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:29:50.220319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:29:50.220319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:50.220319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:29:50.220319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:29:50.220319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:29:50.220319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:29:50.220319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:29:50.220319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:50.220319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:50.220319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:29:50.220319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:29:50.220787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:29:50.220787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:29:50.220787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:29:50.220787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:29:50.220787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:29:50.220787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:50.220787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:50.220787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:50.220787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:29:50.220787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:50.220787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:50.220787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:50.220787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:29:50.220787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:50.220787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:29:50.220787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:29:50.220787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:50.220787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:29:50.221300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:50.221300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:29:50.221300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:29:50.221300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:50.221300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:29:50.221300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:50.221300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:29:50.239087 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:29:50.239432 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:29:50.239664 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:29:50.239858 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:29:50.240356 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:50.240758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:50.241513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:29:50.241513 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:50.241513 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:29:50.241513 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:29:50.241513 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:50.241513 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:29:50.241513 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:50.241513 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:29:50.242646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:50.242646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:29:50.242646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:29:50.242646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:29:50.242646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:50.242646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:29:50.242646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:50.242646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:29:50.242646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:50.242646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:29:50.242646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:50.242646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:29:50.242646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:50.242646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:29:50.242646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:29:50.243340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:50.243340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:29:50.243340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:50.243340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:29:50.243340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:29:50.243340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:50.243340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:29:50.243340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:50.243340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:29:50.244032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:50.244032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:29:50.244032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:50.244032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:29:50.244032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:29:50.244032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:29:50.244032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:29:50.244032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:29:50.244032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:29:50.244032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:50.244032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:50.244594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:50.244594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:29:50.244594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:50.244594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:50.244594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:50.244594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:29:50.244594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:50.244594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:29:50.244594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:29:50.244594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:50.244594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:29:50.244594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:50.244594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:29:50.244594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:29:50.245227 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:50.245227 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:29:50.245227 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:50.245227 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:29:50.245679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:50.245679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:29:50.245679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:29:50.245679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:29:50.245679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:29:50.245679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:29:50.245679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:50.245679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:29:50.245679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:29:50.245679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:29:50.245679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:29:50.245679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:29:50.245679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:50.245679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:50.245679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:29:50.245679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:29:50.245679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:29:50.246472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:29:50.246472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:29:50.246472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:29:50.246472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:50.246472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:50.246472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:50.246472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:29:50.246472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:50.246472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:50.246472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:50.246472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:29:50.246472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:50.246472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:29:50.246472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:29:50.246472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:50.246472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:29:50.246472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:50.246472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:29:50.246472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:29:50.247393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:50.247393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:29:50.247393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:50.247393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:29:50.247393 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:29:50.247393 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:29:50.247393 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:29:50.247393 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:29:50.247731 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:50.247883 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:50.248101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:50.248893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:29:50.248893 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:50.248893 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:29:50.248893 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:29:50.248893 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:50.248893 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:29:50.248893 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:50.248893 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:29:50.250157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:50.250157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:29:50.250157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:50.250157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:29:50.250157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:29:50.250157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:29:50.250157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:29:50.250157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:50.250157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:50.250157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:50.250157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:29:50.250157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:50.250578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:50.250578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:50.250578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:29:50.250578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:50.250578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:29:50.250578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:29:50.250578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:50.250578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:29:50.250578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:50.250578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:29:50.250578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:29:50.250578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:50.250578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:29:50.250947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:50.250947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:29:50.251655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:50.251655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:29:50.251655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:29:50.251655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:29:50.251655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:29:50.251655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:29:50.251655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:50.251655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:29:50.251655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:29:50.251655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:29:50.251655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:29:50.251655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:29:50.251655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:50.251655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:50.251655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:29:50.251655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:29:50.252229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:29:50.252229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:29:50.252229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:50.252229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:50.252229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:50.252229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:29:50.252229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:50.252229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:50.252229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:50.252229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:29:50.252229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:50.252229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:29:50.252229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:29:50.252229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:50.252229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:29:50.252229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:50.252229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:29:50.252229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:29:50.252229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:50.252764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:29:50.252764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:50.252764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:29:54.997671 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:29:54.999227 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:29:54.999227 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:29:54.999785 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:29:55.036181 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:55.036362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:55.036989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:29:55.036989 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:55.036989 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:29:55.036989 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:29:55.036989 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:55.036989 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:29:55.036989 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:55.036989 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:29:55.037881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:55.037881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:29:55.037881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:55.037881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:29:55.037881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:29:55.037881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:29:55.037881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:29:55.037881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:55.037881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:55.037881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:55.037881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:29:55.037881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:55.038752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:55.038752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:55.038752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:29:55.038752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:55.038752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:29:55.038752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:29:55.038752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:55.038752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:29:55.038752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:55.038752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:29:55.038752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:29:55.038752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:55.038752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:29:55.039571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:55.039571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:29:55.039571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:55.039571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:29:55.039571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:29:55.039571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:29:55.039571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:29:55.039571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:29:55.039571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:55.039571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:29:55.039571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:29:55.039571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:29:55.039571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:29:55.039571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:29:55.039571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:55.041253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:55.041253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:29:55.041253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:29:55.041253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:29:55.041253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:29:55.041253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:55.041253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:55.041253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:55.041253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:29:55.041253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:55.041253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:55.041253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:55.041253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:29:55.041253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:55.041253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:29:55.041253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:29:55.041253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:55.041253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:29:55.041253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:55.042343 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:29:55.042343 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:29:55.042343 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:55.042343 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:29:55.042343 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:55.042343 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:29:55.207275 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:29:55.207887 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:29:55.208086 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:29:55.209050 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:55.209344 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:55.209628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:55.210495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:29:55.210495 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:55.210495 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:29:55.210495 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:29:55.210495 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:55.210495 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:29:55.210495 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:55.210495 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:29:55.211920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:55.211920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:29:55.211920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:55.211920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:29:55.211920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:29:55.211920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:29:55.211920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:29:55.211920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:55.211920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:55.211920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:55.211920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:29:55.211920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:55.212238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:55.212238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:55.212238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:29:55.212238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:55.212238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:29:55.212238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:29:55.212238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:55.212238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:29:55.212238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:55.212238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:29:55.212238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:29:55.212238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:55.212238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:29:55.212493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:55.212493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:29:55.213561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:55.213561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:29:55.213561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:29:55.213561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:29:55.213561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:29:55.213561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:29:55.213561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:55.213561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:29:55.213561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:29:55.213561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:29:55.213561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:29:55.213561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:29:55.213561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:55.213561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:55.213561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:29:55.213561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:29:55.213947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:29:55.213947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:29:55.213947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:55.213947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:55.213947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:55.213947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:29:55.213947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:55.213947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:55.213947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:55.213947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:29:55.213947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:55.213947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:29:55.213947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:29:55.213947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:55.213947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:29:55.213947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:55.213947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:29:55.213947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:29:55.213947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:55.214318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:29:55.214318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:55.214318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:29:55.214607 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:29:55.214947 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:29:55.215030 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:29:55.215165 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:29:55.215684 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:55.215963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:55.216680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:29:55.216680 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:55.216680 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:29:55.216680 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:29:55.216680 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:55.216680 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:29:55.216680 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:55.216680 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:29:55.217352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:55.217352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:29:55.217352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:29:55.217352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:29:55.217352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:55.217352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:29:55.217352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:55.217352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:29:55.217352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:55.217352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:29:55.217352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:55.217352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:29:55.217352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:55.217352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:29:55.217352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:29:55.218525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:55.218525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:29:55.218525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:55.218525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:29:55.218525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:29:55.218525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:55.218525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:29:55.218525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:55.218525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:29:55.218525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:55.218525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:29:55.218525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:55.218525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:29:55.218525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:29:55.219634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:29:55.219634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:29:55.219634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:29:55.219634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:29:55.219634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:29:55.219634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:55.219634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:55.219634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:55.219634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:29:55.219634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:55.219634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:55.219634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:55.219634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:29:55.220199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:55.220199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:29:55.220199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:29:55.220199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:55.220199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:29:55.220199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:55.220199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:29:55.220199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:29:55.220199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:55.220199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:29:55.220199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:55.220199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:29:55.220683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:55.220683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:29:55.220683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:29:55.220683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:29:55.220683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:29:55.220683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:29:55.220683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:55.220683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:29:55.220683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:29:55.220683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:29:55.220683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:29:55.220683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:29:55.220683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:55.220683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:55.220683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:29:55.220683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:29:55.221150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:29:55.221150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:29:55.221150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:29:55.221150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:29:55.221150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:29:55.221150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:55.221150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:55.221150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:55.221150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:29:55.221150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:55.221150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:55.221150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:55.221150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:29:55.221150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:55.221150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:29:55.221150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:29:55.221150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:55.221150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:29:55.221671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:55.221671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:29:55.221671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:29:55.221671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:55.221671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:29:55.221671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:55.221671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:29:55.239782 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:29:55.240267 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:29:55.240428 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:29:55.240658 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:29:55.241438 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:55.241878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:55.242921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:29:55.242921 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:55.242921 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:29:55.242921 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:29:55.242921 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:55.242921 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:29:55.242921 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:55.242921 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:29:55.244102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:55.244102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:29:55.244102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:29:55.244102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:29:55.244102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:55.244102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:29:55.244102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:55.244102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:29:55.244102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:55.244102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:29:55.244102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:55.244102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:29:55.244102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:55.244102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:29:55.244102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:29:55.245130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:55.245130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:29:55.245130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:55.245130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:29:55.245130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:29:55.245130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:55.245130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:29:55.245130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:55.245130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:29:55.245728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:55.245728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:29:55.245728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:55.245728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:29:55.245728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:29:55.245728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:29:55.245728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:29:55.245728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:29:55.245728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:29:55.245728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:55.245728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:55.246474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:55.246474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:29:55.246474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:55.246474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:55.246474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:55.246474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:29:55.246474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:55.246474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:29:55.246474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:29:55.246474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:55.246474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:29:55.246474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:55.246474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:29:55.246474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:29:55.247379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:55.247379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:29:55.247379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:55.247379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:29:55.247379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:55.247379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:29:55.247379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:29:55.247379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:29:55.247379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:29:55.247379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:29:55.247379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:55.247379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:29:55.247379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:29:55.247379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:29:55.248456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:29:55.248456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:29:55.248456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:55.248456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:55.248456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:29:55.248456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:29:55.248456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:29:55.248456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:29:55.248456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:29:55.248456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:29:55.248456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:55.248456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:55.248456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:55.248456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:29:55.248456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:55.248456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:55.248456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:55.248456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:29:55.248456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:55.249756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:29:55.249756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:29:55.249756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:55.249756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:29:55.249756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:55.249756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:29:55.249756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:29:55.249756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:55.249756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:29:55.249756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:55.249756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:29:55.249756 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:29:55.249756 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:29:55.250702 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:29:55.250702 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:29:55.250702 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:55.250702 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:29:55.250702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:55.250702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:29:55.250702 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:55.250702 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:29:55.250702 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:29:55.251346 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:55.251346 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:29:55.251346 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:55.251346 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:29:55.251346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:55.251346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:29:55.251346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:55.251346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:29:55.251346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:29:55.251346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:29:55.251346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:29:55.251346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:55.251346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:55.251346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:55.251970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:29:55.251970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:55.251970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:29:55.251970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:55.251970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:29:55.251970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:55.251970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:29:55.251970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:29:55.251970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:55.251970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:29:55.251970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:55.251970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:29:55.251970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:29:55.251970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:55.252356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:29:55.252356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:55.252356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:29:55.252356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:55.252356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:29:55.252356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:29:55.252356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:29:55.252356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:29:55.252356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:29:55.252356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:55.252356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:29:55.252356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:29:55.252356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:29:55.252356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:29:55.252356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:29:55.252356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:29:55.252829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:29:55.252829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:29:55.252829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:29:55.252829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:29:55.252829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:29:55.252829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:55.252829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:55.252829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:29:55.252829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:29:55.252829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:29:55.252829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:29:55.252829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:29:55.252829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:29:55.252829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:29:55.252829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:29:55.252829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:29:55.252829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:29:55.252829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:29:55.252829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:29:55.253342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:29:55.253342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:29:55.253342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:29:55.253342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:29:55.253342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:29:55.253342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:30:00.000832 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:30:00.001730 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:30:00.001730 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:30:00.002370 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:30:00.037956 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:30:00.038177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:00.038744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:30:00.038744 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:00.038744 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:30:00.038744 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:30:00.038744 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:00.038744 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:30:00.038744 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:00.038744 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:30:00.039634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:00.039634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:30:00.039634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:00.039634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:30:00.039634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:30:00.039634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:30:00.039634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:30:00.039634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:00.039634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:00.039634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:00.039634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:30:00.039634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:00.040623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:00.040623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:00.040623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:30:00.040623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:00.040623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:30:00.040623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:30:00.040623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:00.040623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:30:00.040623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:00.040623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:30:00.040623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:30:00.040623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:00.040623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:30:00.041947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:00.041947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:30:00.041947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:00.041947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:30:00.041947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:30:00.041947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:30:00.041947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:30:00.041947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:30:00.041947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:00.041947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:30:00.041947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:30:00.041947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:30:00.041947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:30:00.041947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:30:00.041947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:00.043173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:00.043173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:30:00.043173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:30:00.043173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:30:00.043173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:30:00.043173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:00.043173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:00.043173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:00.043173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:30:00.043173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:00.043173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:00.043173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:00.043173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:30:00.043173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:00.043173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:30:00.043173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:30:00.043173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:00.043173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:30:00.043173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:00.044311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:30:00.044311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:30:00.044311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:00.044311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:30:00.044311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:00.044311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:30:00.208102 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:30:00.208640 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:30:00.208811 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:30:00.209721 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:30:00.209975 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:30:00.210190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:00.210977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:30:00.210977 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:00.210977 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:30:00.210977 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:30:00.210977 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:00.210977 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:30:00.210977 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:00.210977 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:30:00.212306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:00.212306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:30:00.212306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:00.212306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:30:00.212306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:30:00.212306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:30:00.212306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:30:00.212306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:00.212306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:00.212306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:00.212306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:30:00.212306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:00.212949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:00.212949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:00.212949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:30:00.212949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:00.212949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:30:00.212949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:30:00.212949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:00.212949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:30:00.212949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:00.212949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:30:00.212949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:30:00.212949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:00.212949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:30:00.213541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:00.213541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:30:00.214083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:00.214083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:30:00.214083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:30:00.214083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:30:00.214083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:30:00.214083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:30:00.214083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:00.214083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:30:00.214083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:30:00.214083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:30:00.214083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:30:00.214083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:30:00.214083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:00.214083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:00.214083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:30:00.214083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:30:00.214859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:30:00.214859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:30:00.214859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:00.214859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:00.214859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:00.214859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:30:00.214859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:00.214859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:00.214859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:00.214859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:30:00.214859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:00.214859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:30:00.214859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:30:00.214859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:00.214859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:30:00.214859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:00.214859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:30:00.214859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:30:00.214859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:00.215731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:30:00.215731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:00.215731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:30:00.216219 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:30:00.216440 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:30:00.216555 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:30:00.216691 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:30:00.216953 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:30:00.217114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:00.217601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:30:00.217601 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:00.217601 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:30:00.217601 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:30:00.217601 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:00.217601 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:30:00.217601 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:00.217601 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:30:00.218305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:00.218305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:30:00.218305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:30:00.218305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:30:00.218305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:00.218305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:30:00.218305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:00.218305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:30:00.218305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:00.218305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:30:00.218305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:00.218305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:30:00.218305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:00.218305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:30:00.218305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:30:00.218772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:00.218772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:30:00.218772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:00.218772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:30:00.218772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:30:00.218772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:00.218772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:30:00.218772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:00.218772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:30:00.219150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:00.219150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:30:00.219150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:00.219150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:30:00.219150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:30:00.219150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:30:00.219150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:30:00.219150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:30:00.219150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:30:00.219150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:30:00.219150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:00.219150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:00.219509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:00.219509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:30:00.219509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:00.219509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:00.219509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:00.219509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:30:00.219509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:00.219509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:30:00.219509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:30:00.219509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:00.219509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:30:00.219509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:00.219509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:30:00.219509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:30:00.219859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:00.219859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:30:00.219859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:00.219859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:30:00.220251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:00.220251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:30:00.220251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:30:00.220251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:30:00.220251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:30:00.220251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:30:00.220251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:00.220251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:30:00.220251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:30:00.220251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:30:00.220251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:30:00.220251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:30:00.220251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:00.220251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:00.220251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:30:00.220251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:30:00.220768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:30:00.220768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:30:00.220768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:30:00.220768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:30:00.220768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:30:00.220768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:00.220768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:00.220768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:00.220768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:30:00.220768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:00.220768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:00.220768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:00.220768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:30:00.220768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:00.220768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:30:00.220768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:30:00.220768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:00.220768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:30:00.221274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:00.221274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:30:00.221274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:30:00.221274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:00.221274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:30:00.221274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:00.221274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:30:00.240486 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:30:00.240824 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:30:00.241062 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:30:00.241328 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:30:00.241799 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:30:00.242215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:00.242936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:30:00.242936 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:00.242936 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:30:00.242936 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:30:00.242936 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:00.242936 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:30:00.242936 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:00.242936 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:30:00.244027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:00.244027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:30:00.244027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:30:00.244027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:30:00.244027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:00.244027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:30:00.244027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:00.244027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:30:00.244027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:00.244027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:30:00.244027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:00.244027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:30:00.244027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:00.244027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:30:00.244027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:30:00.244737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:00.244737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:30:00.244737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:00.244737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:30:00.244737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:30:00.244737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:00.244737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:30:00.244737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:00.244737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:30:00.245427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:00.245427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:30:00.245427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:00.245427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:30:00.245427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:30:00.245427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:30:00.245427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:30:00.245427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:30:00.245427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:30:00.245427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:00.245427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:00.245971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:00.245971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:30:00.245971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:00.245971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:00.245971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:00.245971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:30:00.245971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:00.245971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:30:00.245971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:30:00.245971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:00.245971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:30:00.245971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:00.245971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:30:00.245971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:30:00.246535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:00.246535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:30:00.246535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:00.246535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:30:00.247219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:00.247219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:30:00.247219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:30:00.247219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:30:00.247219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:30:00.247219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:30:00.247219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:00.247219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:30:00.247219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:30:00.247219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:30:00.247219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:30:00.247219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:30:00.247219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:00.247219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:00.247219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:30:00.247219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:30:00.247219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:30:00.248054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:30:00.248054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:30:00.248054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:30:00.248054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:00.248054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:00.248054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:00.248054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:30:00.248054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:00.248054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:00.248054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:00.248054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:30:00.248054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:00.248054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:30:00.248054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:30:00.248054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:00.248054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:30:00.248054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:00.248054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:30:00.248054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:30:00.248888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:00.248888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:30:00.248888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:00.248888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:30:00.249526 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:30:00.249788 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:30:00.249950 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:30:00.250159 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:30:00.250525 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:30:00.250899 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:30:00.251060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:00.251735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:30:00.251735 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:00.251735 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:30:00.251735 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:30:00.251735 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:00.251735 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:30:00.251735 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:00.251735 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:30:00.252945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:00.252945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:30:00.252945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:00.252945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:30:00.252945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:30:00.252945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:30:00.252945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:30:00.252945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:00.252945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:00.252945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:00.252945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:30:00.252945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:00.253309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:00.253309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:00.253309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:30:00.253309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:00.253309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:30:00.253309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:30:00.253309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:00.253309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:30:00.253309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:00.253309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:30:00.253309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:30:00.253309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:00.253309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:30:00.253675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:00.253675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:30:00.254392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:00.254392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:30:00.254392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:30:00.254392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:30:00.254392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:30:00.254392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:30:00.254392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:00.254392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:30:00.254392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:30:00.254392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:30:00.254392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:30:00.254392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:30:00.254392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:00.254392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:00.254392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:30:00.254392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:30:00.254883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:30:00.254883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:30:00.254883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:00.254883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:00.254883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:00.254883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:30:00.254883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:00.254883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:00.254883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:00.254883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:30:00.254883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:00.254883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:30:00.254883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:30:00.254883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:00.254883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:30:00.254883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:00.254883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:30:00.254883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:30:00.254883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:00.255540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:30:00.255540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:00.255540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:30:05.001351 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:30:05.001890 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:30:05.001979 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:30:05.003184 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:30:05.034743 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:30:05.034906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:05.035319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:30:05.035319 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:05.035319 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:30:05.035319 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:30:05.035319 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:05.035319 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:30:05.035319 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:05.035319 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:30:05.036012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:05.036012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:30:05.036012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:05.036012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:30:05.036012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:30:05.036012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:30:05.036012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:30:05.036012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:05.036012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:05.036012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:05.036012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:30:05.036012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:05.036707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:05.036707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:05.036707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:30:05.036707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:05.036707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:30:05.036707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:30:05.036707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:05.036707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:30:05.036707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:05.036707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:30:05.036707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:30:05.036707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:05.036707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:30:05.037444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:05.037444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:30:05.037444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:05.037444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:30:05.037444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:30:05.037444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:30:05.037444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:30:05.037444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:30:05.037444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:05.037444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:30:05.037444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:30:05.037444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:30:05.037444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:30:05.037444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:30:05.037444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:05.038233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:05.038233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:30:05.038233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:30:05.038233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:30:05.038233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:30:05.038233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:05.038233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:05.038233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:05.038233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:30:05.038233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:05.038233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:05.038233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:05.038233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:30:05.038233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:05.038233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:30:05.038233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:30:05.038233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:05.038233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:30:05.038233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:05.039190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:30:05.039190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:30:05.039190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:05.039190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:30:05.039190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:05.039190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:30:05.208627 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:30:05.209203 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:30:05.209395 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:30:05.210304 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:30:05.210597 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:30:05.210820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:05.211654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:30:05.211654 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:05.211654 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:30:05.211654 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:30:05.211654 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:05.211654 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:30:05.211654 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:05.211654 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:30:05.213049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:05.213049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:30:05.213049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:05.213049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:30:05.213049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:30:05.213049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:30:05.213049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:30:05.213049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:05.213049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:05.213049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:05.213049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:30:05.213049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:05.213967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:05.213967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:05.213967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:30:05.213967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:05.213967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:30:05.213967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:30:05.213967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:05.213967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:30:05.213967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:05.213967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:30:05.213967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:30:05.213967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:05.213967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:30:05.214809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:05.214809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:30:05.214809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:05.214809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:30:05.214809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:30:05.214809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:30:05.214809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:30:05.214809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:30:05.214809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:05.214809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:30:05.214809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:30:05.214809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:30:05.214809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:30:05.214809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:30:05.214809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:05.215843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:05.215843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:30:05.215843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:30:05.215843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:30:05.215843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:30:05.215843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:05.215843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:05.215843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:05.215843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:30:05.215843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:05.215843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:05.215843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:05.215843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:30:05.215843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:05.215843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:30:05.215843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:30:05.215843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:05.215843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:30:05.215843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:05.217085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:30:05.217085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:30:05.217085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:05.217085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:30:05.217085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:05.217085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:30:05.217085 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:30:05.217085 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:30:05.217085 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:30:05.217085 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:30:05.217852 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:30:05.217852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:05.217852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:30:05.217852 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:05.217852 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:30:05.217852 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:30:05.217852 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:05.217852 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:30:05.217852 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:05.217852 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:30:05.218497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:05.218497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:30:05.218497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:30:05.218497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:30:05.218497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:05.218497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:30:05.218497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:05.218497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:30:05.218497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:05.218497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:30:05.218497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:05.218497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:30:05.218497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:05.218497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:30:05.218497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:30:05.219600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:05.219600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:30:05.219600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:05.219600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:30:05.219600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:30:05.219600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:05.219600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:30:05.219600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:05.219600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:30:05.219600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:05.219600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:30:05.219600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:05.219600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:30:05.219600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:30:05.220942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:30:05.220942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:30:05.220942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:30:05.220942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:30:05.220942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:30:05.220942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:05.220942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:05.220942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:05.220942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:30:05.220942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:05.220942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:05.220942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:05.220942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:30:05.221604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:05.221604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:30:05.221604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:30:05.221604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:05.221604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:30:05.221604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:05.221604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:30:05.221604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:30:05.221604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:05.221604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:30:05.221604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:05.221604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:30:05.221961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:05.221961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:30:05.221961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:30:05.221961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:30:05.221961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:30:05.221961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:30:05.221961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:05.221961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:30:05.221961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:30:05.221961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:30:05.221961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:30:05.221961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:30:05.221961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:05.221961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:05.221961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:30:05.221961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:30:05.222442 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:30:05.222442 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:30:05.222442 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:30:05.222442 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:30:05.222442 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:30:05.222442 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:05.222442 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:05.222442 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:05.222442 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:30:05.222442 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:05.222442 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:05.222442 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:05.222442 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:30:05.222442 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:05.222442 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:30:05.222442 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:30:05.222442 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:05.222442 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:30:05.222980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:05.222980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:30:05.222980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:30:05.222980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:05.222980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:30:05.222980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:05.222980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:30:05.240897 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:30:05.241284 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:30:05.241510 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:30:05.241701 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:30:05.242051 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:30:05.242396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:05.243303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:30:05.243303 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:05.243303 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:30:05.243303 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:30:05.243303 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:05.243303 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:30:05.243303 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:05.243303 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:30:05.244529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:05.244529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:30:05.244529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:30:05.244529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:30:05.244529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:05.244529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:30:05.244529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:05.244529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:30:05.244529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:05.244529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:30:05.244529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:05.244529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:30:05.244529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:05.244529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:30:05.244529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:30:05.245208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:05.245208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:30:05.245208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:05.245208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:30:05.245208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:30:05.245208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:05.245208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:30:05.245208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:05.245208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:30:05.245885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:05.245885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:30:05.245885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:05.245885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:30:05.245885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:30:05.245885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:30:05.245885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:30:05.245885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:30:05.245885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:30:05.245885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:05.245885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:05.246683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:05.246683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:30:05.246683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:05.246683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:05.246683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:05.246683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:30:05.246683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:05.246683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:30:05.246683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:30:05.246683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:05.246683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:30:05.246683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:05.246683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:30:05.246683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:30:05.247641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:05.247641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:30:05.247641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:05.247641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:30:05.248328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:05.248328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:30:05.248328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:30:05.248328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:30:05.248328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:30:05.248328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:30:05.248328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:05.248328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:30:05.248328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:30:05.248328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:30:05.248328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:30:05.248328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:30:05.248328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:05.248328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:05.248328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:30:05.248328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:30:05.248328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:30:05.249242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:30:05.249242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:30:05.249242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:30:05.249242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:05.249242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:05.249242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:05.249242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:30:05.249242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:05.249242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:05.249242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:05.249242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:30:05.249242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:05.249242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:30:05.249242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:30:05.249242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:05.249242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:30:05.249242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:05.249242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:30:05.249242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:30:05.250276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:05.250276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:30:05.250276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:05.250276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:30:05.250276 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:30:05.250276 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:30:05.250276 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:30:05.250276 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:30:05.250933 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:30:05.251042 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:30:05.251280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:05.251999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:30:05.251999 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:05.251999 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:30:05.251999 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:30:05.251999 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:05.251999 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:30:05.251999 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:05.251999 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:30:05.252776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:05.252776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:30:05.252776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:05.252776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:30:05.252776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:30:05.252776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:30:05.252776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:30:05.252776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:05.252776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:05.252776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:05.252776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:30:05.252776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:05.253733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:05.253733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:05.253733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:30:05.253733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:05.253733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:30:05.253733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:30:05.253733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:05.253733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:30:05.253733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:05.253733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:30:05.253733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:30:05.253733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:05.253733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:30:05.254648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:05.254648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:30:05.254648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:05.254648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:30:05.254648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:30:05.254648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:30:05.254648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:30:05.254648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:30:05.254648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:05.254648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:30:05.254648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:30:05.254648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:30:05.254648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:30:05.254648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:30:05.254648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:05.255775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:05.255775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:30:05.255775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:30:05.255775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:30:05.255775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:30:05.255775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:05.255775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:05.255775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:05.255775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:30:05.255775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:05.255775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:05.255775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:05.255775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:30:05.255775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:05.255775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:30:05.255775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:30:05.255775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:05.255775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:30:05.255775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:05.257205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:30:05.257205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:30:05.257205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:05.257205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:30:05.257205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:05.257205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:30:10.001861 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:30:10.002878 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:30:10.002878 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:30:10.003094 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:30:10.030074 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:30:10.030690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:10.031590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:30:10.031590 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:10.031590 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:30:10.031590 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:30:10.031590 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:10.031590 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:30:10.031590 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:10.031590 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:30:10.032305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.029 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:10.032305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:30:10.032305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:10.032305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:30:10.032305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:30:10.032305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:30:10.032305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:30:10.032305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:10.032305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:10.032305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:10.032305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:30:10.032305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:10.033282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:10.033282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:10.033282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:30:10.033282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:10.033282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:30:10.033282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:30:10.033282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:10.033282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:30:10.033282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:10.033282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:30:10.033282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:30:10.033282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:10.033282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:30:10.033950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:10.033950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:30:10.033950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:10.033950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:30:10.033950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:30:10.033950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:30:10.033950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:30:10.033950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:30:10.033950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:10.033950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:30:10.033950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:30:10.033950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:30:10.033950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:30:10.033950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:30:10.033950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:10.034722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:10.034722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:30:10.034722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:30:10.034722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:30:10.034722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:30:10.034722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:10.034722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:10.034722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:10.034722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:30:10.034722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:10.034722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:10.034722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:10.034722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:30:10.034722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:10.034722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:30:10.034722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:30:10.034722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:10.034722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:30:10.034722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:10.036542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:30:10.036542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:30:10.036542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:10.036542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:30:10.036542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:10.036542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:30:10.209129 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:30:10.209765 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:30:10.209930 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:30:10.211119 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:30:10.211565 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:30:10.211831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:10.212813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:30:10.212813 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:10.212813 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:30:10.212813 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:30:10.212813 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:10.212813 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:30:10.212813 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:10.212813 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:30:10.214296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:10.214296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:30:10.214296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:10.214296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:30:10.214296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:30:10.214296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:30:10.214296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:30:10.214296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:10.214296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:10.214296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:10.214296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:30:10.214296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:10.215847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:10.215847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:10.215847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:30:10.215847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:10.215847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:30:10.215847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:30:10.215847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:10.215847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:30:10.215847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:10.215847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:30:10.215847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:30:10.215847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:10.215847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:30:10.216685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:10.216685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:30:10.216685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:10.216685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:30:10.216685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:30:10.216685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:30:10.216685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:30:10.216685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:30:10.216685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:10.216685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:30:10.216685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:30:10.216685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:30:10.216685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:30:10.216685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:30:10.216685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:10.217309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:10.217309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:30:10.217309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:30:10.217309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:30:10.217309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:30:10.217309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:10.217309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:10.217309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:10.217309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:30:10.217309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:10.217309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:10.217309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:10.217309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:30:10.217309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:10.217309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:30:10.217309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:30:10.217309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:10.217309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:30:10.217309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:10.219074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:30:10.219074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:30:10.219074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:10.219074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:30:10.219074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:10.219074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:30:10.219074 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:30:10.219074 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:30:10.219074 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:30:10.219074 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:30:10.220068 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:30:10.220068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:10.220068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:30:10.220068 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:10.220068 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:30:10.220068 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:30:10.220068 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:10.220068 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:30:10.220068 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:10.220068 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:30:10.220068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:10.220068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:30:10.220068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:30:10.220068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:30:10.221223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:10.221223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:30:10.221223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:10.221223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:30:10.221223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:10.221223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:30:10.221223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:10.221223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:30:10.221223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:10.221223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:30:10.221223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:30:10.221223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:10.221223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:30:10.221223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:10.221223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:30:10.221223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:30:10.221223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:10.221223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:30:10.222079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:10.222079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:30:10.222079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:10.222079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:30:10.222079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:10.222079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:30:10.222079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:30:10.222079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:30:10.222079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:30:10.222079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:30:10.222079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:30:10.222079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:30:10.222945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:10.222945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:10.222945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:10.222945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:30:10.222945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:10.222945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:10.222945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:10.222945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:30:10.222945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:10.222945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:30:10.222945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:30:10.222945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:10.222945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:30:10.222945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:10.223684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:30:10.223684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:30:10.223684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:10.223684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:30:10.223684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:10.223684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:30:10.223684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:10.223684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:30:10.223684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:30:10.223684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:30:10.223684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:30:10.223684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:30:10.223684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:10.223684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:30:10.224160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:30:10.224160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:30:10.224160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:30:10.224160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:30:10.224160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:10.224160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:10.224160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:30:10.224160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:30:10.224160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:30:10.224160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:30:10.224160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:30:10.224160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:30:10.224160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:30:10.224160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:10.224160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:10.224160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:10.224160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:30:10.224160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:10.224160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:10.225135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:10.225135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:30:10.225135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:10.225135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:30:10.225135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:30:10.225135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:10.225135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:30:10.225135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:10.225135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:30:10.225135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:30:10.225135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:10.225135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:30:10.225135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:10.225135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:30:10.241369 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:30:10.241752 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:30:10.241919 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:30:10.242114 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:30:10.242711 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:30:10.243079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:10.243826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:30:10.243826 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:10.243826 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:30:10.243826 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:30:10.243826 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:10.243826 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:30:10.243826 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:10.243826 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:30:10.244982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:10.244982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:30:10.244982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:30:10.244982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:30:10.244982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:10.244982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:30:10.244982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:10.244982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:30:10.244982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:10.244982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:30:10.244982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:10.244982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:30:10.244982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:10.244982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:30:10.244982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:30:10.245700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:10.245700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:30:10.245700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:10.245700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:30:10.245700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:30:10.245700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:10.245700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:30:10.245700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:10.245700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:30:10.246323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:10.246323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:30:10.246323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:10.246323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:30:10.246323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:30:10.246323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:30:10.246323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:30:10.246323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:30:10.246323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:30:10.246323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:10.246323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:10.247072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:10.247072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:30:10.247072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:10.247072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:10.247072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:10.247072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:30:10.247072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:10.247072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:30:10.247072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:30:10.247072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:10.247072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:30:10.247072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:10.247072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:30:10.247072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:30:10.247743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:10.247743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:30:10.247743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:10.247743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:30:10.248075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:10.248075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:30:10.248075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:30:10.248075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:30:10.248075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:30:10.248075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:30:10.248075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:10.248075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:30:10.248075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:30:10.248075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:30:10.248075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:30:10.248075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:30:10.248075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:10.248075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:10.248075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:30:10.248075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:30:10.248075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:30:10.249195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:30:10.249195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:30:10.249195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:30:10.249195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:10.249195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:10.249195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:10.249195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:30:10.249195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:10.249195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:10.249195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:10.249195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:30:10.249195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:10.249195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:30:10.249195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:30:10.249195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:10.249195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:30:10.249195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:10.249195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:30:10.249195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:30:10.250958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:10.250958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:30:10.250958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:10.250958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:30:10.250958 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:30:10.250958 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:30:10.250958 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:30:10.250958 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:30:10.251563 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:30:10.251820 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:30:10.252082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:10.252896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:30:10.252896 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:10.252896 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:30:10.252896 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:30:10.252896 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:10.252896 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:30:10.252896 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:10.252896 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:30:10.253799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:10.253799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:30:10.253799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:10.253799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:30:10.253799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:30:10.253799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:30:10.253799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:30:10.253799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:10.253799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:10.253799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:10.253799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:30:10.253799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:10.254165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:10.254165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:10.254165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:30:10.254165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:10.254165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:30:10.254165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:30:10.254165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:10.254165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:30:10.254165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:10.254165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:30:10.254165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:30:10.254165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:10.254165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:30:10.254537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:10.254537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:30:10.254760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:10.254760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:30:10.254760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:30:10.254760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:30:10.254760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:30:10.254760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:30:10.254760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:10.254760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:30:10.254760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:30:10.254760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:30:10.254760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:30:10.254760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:30:10.254760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:10.254760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:10.254760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:30:10.254760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:30:10.255273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:30:10.255273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:30:10.255273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:10.255273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:10.255273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:10.255273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:30:10.255273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:10.255273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:10.255273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:10.255273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:30:10.255273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:10.255273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:30:10.255273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:30:10.255273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:10.255273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:30:10.255273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:10.255273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:30:10.255273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:30:10.255273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:10.255847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:30:10.255847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:10.255847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:30:15.002471 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:30:15.003367 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:30:15.003367 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:30:15.003819 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:30:15.034902 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:30:15.035177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:15.035943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:30:15.035943 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:15.035943 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:30:15.035943 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:30:15.035943 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:15.035943 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:30:15.035943 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:15.035943 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:30:15.037140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:15.037140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:30:15.037140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:15.037140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:30:15.037140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:30:15.037140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:30:15.037140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:30:15.037140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:15.037140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:15.037140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:15.037140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:30:15.037140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:15.037968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:15.037968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:15.037968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:30:15.037968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:15.037968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:30:15.037968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:30:15.037968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:15.037968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:30:15.037968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:15.037968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:30:15.037968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:30:15.037968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:15.037968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:30:15.038845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:15.038845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:30:15.038845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:15.038845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:30:15.038845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:30:15.038845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:30:15.038845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:30:15.038845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:30:15.038845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:15.038845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:30:15.038845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:30:15.038845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:30:15.038845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:30:15.038845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:30:15.038845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:15.039754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:15.039754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:30:15.039754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:30:15.039754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:30:15.039754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:30:15.039754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:15.039754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:15.039754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:15.039754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:30:15.039754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:15.039754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:15.039754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:15.039754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:30:15.039754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:15.039754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:30:15.039754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:30:15.039754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:15.039754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:30:15.039754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:15.040833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:30:15.040833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:30:15.040833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:15.040833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:30:15.040833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:15.040833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:30:15.210142 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:30:15.210667 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:30:15.210765 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:30:15.211821 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:30:15.212163 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:30:15.212420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:15.213231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:30:15.213231 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:15.213231 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:30:15.213231 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:30:15.213231 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:15.213231 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:30:15.213231 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:15.213231 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:30:15.214248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:15.214248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:30:15.214248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:15.214248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:30:15.214248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:30:15.214248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:30:15.214248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:30:15.214248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:15.214248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:15.214248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:15.214248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:30:15.214248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:15.214644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:15.214644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:15.214644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:30:15.214644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:15.214644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:30:15.214644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:30:15.214644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:15.214644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:30:15.214644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:15.214644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:30:15.214644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:30:15.214644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:15.214644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:30:15.214983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:15.214983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:30:15.215265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:15.215265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:30:15.215265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:30:15.215265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:30:15.215265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:30:15.215265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:30:15.215265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:15.215265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:30:15.215265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:30:15.215265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:30:15.215265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:30:15.215265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:30:15.215265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:15.215265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:15.215265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:30:15.215265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:30:15.215693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:30:15.215693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:30:15.215693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:15.215693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:15.215693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:15.215693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:30:15.215693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:15.215693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:15.215693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:15.215693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:30:15.215693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:15.215693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:30:15.215693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:30:15.215693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:15.215693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:30:15.215693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:15.215693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:30:15.215693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:30:15.215693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:15.216172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:30:15.216172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:15.216172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:30:15.217304 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:30:15.217670 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:30:15.217821 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:30:15.217995 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:30:15.218492 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:30:15.218785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:15.219532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:30:15.219532 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:15.219532 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:30:15.219532 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:30:15.219532 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:15.219532 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:30:15.219532 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:15.219532 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:30:15.220397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:15.220397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:30:15.220397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:30:15.220397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:30:15.220397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:15.220397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:30:15.220397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:15.220397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:30:15.220397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:15.220397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:30:15.220397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:15.220397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:30:15.220397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:15.220397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:30:15.220397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:30:15.221066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:15.221066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:30:15.221066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:15.221066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:30:15.221066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:30:15.221066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:15.221066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:30:15.221066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:15.221066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:30:15.221636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:15.221636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:30:15.221636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:15.221636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:30:15.221636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:30:15.221636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:30:15.221636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:30:15.221636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:30:15.221636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:30:15.221636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:30:15.221636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:15.221636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:15.222127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:15.222127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:30:15.222127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:15.222127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:15.222127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:15.222127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:30:15.222127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:15.222127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:30:15.222127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:30:15.222127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:15.222127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:30:15.222127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:15.222127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:30:15.222127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:30:15.222686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:15.222686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:30:15.222686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:15.222686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:30:15.223072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:15.223072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:30:15.223072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:30:15.223072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:30:15.223072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:30:15.223072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:30:15.223072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:15.223072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:30:15.223072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:30:15.223072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:30:15.223072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:30:15.223072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:30:15.223072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:15.223072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:15.223072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:30:15.223072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:30:15.223534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:30:15.223534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:30:15.223534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:30:15.223534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:30:15.223534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:30:15.223534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:15.223534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:15.223534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:15.223534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:30:15.223534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:15.223534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:15.223534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:15.223534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:30:15.223534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:15.223534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:30:15.223534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:30:15.223534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:15.223534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:30:15.224013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:15.224013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:30:15.224013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:30:15.224013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:15.224013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:30:15.224013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:15.224013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:30:15.241921 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:30:15.242372 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:30:15.242871 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:30:15.242871 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:30:15.244545 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:30:15.244545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:15.244955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:30:15.244955 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:15.244955 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:30:15.244955 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:30:15.244955 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:15.244955 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:30:15.244955 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:15.244955 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:30:15.246496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:15.246496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:30:15.246496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:30:15.246496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:30:15.246496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:15.246496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:30:15.246496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:15.246496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:30:15.246496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:15.246496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:30:15.246496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:15.246496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:30:15.246496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:15.246496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:30:15.246496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:30:15.247210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:15.247210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:30:15.247210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:15.247210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:30:15.247210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:30:15.247210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:15.247210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:30:15.247210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:15.247210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:30:15.248033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:15.248033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:30:15.248033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:15.248033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:30:15.248033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:30:15.248033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:30:15.248033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:30:15.248033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:30:15.248033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:30:15.248033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:15.248033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:15.248639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:15.248639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:30:15.248639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:15.248639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:15.248639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:15.248639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:30:15.248639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:15.248639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:30:15.248639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:30:15.248639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:15.248639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:30:15.248639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:15.248639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:30:15.248639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:30:15.249428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:15.249428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:30:15.249428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:15.249428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:30:15.250084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:15.250084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:30:15.250084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:30:15.250084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:30:15.250084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:30:15.250084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:30:15.250084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:15.250084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:30:15.250084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:30:15.250084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:30:15.250084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:30:15.250084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:30:15.250084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:15.250084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:15.250084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:30:15.250084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:30:15.250084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:30:15.250939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:30:15.250939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:30:15.250939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:30:15.250939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:15.250939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:15.250939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:15.250939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:30:15.250939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:15.250939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:15.250939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:15.250939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:30:15.250939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:15.250939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:30:15.250939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:30:15.250939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:15.250939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:30:15.250939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:15.250939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:30:15.250939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:30:15.251827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:15.251827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:30:15.251827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:15.251827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:30:15.251827 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:30:15.251827 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:30:15.251827 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:30:15.251827 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:30:15.252268 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:30:15.252268 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:30:15.252268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:15.252802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:30:15.252802 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:15.252802 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:30:15.252802 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:30:15.252802 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:15.252802 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:30:15.252802 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:15.252802 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:30:15.254245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:15.254245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:30:15.254245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:15.254245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:30:15.254245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:30:15.254245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:30:15.254245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:30:15.254245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:15.254245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:15.254245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:15.254245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:30:15.254245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:15.254636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:15.254636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:15.254636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:30:15.254636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:15.254636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:30:15.254636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:30:15.254636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:15.254636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:30:15.254636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:15.254636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:30:15.254636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:30:15.254636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:15.254636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:30:15.254977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:15.254977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:30:15.255060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:15.255060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:30:15.255060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:30:15.255060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:30:15.255060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:30:15.255060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:30:15.255060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:15.255060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:30:15.255060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:30:15.255060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:30:15.255060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:30:15.255060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:30:15.255060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:15.255060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:15.255060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:30:15.255060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:30:15.255516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:30:15.255516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:30:15.255516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:15.255516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:15.255516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:15.255516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:30:15.255516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:15.255516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:15.255516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:15.255516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:30:15.255516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:15.255516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:30:15.255516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:30:15.255516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:15.255516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:30:15.255516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:15.255516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:30:15.255516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:30:15.255516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:15.256010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:30:15.256010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:15.256010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:30:20.003832 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:30:20.004549 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:30:20.004630 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:30:20.005340 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:30:20.035235 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:30:20.035558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:20.036154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:30:20.036154 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:20.036154 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:30:20.036154 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:30:20.036154 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:20.036154 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:30:20.036154 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:20.036154 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:30:20.036950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:20.036950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:30:20.036950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:20.036950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:30:20.036950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:30:20.036950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:30:20.036950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:30:20.036950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:20.036950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:20.036950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:20.036950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:30:20.036950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:20.037317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:20.037317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:20.037317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:30:20.037317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:20.037317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:30:20.037317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:30:20.037317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:20.037317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:30:20.037317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:20.037317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:30:20.037317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:30:20.037317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:20.037317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:30:20.037703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:20.037703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:30:20.037768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:20.037768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:30:20.037768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:30:20.037768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:30:20.037768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:30:20.037768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:30:20.037768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:20.037768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:30:20.037768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:30:20.037768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:30:20.037768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:30:20.037768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:30:20.037768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:20.037768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:20.037768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:30:20.037768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:30:20.038184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:30:20.038184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:30:20.038184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:20.038184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:20.038184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:20.038184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:30:20.038184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:20.038184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:20.038184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:20.038184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:30:20.038184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:20.038184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:30:20.038184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:30:20.038184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:20.038184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:30:20.038184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:20.038184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:30:20.038184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:30:20.038184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:20.038695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:30:20.038695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:20.038695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:30:20.210757 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:30:20.211334 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:30:20.211573 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:30:20.212594 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:30:20.212843 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:30:20.212993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:20.213667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:30:20.213667 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:20.213667 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:30:20.213667 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:30:20.213667 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:20.213667 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:30:20.213667 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:20.213667 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:30:20.215220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:20.215220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:30:20.215220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:20.215220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:30:20.215220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:30:20.215220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:30:20.215220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:30:20.215220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:20.215220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:20.215220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:20.215220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:30:20.215220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:20.215699 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:20.215699 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:20.215699 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:30:20.215699 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:20.215699 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:30:20.215699 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:30:20.215699 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:20.215699 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:30:20.215699 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:20.215699 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:30:20.215699 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:30:20.215699 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:20.215699 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:30:20.216196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:20.216196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:30:20.217228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:20.217228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:30:20.217228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:30:20.217228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:30:20.217228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:30:20.217228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:30:20.217228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:20.217228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:30:20.217228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:30:20.217228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:30:20.217228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:30:20.217228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:30:20.217228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:20.217228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:20.217228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:30:20.217228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:30:20.217850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:30:20.217850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:30:20.217850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:20.217850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:20.217850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:20.217850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:30:20.217850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:20.217850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:20.217850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:20.217850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:30:20.217850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:20.217850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:30:20.217850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:30:20.217850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:20.217850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:30:20.217850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:20.217850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:30:20.217850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:30:20.217850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:20.218497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:30:20.218497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:20.218497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:30:20.218497 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:30:20.218497 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:30:20.218497 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:30:20.218718 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:30:20.219226 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:30:20.219647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:20.220935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:30:20.220935 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:20.220935 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:30:20.220935 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:30:20.220935 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:20.220935 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:30:20.220935 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:20.220935 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:30:20.221540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:20.221540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:30:20.221540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:30:20.221540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:30:20.221540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:20.221540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:30:20.221540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:20.221540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:30:20.221540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:20.221540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:30:20.221540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:20.221540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:30:20.221540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:20.221540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:30:20.221540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:30:20.221996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:20.221996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:30:20.221996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:20.221996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:30:20.221996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:30:20.221996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:20.221996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:30:20.221996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:20.221996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:30:20.222504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:20.222504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:30:20.222504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:20.222504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:30:20.222504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:30:20.222504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:30:20.222504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:30:20.222504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:30:20.222504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:30:20.222504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:30:20.222504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:20.222504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:20.222874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:20.222874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:30:20.222874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:20.222874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:20.222874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:20.222874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:30:20.222874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:20.222874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:30:20.222874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:30:20.222874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:20.222874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:30:20.222874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:20.222874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:30:20.222874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:30:20.223218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:20.223218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:30:20.223218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:20.223218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:30:20.223427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:20.223427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:30:20.223427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:30:20.223427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:30:20.223427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:30:20.223427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:30:20.223427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:20.223427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:30:20.223427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:30:20.223427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:30:20.223427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:30:20.223427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:30:20.223427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:20.223427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:20.223427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:30:20.223427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:30:20.223832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:30:20.223832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:30:20.223832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:30:20.223832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:30:20.223832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:30:20.223832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:20.223832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:20.223832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:20.223832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:30:20.223832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:20.223832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:20.223832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:20.223832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:30:20.223832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:20.223832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:30:20.223832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:30:20.223832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:20.223832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:30:20.224521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:20.224521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:30:20.224521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:30:20.224521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:20.224521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:30:20.224521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:20.224521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:30:20.242518 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:30:20.242865 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:30:20.243035 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:30:20.243227 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:30:20.243961 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:30:20.244413 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:20.245221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:30:20.245221 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:20.245221 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:30:20.245221 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:30:20.245221 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:20.245221 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:30:20.245221 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:20.245221 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:30:20.246579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:20.246579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:30:20.246579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:30:20.246579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:30:20.246579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:20.246579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:30:20.246579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:20.246579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:30:20.246579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:20.246579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:30:20.246579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:20.246579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:30:20.246579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:20.246579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:30:20.246579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:30:20.247033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:20.247033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:30:20.247033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:20.247033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:30:20.247033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:30:20.247033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:20.247033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:30:20.247033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:20.247033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:30:20.247973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:20.247973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:30:20.247973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:20.247973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:30:20.247973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:30:20.247973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:30:20.247973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:30:20.247973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:30:20.247973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:30:20.247973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:20.247973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:20.248359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:20.248359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:30:20.248359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:20.248359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:20.248359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:20.248359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:30:20.248359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:20.248359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:30:20.248359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:30:20.248359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:20.248359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:30:20.248359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:20.248359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:30:20.248359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:30:20.248778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:20.248778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:30:20.248778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:20.248778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:30:20.249963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:20.249963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:30:20.249963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:30:20.249963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:30:20.249963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:30:20.249963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:30:20.249963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:20.249963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:30:20.249963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:30:20.249963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:30:20.249963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:30:20.249963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:30:20.249963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:20.249963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:20.249963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:30:20.249963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:30:20.249963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:30:20.250536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:30:20.250536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:30:20.250536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:30:20.250536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:20.250536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:20.250536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:20.250536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:30:20.250536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:20.250536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:20.250536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:20.250536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:30:20.250536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:20.250536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:30:20.250536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:30:20.250536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:20.250536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:30:20.250536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:20.250536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:30:20.250536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:30:20.251078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:20.251078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:30:20.251078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:20.251078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:30:20.251078 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:30:20.251602 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:30:20.251672 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:30:20.251914 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:30:20.252235 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:30:20.252418 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:30:20.252539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:20.252948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:30:20.252948 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:20.252948 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:30:20.252948 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:30:20.252948 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:20.252948 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:30:20.252948 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:20.252948 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:30:20.253747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:20.253747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:30:20.253747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:20.253747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:30:20.253747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:30:20.253747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:30:20.253747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:30:20.253747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:20.253747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:20.253747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:20.253747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:30:20.253747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:20.254150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:20.254150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:20.254150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:30:20.254150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:20.254150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:30:20.254150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:30:20.254150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:20.254150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:30:20.254150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:20.254150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:30:20.254150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:30:20.254150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:20.254150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:30:20.254666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:20.254666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:30:20.254666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:20.254666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:30:20.254666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:30:20.254666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:30:20.254666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:30:20.254666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:30:20.254666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:20.254666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:30:20.254666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:30:20.254666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:30:20.254666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:30:20.254666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:30:20.254666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:20.255071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:20.255071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:30:20.255071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:30:20.255071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:30:20.255071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:30:20.255071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:20.255071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:20.255071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:20.255071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:30:20.255071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:20.255071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:20.255071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:20.255071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:30:20.255071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:20.255071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:30:20.255071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:30:20.255071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:20.255071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:30:20.255071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:20.255581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:30:20.255581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:30:20.255581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:20.255581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:30:20.255581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:20.255581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:30:25.004397 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:30:25.005302 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:30:25.005302 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:30:25.006781 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:30:25.008673 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG dbcounter [-] [105581] Writing DB stats neutron:SELECT=434 {{(pid=105581) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 14:30:25.046906 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:30:25.047311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:25.048149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:30:25.048149 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:25.048149 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:30:25.048149 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:30:25.048149 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:25.048149 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:30:25.048149 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:25.048149 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:30:25.048906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:25.048906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:30:25.048906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:25.048906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:30:25.048906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:30:25.048906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:30:25.048906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:30:25.048906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:25.048906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:25.048906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:25.048906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:30:25.048906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:25.049558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:25.049558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:25.049558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:30:25.049558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:25.049558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:30:25.049558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:30:25.049558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:25.049558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:30:25.049558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:25.049558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:30:25.049558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:30:25.049558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:25.049558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:30:25.050262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:25.050262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:30:25.050262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:25.050262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:30:25.050262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:30:25.050262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:30:25.050262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:30:25.050262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:30:25.050262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:25.050262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:30:25.050262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:30:25.050262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:30:25.050262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:30:25.050262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:30:25.050262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:25.051118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:25.051118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:30:25.051118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:30:25.051118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:30:25.051118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:30:25.051118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:25.051118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:25.051118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:25.051118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:30:25.051118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:25.051118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:25.051118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:25.051118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:30:25.051118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:25.051118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:30:25.051118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:30:25.051118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:25.051118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:30:25.051118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:25.051943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:30:25.051943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:30:25.051943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:25.051943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:30:25.051943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:25.051943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:30:25.211801 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:30:25.212470 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:30:25.212673 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:30:25.213719 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:30:25.214058 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:30:25.214293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:25.215229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:30:25.215229 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:25.215229 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:30:25.215229 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:30:25.215229 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:25.215229 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:30:25.215229 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:25.215229 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:30:25.216857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:25.216857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:30:25.216857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:25.216857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:30:25.216857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:30:25.216857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:30:25.216857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:30:25.216857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:25.216857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:25.216857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:25.216857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:30:25.216857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:25.217144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:25.217144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:25.217144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:30:25.217144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:25.217144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:30:25.217144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:30:25.217144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:25.217144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:30:25.217144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:25.217144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:30:25.217144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:30:25.217144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:25.217144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:30:25.217383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:25.217383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:30:25.218787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:25.218787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:30:25.218787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:30:25.218787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:30:25.218787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:30:25.218787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:30:25.218787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:25.218787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:30:25.218787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:30:25.218787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:30:25.218787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:30:25.218787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:30:25.218787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:25.218787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:25.218787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:30:25.218787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:30:25.219086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:30:25.219086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:30:25.219086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:25.219086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:25.219086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:25.219086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:30:25.219086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:25.219086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:25.219086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:25.219086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:30:25.219086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:25.219086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:30:25.219086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:30:25.219086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:25.219086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:30:25.219086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:25.219086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:30:25.219086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:30:25.219086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:25.219431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:30:25.219431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:25.219431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:30:25.219431 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:30:25.219431 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:30:25.219722 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:30:25.219916 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:30:25.220417 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:30:25.220745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:25.221431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:30:25.221431 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:25.221431 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:30:25.221431 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:30:25.221431 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:25.221431 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:30:25.221431 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:25.221431 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:30:25.222153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:25.222153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:30:25.222153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:30:25.222153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:30:25.222153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:25.222153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:30:25.222153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:25.222153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:30:25.222153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:25.222153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:30:25.222153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:25.222153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:30:25.222153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:25.222153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:30:25.222153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:30:25.222475 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:25.222475 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:30:25.222475 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:25.222475 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:30:25.222475 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:30:25.222475 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:25.222475 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:30:25.222475 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:25.222475 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:30:25.223033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:25.223033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:30:25.223033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:25.223033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:30:25.223033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:30:25.223033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:30:25.223033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:30:25.223033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:30:25.223033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:30:25.223033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:30:25.223033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:25.223033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:25.223265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:25.223265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:30:25.223265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:25.223265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:25.223265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:25.223265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:30:25.223265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:25.223265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:30:25.223265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:30:25.223265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:25.223265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:30:25.223265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:25.223265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:30:25.223265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:30:25.223635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:25.223635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:30:25.223635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:25.223635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:30:25.224118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:25.224118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:30:25.224118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:30:25.224118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:30:25.224118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:30:25.224118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:30:25.224118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:25.224118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:30:25.224118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:30:25.224118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:30:25.224118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:30:25.224118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:30:25.224118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:25.224118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:25.224118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:30:25.224118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:30:25.224438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:30:25.224438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:30:25.224438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:30:25.224438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:30:25.224438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:30:25.224438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:25.224438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:25.224438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:25.224438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:30:25.224438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:25.224438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:25.224438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:25.224438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:30:25.224438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:25.224438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:30:25.224438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:30:25.224438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:25.224438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:30:25.224772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:25.224772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:30:25.224772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:30:25.224772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:25.224772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:30:25.224772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:25.224772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:30:25.242984 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:30:25.243288 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:30:25.243420 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:30:25.243598 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:30:25.244053 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:30:25.244395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:25.245160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:30:25.245160 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:25.245160 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:30:25.245160 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:30:25.245160 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:25.245160 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:30:25.245160 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:25.245160 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:30:25.246317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:25.246317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:30:25.246317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:30:25.246317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:30:25.246317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:25.246317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:30:25.246317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:25.246317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:30:25.246317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:25.246317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:30:25.246317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:25.246317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:30:25.246317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:25.246317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:30:25.246317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:30:25.246612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:25.246612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:30:25.246612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:25.246612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:30:25.246612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:30:25.246612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:25.246612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:30:25.246612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:25.246612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:30:25.247724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:25.247724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:30:25.247724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:25.247724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:30:25.247724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:30:25.247724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:30:25.247724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:30:25.247724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:30:25.247724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:30:25.247724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:25.247724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:25.247936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:25.247936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:30:25.247936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:25.247936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:25.247936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:25.247936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:30:25.247936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:25.247936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:30:25.247936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:30:25.247936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:25.247936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:30:25.247936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:25.247936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:30:25.247936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:30:25.248203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:25.248203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:30:25.248203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:25.248203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:30:25.249366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:25.249366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:30:25.249366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:30:25.249366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:30:25.249366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:30:25.249366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:30:25.249366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:25.249366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:30:25.249366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:30:25.249366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:30:25.249366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:30:25.249366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:30:25.249366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:25.249366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:25.249366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:30:25.249366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:30:25.249366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:30:25.249690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:30:25.249690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:30:25.249690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:30:25.249690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:25.249690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:25.249690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:25.249690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:30:25.249690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:25.249690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:25.249690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:25.249690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:30:25.249690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:25.249690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:30:25.249690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:30:25.249690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:25.249690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:30:25.249690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:25.249690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:30:25.249690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:30:25.250011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:25.250011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:30:25.250011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:25.250011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:30:25.251619 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:30:25.251896 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:30:25.252008 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:30:25.252151 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:30:25.252593 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:30:25.252937 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:30:25.253161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:25.253869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:30:25.253869 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:25.253869 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:30:25.253869 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:30:25.253869 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:25.253869 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:30:25.253869 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:25.253869 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:30:25.254888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:25.254888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:30:25.254888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:25.254888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:30:25.254888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:30:25.254888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:30:25.254888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:30:25.254888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:25.254888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:25.254888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:25.254888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:30:25.254888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:25.255145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:25.255145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:25.255145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:30:25.255145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:25.255145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:30:25.255145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:30:25.255145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:25.255145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:30:25.255145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:25.255145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:30:25.255145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:30:25.255145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:25.255145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:30:25.255470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:25.255470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:30:25.255859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:25.255859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:30:25.255859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:30:25.255859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:30:25.255859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:30:25.255859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:30:25.255859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:25.255859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:30:25.255859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:30:25.255859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:30:25.255859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:30:25.255859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:30:25.255859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:25.255859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:25.255859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:30:25.255859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:30:25.256156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:30:25.256156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:30:25.256156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:25.256156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:25.256156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:25.256156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:30:25.256156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:25.256156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:25.256156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:25.256156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:30:25.256156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:25.256156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:30:25.256156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:30:25.256156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:25.256156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:30:25.256156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:25.256156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:30:25.256156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:30:25.256156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:25.256595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:30:25.256595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:25.256595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:30:30.005033 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:30:30.005928 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:30:30.005928 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:30:30.007122 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:30:30.038431 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:30:30.038883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:30.039784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:30:30.039784 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:30.039784 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:30:30.039784 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:30:30.039784 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:30.039784 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:30:30.039784 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:30.039784 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:30:30.040789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:30.040789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:30:30.040789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:30.040789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:30:30.040789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:30:30.040789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:30:30.040789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:30:30.040789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:30.040789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:30.040789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:30.040789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:30:30.040789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:30.041580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:30.041580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:30.041580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:30:30.041580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:30.041580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:30:30.041580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:30:30.041580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:30.041580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:30:30.041580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:30.041580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:30:30.041580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:30:30.041580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:30.041580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:30:30.042255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:30.042255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:30:30.042255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:30.042255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:30:30.042255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:30:30.042255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:30:30.042255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:30:30.042255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:30:30.042255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:30.042255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:30:30.042255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:30:30.042255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:30:30.042255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:30:30.042255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:30:30.042255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:30.043102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:30.043102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:30:30.043102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:30:30.043102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:30:30.043102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:30:30.043102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:30.043102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:30.043102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:30.043102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:30:30.043102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:30.043102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:30.043102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:30.043102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:30:30.043102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:30.043102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:30:30.043102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:30:30.043102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:30.043102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:30:30.043102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:30.044118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:30:30.044118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:30:30.044118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:30.044118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:30:30.044118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:30.044118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:30:30.212489 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:30:30.213022 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:30:30.213149 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:30:30.214252 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:30:30.214650 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:30:30.214909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:30.215796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:30:30.215796 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:30.215796 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:30:30.215796 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:30:30.215796 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:30.215796 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:30:30.215796 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:30.215796 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:30:30.217161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:30.217161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:30:30.217161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:30.217161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:30:30.217161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:30:30.217161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:30:30.217161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:30:30.217161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:30.217161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:30.217161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:30.217161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:30:30.217161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:30.217824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:30.217824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:30.217824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:30:30.217824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:30.217824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:30:30.217824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:30:30.217824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:30.217824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:30:30.217824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:30.217824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:30:30.217824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:30:30.217824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:30.217824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:30:30.218435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:30.218435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:30:30.218719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:30.218719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:30:30.218719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:30:30.218719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:30:30.218719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:30:30.218719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:30:30.218719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:30.218719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:30:30.218719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:30:30.218719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:30:30.218719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:30:30.218719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:30:30.218719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:30.218719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:30.218719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:30:30.218719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:30:30.219493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:30:30.219493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:30:30.219493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:30.219493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:30.219493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:30.219493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:30:30.219493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:30.219493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:30.219493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:30.219493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:30:30.219493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:30.219493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:30:30.219493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:30:30.219493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:30.219493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:30:30.219493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:30.219493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:30:30.219493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:30:30.219493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:30.220379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:30:30.220379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:30.220379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:30:30.220379 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:30:30.220379 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:30:30.220379 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:30:30.220379 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:30:30.220766 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:30:30.220979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:30.221781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:30:30.221781 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:30.221781 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:30:30.221781 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:30:30.221781 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:30.221781 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:30:30.221781 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:30.221781 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:30:30.222832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:30.222832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:30:30.222832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:30:30.222832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:30:30.222832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:30.222832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:30:30.222832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:30.222832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:30:30.222832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:30.222832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:30:30.222832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:30.222832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:30:30.222832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:30.222832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:30:30.222832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:30:30.223829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:30.223829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:30:30.223829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:30.223829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:30:30.223829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:30:30.223829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:30.223829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:30:30.223829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:30.223829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:30:30.223829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:30.223829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:30:30.223829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:30.223829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:30:30.223829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:30:30.224505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:30:30.224505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:30:30.224505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:30:30.224505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:30:30.224505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:30:30.224505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:30.224505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:30.224505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:30.224505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:30:30.224505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:30.224505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:30.224505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:30.224505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:30:30.225093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:30.225093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:30:30.225093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:30:30.225093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:30.225093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:30:30.225093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:30.225093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:30:30.225093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:30:30.225093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:30.225093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:30:30.225093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:30.225093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:30:30.225710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:30.225710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:30:30.225710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:30:30.225710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:30:30.225710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:30:30.225710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:30:30.225710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:30.225710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:30:30.225710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:30:30.225710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:30:30.225710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:30:30.225710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:30:30.225710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:30.225710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:30.225710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:30:30.225710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:30:30.226100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:30:30.226100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:30:30.226100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:30:30.226100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:30:30.226100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:30:30.226100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:30.226100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:30.226100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:30.226100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:30:30.226100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:30.226100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:30.226100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:30.226100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:30:30.226100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:30.226100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:30:30.226100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:30:30.226100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:30.226100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:30:30.226447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:30.226447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:30:30.226447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:30:30.226447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:30.226447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:30:30.226447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:30.226447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:30:30.243516 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:30:30.243899 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:30:30.244085 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:30:30.244276 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:30:30.244769 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:30:30.245123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:30.245937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:30:30.245937 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:30.245937 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:30:30.245937 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:30:30.245937 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:30.245937 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:30:30.245937 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:30.245937 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:30:30.247100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:30.247100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:30:30.247100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:30:30.247100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:30:30.247100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:30.247100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:30:30.247100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:30.247100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:30:30.247100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:30.247100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:30:30.247100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:30.247100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:30:30.247100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:30.247100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:30:30.247100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:30:30.247827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:30.247827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:30:30.247827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:30.247827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:30:30.247827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:30:30.247827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:30.247827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:30:30.247827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:30.247827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:30:30.248504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:30.248504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:30:30.248504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:30.248504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:30:30.248504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:30:30.248504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:30:30.248504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:30:30.248504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:30:30.248504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:30:30.248504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:30.248504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:30.249142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:30.249142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:30:30.249142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:30.249142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:30.249142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:30.249142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:30:30.249142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:30.249142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:30:30.249142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:30:30.249142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:30.249142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:30:30.249142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:30.249142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:30:30.249142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:30:30.249945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:30.249945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:30:30.249945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:30.249945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:30:30.250145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:30.250145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:30:30.250145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:30:30.250145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:30:30.250145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:30:30.250145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:30:30.250145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:30.250145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:30:30.250145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:30:30.250145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:30:30.250145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:30:30.250145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:30:30.250145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:30.250145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:30.250145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:30:30.250145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:30:30.250145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:30:30.251003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:30:30.251003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:30:30.251003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:30:30.251003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:30.251003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:30.251003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:30.251003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:30:30.251003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:30.251003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:30.251003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:30.251003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:30:30.251003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:30.251003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:30:30.251003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:30:30.251003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:30.251003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:30:30.251003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:30.251003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:30:30.251003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:30:30.251872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:30.251872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:30:30.251872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:30.251872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:30:30.252254 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:30:30.252325 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:30:30.252540 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:30:30.252794 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:30:30.253187 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:30:30.253596 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:30:30.253902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:30.254490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:30:30.254490 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:30.254490 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:30:30.254490 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:30:30.254490 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:30.254490 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:30:30.254490 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:30.254490 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:30:30.255362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:30.255362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:30:30.255362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:30.255362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:30:30.255362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:30:30.255362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:30:30.255362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:30:30.255362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:30.255362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:30.255362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:30.255362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:30:30.255362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:30.255762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:30.255762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:30.255762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:30:30.255762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:30.255762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:30:30.255762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:30:30.255762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:30.255762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:30:30.255762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:30.255762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:30:30.255762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:30:30.255762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:30.255762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:30:30.256119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:30.256119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:30:30.256246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:30.256246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:30:30.256246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:30:30.256246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:30:30.256246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:30:30.256246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:30:30.256246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:30.256246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:30:30.256246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:30:30.256246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:30:30.256246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:30:30.256246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:30:30.256246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:30.256246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:30.256246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:30:30.256246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:30:30.256716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:30:30.256716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:30:30.256716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:30.256716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:30.256716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:30.256716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:30:30.256716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:30.256716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:30.256716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:30.256716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:30:30.256716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:30.256716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:30:30.256716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:30:30.256716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:30.256716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:30:30.256716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:30.256716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:30:30.256716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:30:30.256716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:30.257250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:30:30.257250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:30.257250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:30:35.005829 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:30:35.006638 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:30:35.006638 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:30:35.007328 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:30:35.043645 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:30:35.044110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:35.044947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:30:35.044947 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:35.044947 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:30:35.044947 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:30:35.044947 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:35.044947 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:30:35.044947 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:35.044947 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:30:35.045713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:35.045713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:30:35.045713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:35.045713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:30:35.045713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:30:35.045713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:30:35.045713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:30:35.045713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:35.045713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:35.045713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:35.045713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:30:35.045713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:35.046481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:35.046481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:35.046481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:30:35.046481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:35.046481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:30:35.046481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:30:35.046481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:35.046481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:30:35.046481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:35.046481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:30:35.046481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:30:35.046481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:35.046481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:30:35.047026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:35.047026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:30:35.047026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:35.047026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:30:35.047026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:30:35.047026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:30:35.047026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:30:35.047026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:30:35.047026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:35.047026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:30:35.047026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:30:35.047026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:30:35.047026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:30:35.047026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:30:35.047026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:35.047817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:35.047817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:30:35.047817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:30:35.047817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:30:35.047817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:30:35.047817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:35.047817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:35.047817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:35.047817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:30:35.047817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:35.047817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:35.047817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:35.047817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:30:35.047817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:35.047817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:30:35.047817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:30:35.047817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:35.047817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:30:35.047817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:35.048660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:30:35.048660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:30:35.048660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:35.048660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:30:35.048660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:35.048660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:30:35.213288 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:30:35.213858 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:30:35.214050 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:30:35.215342 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:30:35.215876 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:30:35.216309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:35.217440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:30:35.217440 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:35.217440 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:30:35.217440 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:30:35.217440 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:35.217440 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:30:35.217440 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:35.217440 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:30:35.219084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:35.219084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:30:35.219084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:35.219084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:30:35.219084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:30:35.219084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:30:35.219084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:30:35.219084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:35.219084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:35.219084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:35.219084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:30:35.219084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:35.219689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:35.219689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:35.219689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:30:35.219689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:35.219689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:30:35.219689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:30:35.219689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:35.219689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:30:35.219689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:35.219689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:30:35.219689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:30:35.219689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:35.219689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:30:35.220445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:35.220445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:30:35.220856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:35.220856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:30:35.220856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:30:35.220856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:30:35.220856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:30:35.220856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:30:35.220856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:35.220856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:30:35.220856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:30:35.220856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:30:35.220856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:30:35.220856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:30:35.220856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:35.220856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:35.220856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:30:35.220856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:30:35.221583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:30:35.221583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:30:35.221583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:35.221583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:35.221583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:35.221583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:30:35.221583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:35.221583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:35.221583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:35.221583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:30:35.221583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:35.221583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:30:35.221583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:30:35.221583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:35.221583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:30:35.221583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:35.221583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:30:35.221583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:30:35.221583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:35.222345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:30:35.222345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:35.222345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:30:35.222345 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:30:35.222345 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:30:35.222345 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:30:35.222345 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:30:35.222765 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:30:35.222970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:35.223512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:30:35.223512 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:35.223512 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:30:35.223512 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:30:35.223512 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:35.223512 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:30:35.223512 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:35.223512 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:30:35.224727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:35.224727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:30:35.224727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:30:35.224727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:30:35.224727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:35.224727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:30:35.224727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:35.224727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:30:35.224727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:35.224727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:30:35.224727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:35.224727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:30:35.224727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:35.224727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:30:35.224727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:30:35.225119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:35.225119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:30:35.225119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:35.225119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:30:35.225119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:30:35.225119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:35.225119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:30:35.225119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:35.225119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:30:35.226059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:35.226059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:30:35.226059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:35.226059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:30:35.226059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:30:35.226059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:30:35.226059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:30:35.226059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:30:35.226059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:30:35.226059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:30:35.226059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:35.226059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:35.226398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:35.226398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:30:35.226398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:35.226398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:35.226398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:35.226398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:30:35.226398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:35.226398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:30:35.226398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:30:35.226398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:35.226398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:30:35.226398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:35.226398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:30:35.226398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:30:35.226823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:35.226823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:30:35.226823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:35.226823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:30:35.227686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:35.227686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:30:35.227686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:30:35.227686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:30:35.227686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:30:35.227686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:30:35.227686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:35.227686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:30:35.227686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:30:35.227686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:30:35.227686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:30:35.227686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:30:35.227686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:35.227686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:35.227686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:30:35.227686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:30:35.228172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:30:35.228172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:30:35.228172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:30:35.228172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:30:35.228172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:30:35.228172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:35.228172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:35.228172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:35.228172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:30:35.228172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:35.228172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:35.228172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:35.228172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:30:35.228172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:35.228172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:30:35.228172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:30:35.228172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:35.228172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:30:35.228674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:35.228674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:30:35.228674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:30:35.228674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:35.228674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:30:35.228674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:35.228674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:30:35.243892 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:30:35.244385 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:30:35.244587 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:30:35.244880 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:30:35.245294 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:30:35.245555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:35.246151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:30:35.246151 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:35.246151 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:30:35.246151 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:30:35.246151 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:35.246151 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:30:35.246151 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:35.246151 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:30:35.247600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:35.247600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:30:35.247600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:30:35.247600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:30:35.247600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:35.247600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:30:35.247600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:35.247600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:30:35.247600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:35.247600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:30:35.247600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:35.247600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:30:35.247600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:35.247600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:30:35.247600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:30:35.248395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:35.248395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:30:35.248395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:35.248395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:30:35.248395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:30:35.248395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:35.248395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:30:35.248395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:35.248395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:30:35.249312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:35.249312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:30:35.249312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:35.249312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:30:35.249312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:30:35.249312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:30:35.249312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:30:35.249312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:30:35.249312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:30:35.249312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:35.249312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:35.249914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:35.249914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:30:35.249914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:35.249914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:35.249914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:35.249914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:30:35.249914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:35.249914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:30:35.249914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:30:35.249914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:35.249914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:30:35.249914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:35.249914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:30:35.249914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:30:35.250700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:35.250700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:30:35.250700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:35.250700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:30:35.251141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:35.251141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:30:35.251141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:30:35.251141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:30:35.251141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:30:35.251141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:30:35.251141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:35.251141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:30:35.251141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:30:35.251141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:30:35.251141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:30:35.251141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:30:35.251141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:35.251141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:35.251141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:30:35.251141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:30:35.251141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:30:35.252222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:30:35.252222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:30:35.252222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:30:35.252222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:35.252222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:35.252222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:35.252222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:30:35.252222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:35.252222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:35.252222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:35.252222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:30:35.252222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:35.252222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:30:35.252222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:30:35.252222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:35.252222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:30:35.252222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:35.252222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:30:35.252222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:30:35.253074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:35.253074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:30:35.253074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:35.253074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:30:35.253074 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:30:35.253074 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:30:35.253074 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:30:35.253505 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:30:35.253707 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:30:35.254065 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:30:35.254291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:35.255050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:30:35.255050 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:35.255050 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:30:35.255050 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:30:35.255050 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:35.255050 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:30:35.255050 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:35.255050 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:30:35.255937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:35.255937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:30:35.255937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:35.255937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:30:35.255937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:30:35.255937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:30:35.255937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:30:35.255937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:35.255937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:35.255937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:35.255937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:30:35.255937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:35.256619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:35.256619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:35.256619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:30:35.256619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:35.256619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:30:35.256619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:30:35.256619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:35.256619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:30:35.256619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:35.256619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:30:35.256619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:30:35.256619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:35.256619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:30:35.257158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:35.257158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:30:35.257158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:35.257158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:30:35.257158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:30:35.257158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:30:35.257158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:30:35.257158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:30:35.257158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:35.257158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:30:35.257158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:30:35.257158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:30:35.257158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:30:35.257158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:30:35.257158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:35.257598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:35.257598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:30:35.257598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:30:35.257598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:30:35.257598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:30:35.257598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:35.257598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:35.257598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:35.257598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:30:35.257598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:35.257598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:35.257598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:35.257598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:30:35.257598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:35.257598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:30:35.257598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:30:35.257598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:35.257598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:30:35.257598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:35.258089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:30:35.258089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:30:35.258089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:35.258089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:30:35.258089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:35.258089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:30:40.006590 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:30:40.007256 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:30:40.007479 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:30:40.008710 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:30:40.043487 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:30:40.043753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:40.044545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:30:40.044545 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:40.044545 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:30:40.044545 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:30:40.044545 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:40.044545 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:30:40.044545 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:40.044545 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:30:40.045166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:40.045166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:30:40.045166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:40.045166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:30:40.045166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:30:40.045166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:30:40.045166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:30:40.045166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:40.045166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:40.045166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:40.045166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:30:40.045166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:40.045634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:40.045634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:40.045634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:30:40.045634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:40.045634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:30:40.045634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:30:40.045634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:40.045634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:30:40.045634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:40.045634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:30:40.045634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:30:40.045634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:40.045634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:30:40.046039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:40.046039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:30:40.046039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:40.046039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:30:40.046039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:30:40.046039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:30:40.046039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:30:40.046039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:30:40.046039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:40.046039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:30:40.046039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:30:40.046039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:30:40.046039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:30:40.046039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:30:40.046039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:40.046599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:40.046599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:30:40.046599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:30:40.046599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:30:40.046599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:30:40.046599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:40.046599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:40.046599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:40.046599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:30:40.046599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:40.046599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:40.046599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:40.046599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:30:40.046599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:40.046599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:30:40.046599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:30:40.046599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:40.046599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:30:40.046599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:40.047199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:30:40.047199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:30:40.047199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:40.047199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:30:40.047199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:40.047199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:30:40.213786 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:30:40.214369 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:30:40.214561 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:30:40.215769 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:30:40.216094 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:30:40.216348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:40.217377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:30:40.217377 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:40.217377 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:30:40.217377 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:30:40.217377 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:40.217377 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:30:40.217377 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:40.217377 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:30:40.218912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:40.218912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:30:40.218912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:40.218912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:30:40.218912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:30:40.218912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:30:40.218912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:30:40.218912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:40.218912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:40.218912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:40.218912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:30:40.218912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:40.219502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:40.219502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:40.219502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:30:40.219502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:40.219502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:30:40.219502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:30:40.219502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:40.219502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:30:40.219502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:40.219502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:30:40.219502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:30:40.219502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:40.219502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:30:40.219866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:40.219866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:30:40.220712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:40.220712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:30:40.220712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:30:40.220712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:30:40.220712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:30:40.220712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:30:40.220712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:40.220712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:30:40.220712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:30:40.220712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:30:40.220712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:30:40.220712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:30:40.220712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:40.220712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:40.220712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:30:40.220712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:30:40.222944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:30:40.222944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:30:40.222944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:40.222944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:40.222944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:40.222944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:30:40.222944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:40.222944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:40.222944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:40.222944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:30:40.222944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:40.222944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:30:40.222944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:30:40.222944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:40.222944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:30:40.222944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:40.222944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:30:40.222944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:30:40.222944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:40.223935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:30:40.223935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:40.223935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:30:40.223935 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:30:40.223935 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:30:40.223935 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:30:40.223935 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:30:40.223935 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:30:40.224450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:40.224450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:30:40.224450 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:40.224450 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:30:40.224450 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:30:40.224450 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:40.224450 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:30:40.224450 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:40.224450 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:30:40.225105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:40.225105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:30:40.225105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:30:40.225105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:30:40.225105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:40.225105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:30:40.225105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:40.225105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:30:40.225105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:40.225105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:30:40.225105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:40.225105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:30:40.225105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:40.225105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:30:40.225105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:30:40.226028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:40.226028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:30:40.226028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:40.226028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:30:40.226028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:30:40.226028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:40.226028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:30:40.226028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:40.226028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:30:40.226028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:40.226028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:30:40.226028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:40.226028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:30:40.226028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:30:40.226874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:30:40.226874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:30:40.226874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:30:40.226874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:30:40.226874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:30:40.226874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:40.226874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:40.226874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:40.226874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:30:40.226874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:40.226874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:40.226874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:40.226874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:30:40.227500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:40.227500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:30:40.227500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:30:40.227500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:40.227500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:30:40.227500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:40.227500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:30:40.227500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:30:40.227500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:40.227500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:30:40.227500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:40.227500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:30:40.227859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:40.227859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:30:40.227859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:30:40.227859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:30:40.227859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:30:40.227859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:30:40.227859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:40.227859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:30:40.227859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:30:40.227859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:30:40.227859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:30:40.227859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:30:40.227859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:40.227859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:40.227859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:30:40.227859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:30:40.228340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:30:40.228340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:30:40.228340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:30:40.228340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:30:40.228340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:30:40.228340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:40.228340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:40.228340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:40.228340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:30:40.228340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:40.228340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:40.228340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:40.228340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:30:40.228340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:40.228340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:30:40.228340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:30:40.228340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:40.228340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:30:40.228903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:40.228903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:30:40.228903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:30:40.228903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:40.228903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:30:40.228903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:40.228903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:30:40.244346 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:30:40.244653 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:30:40.244740 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:30:40.244873 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:30:40.245353 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:30:40.245742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:40.246633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:30:40.246633 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:40.246633 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:30:40.246633 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:30:40.246633 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:40.246633 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:30:40.246633 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:40.246633 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:30:40.247783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:40.247783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:30:40.247783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:30:40.247783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:30:40.247783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:40.247783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:30:40.247783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:40.247783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:30:40.247783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:40.247783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:30:40.247783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:40.247783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:30:40.247783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:40.247783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:30:40.247783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:30:40.248613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:40.248613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:30:40.248613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:40.248613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:30:40.248613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:30:40.248613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:40.248613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:30:40.248613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:40.248613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:30:40.249100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:40.249100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:30:40.249100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:40.249100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:30:40.249100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:30:40.249100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:30:40.249100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:30:40.249100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:30:40.249100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:30:40.249100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:40.249100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:40.249670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:40.249670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:30:40.249670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:40.249670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:40.249670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:40.249670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:30:40.249670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:40.249670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:30:40.249670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:30:40.249670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:40.249670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:30:40.249670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:40.249670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:30:40.249670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:30:40.250251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:40.250251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:30:40.250251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:40.250251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:30:40.250738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:40.250738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:30:40.250738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:30:40.250738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:30:40.250738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:30:40.250738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:30:40.250738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:40.250738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:30:40.250738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:30:40.250738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:30:40.250738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:30:40.250738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:30:40.250738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:40.250738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:40.250738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:30:40.250738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:30:40.250738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:30:40.251598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:30:40.251598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:30:40.251598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:30:40.251598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:40.251598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:40.251598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:40.251598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:30:40.251598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:40.251598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:40.251598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:40.251598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:30:40.251598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:40.251598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:30:40.251598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:30:40.251598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:40.251598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:30:40.251598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:40.251598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:30:40.251598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:30:40.252360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:40.252360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:30:40.252360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:40.252360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:30:40.252790 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:30:40.253121 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:30:40.253268 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:30:40.253433 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:30:40.253860 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:30:40.254200 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:30:40.254434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:40.255135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:30:40.255135 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:40.255135 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:30:40.255135 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:30:40.255135 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:40.255135 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:30:40.255135 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:40.255135 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:30:40.255937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:40.255937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:30:40.255937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:40.255937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:30:40.255937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:30:40.255937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:30:40.255937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:30:40.255937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:40.255937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:40.255937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:40.255937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:30:40.255937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:40.256554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:40.256554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:40.256554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:30:40.256554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:40.256554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:30:40.256554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:30:40.256554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:40.256554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:30:40.256554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:40.256554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:30:40.256554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:30:40.256554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:40.256554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:30:40.257174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:40.257174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:30:40.257174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:40.257174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:30:40.257174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:30:40.257174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:30:40.257174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:30:40.257174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:30:40.257174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:40.257174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:30:40.257174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:30:40.257174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:30:40.257174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:30:40.257174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:30:40.257174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:40.257895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:40.257895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:30:40.257895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:30:40.257895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:30:40.257895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:30:40.257895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:40.257895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:40.257895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:40.257895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:30:40.257895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:40.257895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:40.257895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:40.257895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:30:40.257895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:40.257895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:30:40.257895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:30:40.257895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:40.257895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:30:40.257895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:40.258772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:30:40.258772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:30:40.258772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:40.258772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:30:40.258772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:40.258772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:30:45.007336 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:30:45.008223 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:30:45.008307 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:30:45.009644 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:30:45.046598 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:30:45.047011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:45.047853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:30:45.047853 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:45.047853 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:30:45.047853 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:30:45.047853 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:45.047853 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:30:45.047853 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:45.047853 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:30:45.048632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:45.048632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:30:45.048632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:45.048632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:30:45.048632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:30:45.048632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:30:45.048632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:30:45.048632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:45.048632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:45.048632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:45.048632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:30:45.048632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:45.049498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:45.049498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:45.049498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:30:45.049498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:45.049498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:30:45.049498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:30:45.049498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:45.049498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:30:45.049498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:45.049498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:30:45.049498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:30:45.049498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:45.049498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:30:45.050302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:45.050302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:30:45.050302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:45.050302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:30:45.050302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:30:45.050302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:30:45.050302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:30:45.050302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:30:45.050302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:45.050302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:30:45.050302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:30:45.050302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:30:45.050302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:30:45.050302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:30:45.050302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:45.051085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:45.051085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:30:45.051085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:30:45.051085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:30:45.051085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:30:45.051085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:45.051085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:45.051085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:45.051085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:30:45.051085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:45.051085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:45.051085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:45.051085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:30:45.051085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:45.051085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:30:45.051085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:30:45.051085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:45.051085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:30:45.051085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:45.052027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:30:45.052027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:30:45.052027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:45.052027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:30:45.052027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:45.052027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:30:45.214527 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:30:45.215002 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:30:45.215191 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:30:45.216103 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:30:45.216472 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:30:45.216725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:45.217530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:30:45.217530 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:45.217530 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:30:45.217530 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:30:45.217530 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:45.217530 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:30:45.217530 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:45.217530 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:30:45.218939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:45.218939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:30:45.218939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:45.218939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:30:45.218939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:30:45.218939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:30:45.218939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:30:45.218939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:45.218939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:45.218939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:45.218939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:30:45.218939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:45.219647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:45.219647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:45.219647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:30:45.219647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:45.219647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:30:45.219647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:30:45.219647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:45.219647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:30:45.219647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:45.219647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:30:45.219647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:30:45.219647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:45.219647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:30:45.220378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:45.220378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:30:45.220566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:45.220566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:30:45.220566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:30:45.220566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:30:45.220566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:30:45.220566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:30:45.220566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:45.220566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:30:45.220566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:30:45.220566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:30:45.220566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:30:45.220566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:30:45.220566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:45.220566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:45.220566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:30:45.220566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:30:45.221594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:30:45.221594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:30:45.221594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:45.221594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:45.221594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:45.221594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:30:45.221594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:45.221594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:45.221594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:45.221594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:30:45.221594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:45.221594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:30:45.221594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:30:45.221594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:45.221594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:30:45.221594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:45.221594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:30:45.221594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:30:45.221594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:45.222606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:30:45.222606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:45.222606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:30:45.222606 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:30:45.222865 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:30:45.223050 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:30:45.223221 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:30:45.223736 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:30:45.224012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:45.224687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:30:45.224687 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:45.224687 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:30:45.224687 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:30:45.224687 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:45.224687 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:30:45.224687 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:45.224687 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:30:45.225434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:45.225434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:30:45.225434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:30:45.225434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:30:45.225434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:45.225434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:30:45.225434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:45.225434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:30:45.225434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:45.225434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:30:45.225434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:45.225434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:30:45.225434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:45.225434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:30:45.225434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:30:45.226464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:45.226464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:30:45.226464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:45.226464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:30:45.226464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:30:45.226464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:45.226464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:30:45.226464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:45.226464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:30:45.226464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:45.226464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:30:45.226464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:45.226464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:30:45.226464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:30:45.227470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:30:45.227470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:30:45.227470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:30:45.227470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:30:45.227470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:30:45.227470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:45.227470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:45.227470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:45.227470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:30:45.227470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:45.227470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:45.227470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:45.227470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:30:45.228194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:45.228194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:30:45.228194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:30:45.228194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:45.228194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:30:45.228194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:45.228194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:30:45.228194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:30:45.228194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:45.228194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:30:45.228194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:45.228194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:30:45.228872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:45.228872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:30:45.228872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:30:45.228872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:30:45.228872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:30:45.228872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:30:45.228872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:45.228872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:30:45.228872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:30:45.228872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:30:45.228872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:30:45.228872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:30:45.228872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:45.228872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:45.228872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:30:45.228872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:30:45.229713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:30:45.229713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:30:45.229713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:30:45.229713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:30:45.229713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:30:45.229713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:45.229713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:45.229713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:45.229713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:30:45.229713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:45.229713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:45.229713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:45.229713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:30:45.229713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:45.229713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:30:45.229713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:30:45.229713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:45.229713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:30:45.230510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:45.230510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:30:45.230510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:30:45.230510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:45.230510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:30:45.230510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:45.230510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:30:45.244822 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:30:45.245064 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:30:45.245150 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:30:45.245322 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:30:45.245769 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:30:45.246109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:45.246820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:30:45.246820 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:45.246820 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:30:45.246820 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:30:45.246820 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:45.246820 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:30:45.246820 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:45.246820 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:30:45.247536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:45.247536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:30:45.247536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:30:45.247536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:30:45.247536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:45.247536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:30:45.247536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:45.247536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:30:45.247536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:45.247536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:30:45.247536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:45.247536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:30:45.247536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:45.247536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:30:45.247536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:30:45.248186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:45.248186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:30:45.248186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:45.248186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:30:45.248186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:30:45.248186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:45.248186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:30:45.248186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:45.248186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:30:45.248636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:45.248636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:30:45.248636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:45.248636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:30:45.248636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:30:45.248636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:30:45.248636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:30:45.248636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:30:45.248636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:30:45.248636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:45.248636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:45.249136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:45.249136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:30:45.249136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:45.249136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:45.249136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:45.249136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:30:45.249136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:45.249136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:30:45.249136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:30:45.249136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:45.249136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:30:45.249136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:45.249136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:30:45.249136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:30:45.249849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:45.249849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:30:45.249849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:45.249849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:30:45.249849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:45.249849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:30:45.249849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:30:45.249849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:30:45.249849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:30:45.249849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:30:45.249849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:45.249849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:30:45.249849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:30:45.249849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:30:45.250502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:30:45.250502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:30:45.250502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:45.250502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:45.250502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:30:45.250502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:30:45.250502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:30:45.250502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:30:45.250502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:30:45.250502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:30:45.250502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:45.250502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:45.250502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:45.250502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:30:45.250502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:45.250502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:45.250502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:45.250502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:30:45.250502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:45.251065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:30:45.251065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:30:45.251065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:45.251065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:30:45.251065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:45.251065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:30:45.251065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:30:45.251065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:45.251065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:30:45.251065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:45.251065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:30:45.253253 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:30:45.253581 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:30:45.253684 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:30:45.253827 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:30:45.254220 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:30:45.254581 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:30:45.254792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:45.255476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:30:45.255476 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:45.255476 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:30:45.255476 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:30:45.255476 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:45.255476 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:30:45.255476 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:45.255476 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:30:45.256254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:45.256254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:30:45.256254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:45.256254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:30:45.256254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:30:45.256254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:30:45.256254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:30:45.256254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:45.256254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:45.256254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:45.256254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:30:45.256254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:45.256622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:45.256622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:45.256622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:30:45.256622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:45.256622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:30:45.256622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:30:45.256622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:45.256622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:30:45.256622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:45.256622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:30:45.256622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:30:45.256622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:45.256622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:30:45.256973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:45.256973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:30:45.257181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:45.257181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:30:45.257181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:30:45.257181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:30:45.257181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:30:45.257181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:30:45.257181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:45.257181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:30:45.257181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:30:45.257181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:30:45.257181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:30:45.257181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:30:45.257181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:45.257181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:45.257181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:30:45.257181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:30:45.257761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:30:45.257761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:30:45.257761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:45.257761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:45.257761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:45.257761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:30:45.257761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:45.257761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:45.257761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:45.257761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:30:45.257761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:45.257761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:30:45.257761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:30:45.257761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:45.257761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:30:45.257761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:45.257761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:30:45.257761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:30:45.257761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:45.258318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:30:45.258318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:45.258318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:30:50.008160 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:30:50.009090 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:30:50.009090 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:30:50.010321 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:30:50.046548 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:30:50.047113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:50.048271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:30:50.048271 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:50.048271 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:30:50.048271 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:30:50.048271 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:50.048271 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:30:50.048271 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:50.048271 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:30:50.049275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:50.049275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:30:50.049275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:50.049275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:30:50.049275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:30:50.049275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:30:50.049275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:30:50.049275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:50.049275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:50.049275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:50.049275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:30:50.049275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:50.050234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:50.050234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:50.050234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:30:50.050234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:50.050234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:30:50.050234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:30:50.050234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:50.050234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:30:50.050234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:50.050234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:30:50.050234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:30:50.050234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:50.050234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:30:50.050942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:50.050942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:30:50.050942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:50.050942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:30:50.050942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:30:50.050942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:30:50.050942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:30:50.050942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:30:50.050942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:50.050942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:30:50.050942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:30:50.050942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:30:50.050942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:30:50.050942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:30:50.050942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:50.051811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:50.051811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:30:50.051811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:30:50.051811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:30:50.051811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:30:50.051811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:50.051811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:50.051811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:50.051811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:30:50.051811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:50.051811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:50.051811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:50.051811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:30:50.051811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:50.051811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:30:50.051811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:30:50.051811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:50.051811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:30:50.051811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:50.053313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:30:50.053313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:30:50.053313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:50.053313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:30:50.053313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:50.053313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:30:50.215205 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:30:50.215966 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:30:50.216144 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:30:50.217388 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:30:50.217787 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:30:50.218058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:50.218991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:30:50.218991 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:50.218991 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:30:50.218991 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:30:50.218991 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:50.218991 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:30:50.218991 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:50.218991 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:30:50.220578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:50.220578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:30:50.220578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:50.220578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:30:50.220578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:30:50.220578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:30:50.220578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:30:50.220578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:50.220578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:50.220578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:50.220578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:30:50.220578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:50.221171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:50.221171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:50.221171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:30:50.221171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:50.221171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:30:50.221171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:30:50.221171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:50.221171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:30:50.221171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:50.221171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:30:50.221171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:30:50.221171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:50.221171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:30:50.221727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:50.221727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:30:50.222193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:50.222193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:30:50.222193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:30:50.222193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:30:50.222193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:30:50.222193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:30:50.222193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:50.222193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:30:50.222193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:30:50.222193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:30:50.222193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:30:50.222193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:30:50.222193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:50.222193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:50.222193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:30:50.222193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:30:50.222939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:30:50.222939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:30:50.222939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:50.222939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:50.222939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:50.222939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:30:50.222939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:50.222939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:50.222939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:50.222939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:30:50.222939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:50.222939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:30:50.222939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:30:50.222939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:50.222939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:30:50.222939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:50.222939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:30:50.222939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:30:50.222939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:50.223725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:30:50.223725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:50.223725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:30:50.223725 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:30:50.223725 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:30:50.223725 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:30:50.223725 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:30:50.224184 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:30:50.224509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:50.225202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:30:50.225202 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:50.225202 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:30:50.225202 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:30:50.225202 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:50.225202 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:30:50.225202 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:50.225202 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:30:50.225997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:50.225997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:30:50.225997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:30:50.225997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:30:50.225997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:50.225997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:30:50.225997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:50.225997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:30:50.225997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:50.225997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:30:50.225997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:50.225997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:30:50.225997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:50.225997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:30:50.225997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:30:50.226470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:50.226470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:30:50.226470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:50.226470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:30:50.226470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:30:50.226470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:50.226470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:30:50.226470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:50.226470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:30:50.226854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:50.226854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:30:50.226854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:50.226854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:30:50.226854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:30:50.226854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:30:50.226854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:30:50.226854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:30:50.226854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:30:50.226854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:30:50.226854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:50.226854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:50.227192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:50.227192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:30:50.227192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:50.227192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:50.227192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:50.227192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:30:50.227192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:50.227192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:30:50.227192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:30:50.227192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:50.227192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:30:50.227192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:50.227192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:30:50.227192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:30:50.227559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:50.227559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:30:50.227559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:50.227559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:30:50.228073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:50.228073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:30:50.228073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:30:50.228073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:30:50.228073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:30:50.228073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:30:50.228073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:50.228073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:30:50.228073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:30:50.228073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:30:50.228073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:30:50.228073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:30:50.228073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:50.228073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:50.228073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:30:50.228073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:30:50.228507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:30:50.228507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:30:50.228507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:30:50.228507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:30:50.228507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:30:50.228507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:50.228507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:50.228507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:50.228507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:30:50.228507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:50.228507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:50.228507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:50.228507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:30:50.228507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:50.228507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:30:50.228507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:30:50.228507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:50.228507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:30:50.228968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:50.228968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:30:50.228968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:30:50.228968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:50.228968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:30:50.228968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:50.228968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:30:50.245212 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:30:50.245637 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:30:50.245817 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:30:50.246013 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:30:50.246530 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:30:50.246886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:50.247670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:30:50.247670 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:50.247670 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:30:50.247670 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:30:50.247670 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:50.247670 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:30:50.247670 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:50.247670 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:30:50.249010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:50.249010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:30:50.249010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:30:50.249010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:30:50.249010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:50.249010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:30:50.249010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:50.249010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:30:50.249010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:50.249010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:30:50.249010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:50.249010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:30:50.249010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:50.249010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:30:50.249010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:30:50.249640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:50.249640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:30:50.249640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:50.249640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:30:50.249640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:30:50.249640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:50.249640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:30:50.249640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:50.249640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:30:50.250460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:50.250460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:30:50.250460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:50.250460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:30:50.250460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:30:50.250460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:30:50.250460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:30:50.250460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:30:50.250460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:30:50.250460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:50.250460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:50.250989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:50.250989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:30:50.250989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:50.250989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:50.250989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:50.250989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:30:50.250989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:50.250989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:30:50.250989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:30:50.250989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:50.250989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:30:50.250989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:50.250989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:30:50.250989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:30:50.251571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:50.251571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:30:50.251571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:50.251571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:30:50.252215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:50.252215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:30:50.252215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:30:50.252215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:30:50.252215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:30:50.252215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:30:50.252215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:50.252215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:30:50.252215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:30:50.252215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:30:50.252215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:30:50.252215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:30:50.252215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:50.252215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:50.252215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:30:50.252215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:30:50.252215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:30:50.252961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:30:50.252961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:30:50.252961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:30:50.252961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:50.252961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:50.252961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:50.252961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:30:50.252961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:50.252961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:50.252961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:50.252961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:30:50.252961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:50.252961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:30:50.252961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:30:50.252961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:50.252961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:30:50.252961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:50.252961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:30:50.252961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:30:50.253732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:50.253732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:30:50.253732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:50.253732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:30:50.253732 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:30:50.254083 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:30:50.254250 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:30:50.254513 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:30:50.254982 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:30:50.255319 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:30:50.255562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:50.256357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:30:50.256357 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:50.256357 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:30:50.256357 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:30:50.256357 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:50.256357 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:30:50.256357 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:50.256357 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:30:50.257248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:50.257248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:30:50.257248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:50.257248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:30:50.257248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:30:50.257248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:30:50.257248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:30:50.257248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:50.257248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:50.257248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:50.257248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:30:50.257248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:50.257598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:50.257598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:50.257598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:30:50.257598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:50.257598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:30:50.257598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:30:50.257598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:50.257598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:30:50.257598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:50.257598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:30:50.257598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:30:50.257598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:50.257598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:30:50.257925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:50.257925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:30:50.258254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:50.258254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:30:50.258254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:30:50.258254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:30:50.258254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:30:50.258254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:30:50.258254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:50.258254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:30:50.258254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:30:50.258254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:30:50.258254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:30:50.258254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:30:50.258254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:50.258254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:50.258254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:30:50.258254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:30:50.258708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:30:50.258708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:30:50.258708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:50.258708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:50.258708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:50.258708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:30:50.258708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:50.258708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:50.258708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:50.258708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:30:50.258708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:50.258708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:30:50.258708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:30:50.258708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:50.258708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:30:50.258708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:50.258708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:30:50.258708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:30:50.258708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:50.259179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:30:50.259179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:50.259179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:30:55.008854 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:30:55.009534 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:30:55.009588 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:30:55.010610 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:30:55.051230 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:30:55.051230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:55.051450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:30:55.051450 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:55.051450 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:30:55.051450 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:30:55.051450 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:55.051450 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:30:55.051450 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:55.051450 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:30:55.052151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:55.052151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:30:55.052151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:55.052151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:30:55.052151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:30:55.052151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:30:55.052151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:30:55.052151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:55.052151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:55.052151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:55.052151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:30:55.052151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:55.052609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:55.052609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:55.052609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:30:55.052609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:55.052609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:30:55.052609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:30:55.052609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:55.052609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:30:55.052609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:55.052609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:30:55.052609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:30:55.052609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:55.052609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:30:55.053038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:55.053038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:30:55.053223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:55.053223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:30:55.053223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:30:55.053223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:30:55.053223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:30:55.053223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:30:55.053223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:55.053223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:30:55.053223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:30:55.053223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:30:55.053223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:30:55.053223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:30:55.053223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:55.053223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:55.053223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:30:55.053223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:30:55.053639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:30:55.053639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:30:55.053639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:55.053639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:55.053639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:55.053639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:30:55.053639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:55.053639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:55.053639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:55.053639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:30:55.053639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:55.053639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:30:55.053639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:30:55.053639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:55.053639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:30:55.053639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:55.053639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:30:55.053639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:30:55.053639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:55.054027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:30:55.054027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:55.054027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:30:55.215861 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:30:55.216505 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:30:55.216668 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:30:55.218974 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:30:55.218974 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:30:55.218974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:55.218974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:30:55.218974 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:55.218974 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:30:55.218974 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:30:55.218974 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:55.218974 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:30:55.218974 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:55.218974 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:30:55.220338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:55.220338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:30:55.220338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:55.220338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:30:55.220338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:30:55.220338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:30:55.220338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:30:55.220338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:55.220338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:55.220338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:55.220338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:30:55.220338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:55.221177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:55.221177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:55.221177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:30:55.221177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:55.221177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:30:55.221177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:30:55.221177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:55.221177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:30:55.221177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:55.221177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:30:55.221177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:30:55.221177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:55.221177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:30:55.222187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:55.222187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:30:55.222187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:55.222187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:30:55.222187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:30:55.222187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:30:55.222187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:30:55.222187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:30:55.222187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:55.222187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:30:55.222187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:30:55.222187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:30:55.222187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:30:55.222187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:30:55.222187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:55.223237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:55.223237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:30:55.223237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:30:55.223237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:30:55.223237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:30:55.223237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:55.223237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:55.223237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:55.223237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:30:55.223237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:55.223237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:55.223237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:55.223237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:30:55.223237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:55.223237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:30:55.223237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:30:55.223237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:55.223237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:30:55.223237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:55.224576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:30:55.224576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:30:55.224576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:55.224576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:30:55.224576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:55.224576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:30:55.224576 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:30:55.224576 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:30:55.224576 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:30:55.224576 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:30:55.225993 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:30:55.226291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:55.226902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:30:55.226902 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:55.226902 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:30:55.226902 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:30:55.226902 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:55.226902 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:30:55.226902 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:55.226902 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:30:55.227646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:55.227646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:30:55.227646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:30:55.227646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:30:55.227646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:55.227646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:30:55.227646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:55.227646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:30:55.227646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:55.227646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:30:55.227646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:55.227646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:30:55.227646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:55.227646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:30:55.227646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:30:55.228104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:55.228104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:30:55.228104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:55.228104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:30:55.228104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:30:55.228104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:55.228104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:30:55.228104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:55.228104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:30:55.228521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:55.228521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:30:55.228521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:55.228521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:30:55.228521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:30:55.228521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:30:55.228521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:30:55.228521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:30:55.228521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:30:55.228521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:30:55.228521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:55.228521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:55.228871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:55.228871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:30:55.228871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:55.228871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:55.228871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:55.228871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:30:55.228871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:55.228871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:30:55.228871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:30:55.228871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:55.228871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:30:55.228871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:55.228871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:30:55.228871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:30:55.229228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:55.229228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:30:55.229228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:55.229228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:30:55.229685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:55.229685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:30:55.229685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:30:55.229685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:30:55.229685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:30:55.229685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:30:55.229685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:55.229685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:30:55.229685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:30:55.229685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:30:55.229685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:30:55.229685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:30:55.229685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:55.229685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:55.229685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:30:55.229685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:30:55.230296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:30:55.230296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:30:55.230296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:30:55.230296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:30:55.230296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:30:55.230296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:55.230296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:55.230296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:55.230296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:30:55.230296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:55.230296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:55.230296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:55.230296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:30:55.230296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:55.230296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:30:55.230296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:30:55.230296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:55.230296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:30:55.230796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:55.230796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:30:55.230796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:30:55.230796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:55.230796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:30:55.230796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:55.230796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:30:55.245748 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:30:55.246032 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:30:55.246146 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:30:55.246247 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:30:55.247514 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:30:55.247514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:55.247899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:30:55.247899 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:55.247899 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:30:55.247899 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:30:55.247899 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:55.247899 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:30:55.247899 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:55.247899 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:30:55.248575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:55.248575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:30:55.248575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:30:55.248575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:30:55.248575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:55.248575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:30:55.248575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:55.248575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:30:55.248575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:55.248575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:30:55.248575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:55.248575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:30:55.248575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:55.248575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:30:55.248575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:30:55.249027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:55.249027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:30:55.249027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:55.249027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:30:55.249027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:30:55.249027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:55.249027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:30:55.249027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:55.249027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:30:55.249434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:55.249434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:30:55.249434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:55.249434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:30:55.249434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:30:55.249434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:30:55.249434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:30:55.249434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:30:55.249434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:30:55.249434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:55.249434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:55.249754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:55.249754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:30:55.249754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:55.249754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:55.249754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:55.249754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:30:55.249754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:55.249754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:30:55.249754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:30:55.249754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:55.249754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:30:55.249754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:55.249754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:30:55.249754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:30:55.250119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:55.250119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:30:55.250119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:55.250119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:30:55.250593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:55.250593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:30:55.250593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:30:55.250593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:30:55.250593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:30:55.250593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:30:55.250593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:55.250593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:30:55.250593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:30:55.250593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:30:55.250593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:30:55.250593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:30:55.250593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:55.250593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:55.250593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:30:55.250593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:30:55.250593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:30:55.251628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:30:55.251628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:30:55.251628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:30:55.251628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:55.251628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:55.251628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:55.251628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:30:55.251628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:55.251628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:55.251628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:55.251628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:30:55.251628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:55.251628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:30:55.251628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:30:55.251628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:55.251628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:30:55.251628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:55.251628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:30:55.251628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:30:55.252520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:55.252520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:30:55.252520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:55.252520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:30:55.254144 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:30:55.254398 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:30:55.254528 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:30:55.254706 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:30:55.255162 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:30:55.255634 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:30:55.255912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:55.256640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:30:55.256640 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:55.256640 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:30:55.256640 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:30:55.256640 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:55.256640 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:30:55.256640 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:55.256640 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:30:55.257426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:55.257426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:30:55.257426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:55.257426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:30:55.257426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:30:55.257426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:30:55.257426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:30:55.257426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:55.257426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:55.257426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:55.257426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:30:55.257426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:55.258076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:30:55.258076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:55.258076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:30:55.258076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:55.258076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:30:55.258076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:30:55.258076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:55.258076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:30:55.258076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:55.258076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:30:55.258076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:30:55.258076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:55.258076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:30:55.258730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:55.258730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:30:55.258730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:55.258730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:30:55.258730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:30:55.258730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:30:55.258730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:30:55.258730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:30:55.258730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:55.258730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:30:55.258730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:30:55.258730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:30:55.258730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:30:55.258730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:30:55.258730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:30:55.259555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:30:55.259555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:30:55.259555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:30:55.259555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:30:55.259555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:30:55.259555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:55.259555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:55.259555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:30:55.259555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:30:55.259555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:30:55.259555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:30:55.259555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:30:55.259555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:30:55.259555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:30:55.259555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:30:55.259555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:30:55.259555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:30:55.259555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:30:55.259555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:30:55.260465 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:30:55.260465 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:30:55.260465 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:30:55.260465 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:30:55.260465 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:30:55.260465 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:31:00.009669 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:31:00.010575 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:31:00.010575 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:31:00.011740 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:31:00.046513 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:31:00.047004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:00.047758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:31:00.047758 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:00.047758 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:31:00.047758 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:31:00.047758 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:00.047758 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:31:00.047758 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:00.047758 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:31:00.048725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:00.048725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:31:00.048725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:00.048725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:31:00.048725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:31:00.048725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:31:00.048725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:31:00.048725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:00.048725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:00.048725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:00.048725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:31:00.048725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:00.049657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:00.049657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:00.049657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:31:00.049657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:00.049657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:31:00.049657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:31:00.049657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:00.049657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:31:00.049657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:00.049657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:31:00.049657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:31:00.049657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:00.049657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:31:00.050338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:00.050338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:31:00.050338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:00.050338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:31:00.050338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:31:00.050338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:31:00.050338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:31:00.050338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:31:00.050338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:00.050338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:31:00.050338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:31:00.050338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:31:00.050338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:31:00.050338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:31:00.050338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:00.051097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:00.051097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:31:00.051097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:31:00.051097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:31:00.051097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:31:00.051097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:00.051097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:00.051097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:00.051097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:31:00.051097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:00.051097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:00.051097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:00.051097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:31:00.051097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:00.051097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:31:00.051097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:31:00.051097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:00.051097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:31:00.051097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:00.052029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:31:00.052029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:31:00.052029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:00.052029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:31:00.052029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:00.052029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:31:00.217290 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:31:00.217290 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:31:00.217678 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:31:00.218500 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:31:00.218871 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:31:00.220445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:00.220445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:31:00.220445 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:00.220445 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:31:00.220445 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:31:00.220445 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:00.220445 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:31:00.220445 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:00.220445 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:31:00.221738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:00.221738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:31:00.221738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:00.221738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:31:00.221738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:31:00.221738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:31:00.221738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:31:00.221738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:00.221738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:00.221738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:00.221738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:31:00.221738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:00.222346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:00.222346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:00.222346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:31:00.222346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:00.222346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:31:00.222346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:31:00.222346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:00.222346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:31:00.222346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:00.222346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:31:00.222346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:31:00.222346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:00.222346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:31:00.222859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:00.222859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:31:00.223503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:00.223503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:31:00.223503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:31:00.223503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:31:00.223503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:31:00.223503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:31:00.223503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:00.223503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:31:00.223503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:31:00.223503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:31:00.223503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:31:00.223503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:31:00.223503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:00.223503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:00.223503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:31:00.223503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:31:00.224061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:31:00.224061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:31:00.224061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:00.224061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:00.224061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:00.224061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:31:00.224061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:00.224061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:00.224061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:00.224061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:31:00.224061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:00.224061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:31:00.224061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:31:00.224061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:00.224061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:31:00.224061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:00.224061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:31:00.224061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:31:00.224061 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:00.224585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:31:00.224585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:00.224585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:31:00.224585 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:31:00.224585 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:31:00.224585 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:31:00.224585 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:31:00.224805 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:31:00.225511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:00.226308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:31:00.226308 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:00.226308 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:31:00.226308 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:31:00.226308 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:00.226308 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:31:00.226308 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:00.226308 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:31:00.226578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:00.226578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:31:00.226578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:31:00.226578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:31:00.226578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:00.226578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:31:00.226578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:00.226578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:31:00.226578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:00.226578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:31:00.226578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:00.226578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:31:00.226578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:00.226578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:31:00.226578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:31:00.227050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:00.227050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:31:00.227050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:00.227050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:31:00.227050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:31:00.227050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:00.227050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:31:00.227050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:00.227050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:31:00.227332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:00.227332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:31:00.227332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:00.227332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:31:00.227332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:31:00.227332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:31:00.227332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:31:00.227332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:31:00.227332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:31:00.227332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:31:00.227332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:00.227332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:00.227740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:00.227740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:31:00.227740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:00.227740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:00.227740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:00.227740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:31:00.227740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:00.227740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:31:00.227740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:31:00.227740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:00.227740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:31:00.227740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:00.227740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:31:00.227740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:31:00.228781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:00.228781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:31:00.228781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:00.228781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:31:00.228781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:00.228781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:31:00.228781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:31:00.228781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:31:00.228781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:31:00.228781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:31:00.228781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:00.228781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:31:00.228781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:31:00.228781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:31:00.231721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:31:00.231721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:31:00.231721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:00.231721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:00.231721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:31:00.231721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:31:00.231721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:31:00.231721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:31:00.231721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:31:00.231721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:31:00.231721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:31:00.231721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:00.231721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:00.231721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:00.231721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:31:00.231721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:00.231721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:00.231721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:00.231721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:31:00.232092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:00.232092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:31:00.232092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:31:00.232092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:00.232092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:31:00.232092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:00.232092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:31:00.232092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:31:00.232092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:00.232092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:31:00.232092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:00.232092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:31:00.246143 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:31:00.246554 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:31:00.246729 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:31:00.246920 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:31:00.247499 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:31:00.247931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:00.248753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:31:00.248753 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:00.248753 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:31:00.248753 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:31:00.248753 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:00.248753 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:31:00.248753 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:00.248753 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:31:00.250074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:00.250074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:31:00.250074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:31:00.250074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:31:00.250074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:00.250074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:31:00.250074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:00.250074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:31:00.250074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:00.250074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:31:00.250074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:00.250074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:31:00.250074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:00.250074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:31:00.250074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:31:00.250928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:00.250928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:31:00.250928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:00.250928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:31:00.250928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:31:00.250928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:00.250928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:31:00.250928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:00.250928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:31:00.251542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:00.251542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:31:00.251542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:00.251542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:31:00.251542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:31:00.251542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:31:00.251542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:31:00.251542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:31:00.251542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:31:00.251542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:00.251542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:00.252057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:00.252057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:31:00.252057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:00.252057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:00.252057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:00.252057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:31:00.252057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:00.252057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:31:00.252057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:31:00.252057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:00.252057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:31:00.252057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:00.252057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:31:00.252057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:31:00.252859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:00.252859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:31:00.252859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:00.252859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:31:00.253434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:00.253434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:31:00.253434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:31:00.253434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:31:00.253434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:31:00.253434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:31:00.253434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:00.253434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:31:00.253434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:31:00.253434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:31:00.253434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:31:00.253434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:31:00.253434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:00.253434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:00.253434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:31:00.253434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:31:00.253434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:31:00.254214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:31:00.254214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:31:00.254214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:31:00.254214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:00.254214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:00.254214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:00.254214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:31:00.254214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:00.254214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:00.254214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:00.254214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:31:00.254214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:00.254214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:31:00.254214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:31:00.254214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:00.254214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:31:00.254214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:00.254214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:31:00.254214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:31:00.255065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:00.255065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:31:00.255065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:00.255065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:31:00.255065 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:31:00.255065 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:31:00.255065 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:31:00.255065 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:31:00.255502 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:31:00.255580 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:31:00.255851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:00.256201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:31:00.256201 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:00.256201 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:31:00.256201 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:31:00.256201 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:00.256201 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:31:00.256201 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:00.256201 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:31:00.256953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:00.256953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:31:00.256953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:00.256953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:31:00.256953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:31:00.256953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:31:00.256953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:31:00.256953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:00.256953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:00.256953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:00.256953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:31:00.256953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:00.257361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:00.257361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:00.257361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:31:00.257361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:00.257361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:31:00.257361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:31:00.257361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:00.257361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:31:00.257361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:00.257361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:31:00.257361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:31:00.257361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:00.257361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:31:00.257732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:00.257732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:31:00.258002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:00.258002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:31:00.258002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:31:00.258002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:31:00.258002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:31:00.258002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:31:00.258002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:00.258002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:31:00.258002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:31:00.258002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:31:00.258002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:31:00.258002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:31:00.258002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:00.258002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:00.258002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:31:00.258002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:31:00.258464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:31:00.258464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:31:00.258464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:00.258464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:00.258464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:00.258464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:31:00.258464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:00.258464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:00.258464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:00.258464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:31:00.258464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:00.258464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:31:00.258464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:31:00.258464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:00.258464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:31:00.258464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:00.258464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:31:00.258464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:31:00.258464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:00.259116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:31:00.259116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:00.259116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:31:05.010270 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:31:05.010993 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:31:05.011064 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:31:05.012624 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:31:05.042578 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:31:05.044176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:05.045050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:31:05.045050 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:05.045050 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:31:05.045050 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:31:05.045050 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:05.045050 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:31:05.045050 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:05.045050 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:31:05.046064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:05.046064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:31:05.046064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:05.046064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:31:05.046064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:31:05.046064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:31:05.046064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:31:05.046064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:05.046064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:05.046064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:05.046064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:31:05.046064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:05.046738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:05.046738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:05.046738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:31:05.046738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:05.046738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:31:05.046738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:31:05.046738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:05.046738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:31:05.046738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:05.046738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:31:05.046738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:31:05.046738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:05.046738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:31:05.047379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:05.047379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:31:05.047379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:05.047379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:31:05.047379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:31:05.047379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:31:05.047379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:31:05.047379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:31:05.047379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:05.047379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:31:05.047379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:31:05.047379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:31:05.047379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:31:05.047379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:31:05.047379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:05.048182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:05.048182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:31:05.048182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:31:05.048182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:31:05.048182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:31:05.048182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:05.048182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:05.048182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:05.048182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:31:05.048182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:05.048182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:05.048182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:05.048182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:31:05.048182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:05.048182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:31:05.048182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:31:05.048182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:05.048182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:31:05.048182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:05.048801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:31:05.048801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:31:05.048801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:05.048801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:31:05.048801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:05.048801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:31:05.217378 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:31:05.217958 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:31:05.218200 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:31:05.219357 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:31:05.219748 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:31:05.219998 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:05.222141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:31:05.222141 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:05.222141 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:31:05.222141 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:31:05.222141 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:05.222141 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:31:05.222141 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:05.222141 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:31:05.223942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:05.223942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:31:05.223942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:05.223942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:31:05.223942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:31:05.223942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:31:05.223942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:31:05.223942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:05.223942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:05.223942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:05.223942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:31:05.223942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:05.224785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:05.224785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:05.224785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:31:05.224785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:05.224785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:31:05.224785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:31:05.224785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:05.224785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:31:05.224785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:05.224785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:31:05.224785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:31:05.224785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:05.224785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:31:05.225332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:05.225332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:31:05.225332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:05.225332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:31:05.225332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:31:05.225332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:31:05.225332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:31:05.225332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:31:05.225332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:05.225332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:31:05.225332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:31:05.225332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:31:05.225332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:31:05.225332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:31:05.225332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:05.226005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:05.226005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:31:05.226005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:31:05.226005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:31:05.226005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:31:05.226005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:05.226005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:05.226005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:05.226005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:31:05.226005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:05.226005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:05.226005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:05.226005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:31:05.226005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:05.226005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:31:05.226005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:31:05.226005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:05.226005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:31:05.226005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:05.226843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:31:05.226843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:31:05.226843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:05.226843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:31:05.226843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:05.226843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:31:05.226843 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:31:05.226843 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:31:05.226843 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:31:05.226843 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:31:05.227326 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:31:05.227326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:05.227326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:31:05.227326 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:05.227326 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:31:05.227326 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:31:05.227326 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:05.227326 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:31:05.227326 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:05.227326 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:31:05.228134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:05.228134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:31:05.228134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:31:05.228134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:31:05.228134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:05.228134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:31:05.228134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:05.228134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:31:05.228134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:05.228134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:31:05.228134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:05.228134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:31:05.228134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:05.228134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:31:05.228134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:31:05.228875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:05.228875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:31:05.228875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:05.228875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:31:05.228875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:31:05.228875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:05.228875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:31:05.228875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:05.228875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:31:05.228875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:05.228875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:31:05.228875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:05.228875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:31:05.228875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:31:05.229558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:31:05.229558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:31:05.229558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:31:05.229558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:31:05.229558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:31:05.229558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:05.229558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:05.229558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:05.229558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:31:05.229558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:05.229558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:05.229558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:05.229558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:31:05.230132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:05.230132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:31:05.230132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:31:05.230132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:05.230132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:31:05.230132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:05.230132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:31:05.230132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:31:05.230132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:05.230132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:31:05.230132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:05.230132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:31:05.230689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:05.230689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:31:05.230689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:31:05.230689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:31:05.230689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:31:05.230689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:31:05.230689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:05.230689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:31:05.230689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:31:05.230689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:31:05.230689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:31:05.230689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:31:05.230689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:05.230689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:05.230689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:31:05.230689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:31:05.231382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:31:05.231382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:31:05.231382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:31:05.231382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:31:05.231382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:31:05.231382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:05.231382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:05.231382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:05.231382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:31:05.231382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:05.231382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:05.231382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:05.231382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:31:05.231382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:05.231382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:31:05.231382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:31:05.231382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:05.231382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:31:05.232140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:05.232140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:31:05.232140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:31:05.232140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:05.232140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:31:05.232140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:05.232140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:31:05.246549 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:31:05.246905 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:31:05.247082 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:31:05.247274 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:31:05.247879 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:31:05.248294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:05.249200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:31:05.249200 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:05.249200 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:31:05.249200 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:31:05.249200 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:05.249200 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:31:05.249200 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:05.249200 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:31:05.250429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:05.250429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:31:05.250429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:31:05.250429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:31:05.250429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:05.250429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:31:05.250429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:05.250429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:31:05.250429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:05.250429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:31:05.250429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:05.250429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:31:05.250429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:05.250429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:31:05.250429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:31:05.251084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:05.251084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:31:05.251084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:05.251084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:31:05.251084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:31:05.251084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:05.251084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:31:05.251084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:05.251084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:31:05.251981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:05.251981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:31:05.251981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:05.251981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:31:05.251981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:31:05.251981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:31:05.251981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:31:05.251981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:31:05.251981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:31:05.251981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:05.251981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:05.252664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:05.252664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:31:05.252664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:05.252664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:05.252664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:05.252664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:31:05.252664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:05.252664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:31:05.252664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:31:05.252664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:05.252664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:31:05.252664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:05.252664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:31:05.252664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:31:05.253282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:05.253282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:31:05.253282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:05.253282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:31:05.253493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:05.253493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:31:05.253493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:31:05.253493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:31:05.253493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:31:05.253493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:31:05.253493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:05.253493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:31:05.253493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:31:05.253493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:31:05.253493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:31:05.253493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:31:05.253493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:05.253493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:05.253493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:31:05.253493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:31:05.253493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:31:05.254215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:31:05.254215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:31:05.254215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:31:05.254215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:05.254215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:05.254215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:05.254215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:31:05.254215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:05.254215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:05.254215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:05.254215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:31:05.254215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:05.254215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:31:05.254215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:31:05.254215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:05.254215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:31:05.254215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:05.254215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:31:05.254215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:31:05.255032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:05.255032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:31:05.255032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:05.255032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:31:05.255032 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:31:05.255274 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:31:05.255274 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:31:05.255397 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:31:05.256004 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:31:05.256363 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:31:05.256626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:05.257975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:31:05.257975 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:05.257975 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:31:05.257975 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:31:05.257975 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:05.257975 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:31:05.257975 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:05.257975 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:31:05.258809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:05.258809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:31:05.258809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:05.258809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:31:05.258809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:31:05.258809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:31:05.258809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:31:05.258809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:05.258809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:05.258809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:05.258809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:31:05.258809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:05.259257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:05.259257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:05.259257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:31:05.259257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:05.259257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:31:05.259257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:31:05.259257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:05.259257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:31:05.259257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:05.259257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:31:05.259257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:31:05.259257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:05.259257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:31:05.259619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:05.259619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:31:05.259819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:05.259819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:31:05.259819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:31:05.259819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:31:05.259819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:31:05.259819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:31:05.259819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:05.259819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:31:05.259819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:31:05.259819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:31:05.259819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:31:05.259819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:31:05.259819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:05.259819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:05.259819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:31:05.259819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:31:05.260251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:31:05.260251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:31:05.260251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:05.260251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:05.260251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:05.260251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:31:05.260251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:05.260251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:05.260251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:05.260251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:31:05.260251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:05.260251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:31:05.260251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:31:05.260251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:05.260251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:31:05.260251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:05.260251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:31:05.260251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:31:05.260251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:05.260818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:31:05.260818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:05.260818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:31:10.010829 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:31:10.011732 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:31:10.011732 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:31:10.012958 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:31:10.050631 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:31:10.051034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:10.051981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:31:10.051981 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:10.051981 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:31:10.051981 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:31:10.051981 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:10.051981 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:31:10.051981 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:10.051981 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:31:10.052951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:10.052951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:31:10.052951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:10.052951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:31:10.052951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:31:10.052951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:31:10.052951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:31:10.052951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:10.052951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:10.052951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:10.052951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:31:10.052951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:10.053770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:10.053770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:10.053770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:31:10.053770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:10.053770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:31:10.053770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:31:10.053770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:10.053770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:31:10.053770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:10.053770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:31:10.053770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:31:10.053770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:10.053770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:31:10.054584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:10.054584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:31:10.054584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:10.054584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:31:10.054584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:31:10.054584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:31:10.054584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:31:10.054584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:31:10.054584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:10.054584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:31:10.054584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:31:10.054584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:31:10.054584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:31:10.054584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:31:10.054584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:10.055466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:10.055466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:31:10.055466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:31:10.055466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:31:10.055466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:31:10.055466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:10.055466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:10.055466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:10.055466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:31:10.055466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:10.055466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:10.055466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:10.055466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:31:10.055466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:10.055466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:31:10.055466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:31:10.055466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:10.055466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:31:10.055466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:10.056515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:31:10.056515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:31:10.056515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:10.056515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:31:10.056515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:10.056515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:31:10.218052 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:31:10.218732 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:31:10.218932 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:31:10.220108 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:31:10.220517 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:31:10.220778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:10.221742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:31:10.221742 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:10.221742 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:31:10.221742 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:31:10.221742 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:10.221742 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:31:10.221742 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:10.221742 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:31:10.223212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:10.223212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:31:10.223212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:10.223212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:31:10.223212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:31:10.223212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:31:10.223212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:31:10.223212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:10.223212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:10.223212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:10.223212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:31:10.223212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:10.223831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:10.223831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:10.223831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:31:10.223831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:10.223831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:31:10.223831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:31:10.223831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:10.223831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:31:10.223831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:10.223831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:31:10.223831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:31:10.223831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:10.223831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:31:10.224370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:10.224370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:31:10.224819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:10.224819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:31:10.224819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:31:10.224819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:31:10.224819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:31:10.224819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:31:10.224819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:10.224819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:31:10.224819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:31:10.224819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:31:10.224819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:31:10.224819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:31:10.224819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:10.224819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:10.224819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:31:10.224819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:31:10.225551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:31:10.225551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:31:10.225551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:10.225551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:10.225551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:10.225551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:31:10.225551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:10.225551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:10.225551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:10.225551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:31:10.225551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:10.225551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:31:10.225551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:31:10.225551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:10.225551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:31:10.225551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:10.225551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:31:10.225551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:31:10.225551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:10.226457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:31:10.226457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:10.226457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:31:10.226457 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:31:10.226457 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:31:10.226457 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:31:10.226457 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:31:10.226457 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:31:10.226873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:10.227182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:31:10.227182 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:10.227182 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:31:10.227182 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:31:10.227182 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:10.227182 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:31:10.227182 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:10.227182 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:31:10.228371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:10.228371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:31:10.228371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:31:10.228371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:31:10.228371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:10.228371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:31:10.228371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:10.228371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:31:10.228371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:10.228371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:31:10.228371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:10.228371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:31:10.228371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:10.228371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:31:10.228371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:31:10.228856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:10.228856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:31:10.228856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:10.228856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:31:10.228856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:31:10.228856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:10.228856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:31:10.228856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:10.228856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:31:10.229845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:10.229845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:31:10.229845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:10.229845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:31:10.229845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:31:10.229845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:31:10.229845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:31:10.229845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:31:10.229845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:31:10.229845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:31:10.229845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:10.229845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:10.230329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:10.230329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:31:10.230329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:10.230329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:10.230329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:10.230329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:31:10.230329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:10.230329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:31:10.230329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:31:10.230329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:10.230329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:31:10.230329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:10.230329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:31:10.230329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:31:10.230711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:10.230711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:31:10.230711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:10.230711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:31:10.231480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:10.231480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:31:10.231480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:31:10.231480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:31:10.231480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:31:10.231480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:31:10.231480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:10.231480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:31:10.231480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:31:10.231480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:31:10.231480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:31:10.231480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:31:10.231480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:10.231480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:10.231480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:31:10.231480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:31:10.231930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:31:10.231930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:31:10.231930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:31:10.231930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:31:10.231930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:31:10.231930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:10.231930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:10.231930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:10.231930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:31:10.231930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:10.231930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:10.231930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:10.231930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:31:10.231930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:10.231930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:31:10.231930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:31:10.231930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:10.231930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:31:10.232466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:10.232466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:31:10.232466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:31:10.232466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:10.232466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:31:10.232466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:10.232466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:31:10.246987 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:31:10.247427 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:31:10.247614 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:31:10.247807 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:31:10.248344 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:31:10.248845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:10.249674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:31:10.249674 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:10.249674 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:31:10.249674 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:31:10.249674 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:10.249674 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:31:10.249674 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:10.249674 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:31:10.250876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:10.250876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:31:10.250876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:31:10.250876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:31:10.250876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:10.250876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:31:10.250876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:10.250876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:31:10.250876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:10.250876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:31:10.250876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:10.250876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:31:10.250876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:10.250876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:31:10.250876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:31:10.251537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:10.251537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:31:10.251537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:10.251537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:31:10.251537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:31:10.251537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:10.251537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:31:10.251537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:10.251537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:31:10.252247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:10.252247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:31:10.252247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:10.252247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:31:10.252247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:31:10.252247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:31:10.252247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:31:10.252247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:31:10.252247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:31:10.252247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:10.252247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:10.252782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:10.252782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:31:10.252782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:10.252782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:10.252782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:10.252782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:31:10.252782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:10.252782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:31:10.252782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:31:10.252782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:10.252782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:31:10.252782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:10.252782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:31:10.252782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:31:10.253378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:10.253378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:31:10.253378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:10.253378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:31:10.253975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:10.253975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:31:10.253975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:31:10.253975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:31:10.253975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:31:10.253975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:31:10.253975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:10.253975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:31:10.253975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:31:10.253975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:31:10.253975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:31:10.253975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:31:10.253975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:10.253975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:10.253975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:31:10.253975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:31:10.253975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:31:10.254757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:31:10.254757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:31:10.254757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:31:10.254757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:10.254757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:10.254757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:10.254757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:31:10.254757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:10.254757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:10.254757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:10.254757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:31:10.254757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:10.254757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:31:10.254757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:31:10.254757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:10.254757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:31:10.254757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:10.254757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:31:10.254757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:31:10.255610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:10.255610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:31:10.255610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:10.255610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:31:10.255610 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:31:10.255610 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:31:10.255610 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:31:10.255987 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:31:10.256213 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:31:10.256584 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:31:10.256828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:10.257510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:31:10.257510 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:10.257510 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:31:10.257510 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:31:10.257510 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:10.257510 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:31:10.257510 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:10.257510 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:31:10.258303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:10.258303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:31:10.258303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:10.258303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:31:10.258303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:31:10.258303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:31:10.258303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:31:10.258303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:10.258303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:10.258303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:10.258303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:31:10.258303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:10.258666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:10.258666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:10.258666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:31:10.258666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:10.258666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:31:10.258666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:31:10.258666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:10.258666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:31:10.258666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:10.258666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:31:10.258666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:31:10.258666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:10.258666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:31:10.259012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:10.259012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:31:10.259265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:10.259265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:31:10.259265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:31:10.259265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:31:10.259265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:31:10.259265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:31:10.259265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:10.259265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:31:10.259265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:31:10.259265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:31:10.259265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:31:10.259265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:31:10.259265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:10.259265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:10.259265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:31:10.259265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:31:10.259728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:31:10.259728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:31:10.259728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:10.259728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:10.259728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:10.259728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:31:10.259728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:10.259728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:10.259728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:10.259728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:31:10.259728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:10.259728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:31:10.259728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:31:10.259728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:10.259728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:31:10.259728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:10.259728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:31:10.259728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:31:10.259728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:10.260229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:31:10.260229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:10.260229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:31:15.011574 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:31:15.012531 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:31:15.012531 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:31:15.013666 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:31:15.048655 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:31:15.049204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:15.050096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:31:15.050096 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:15.050096 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:31:15.050096 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:31:15.050096 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:15.050096 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:31:15.050096 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:15.050096 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:31:15.050842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:15.050842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:31:15.050842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:15.050842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:31:15.050842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:31:15.050842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:31:15.050842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:31:15.050842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:15.050842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:15.050842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:15.050842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:31:15.050842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:15.051356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:15.051356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:15.051356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:31:15.051356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:15.051356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:31:15.051356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:31:15.051356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:15.051356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:31:15.051356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:15.051356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:31:15.051356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:31:15.051356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:15.051356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:31:15.051790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:15.051790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:31:15.051790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:15.051790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:31:15.051790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:31:15.051790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:31:15.051790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:31:15.051790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:31:15.051790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:15.051790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:31:15.051790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:31:15.051790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:31:15.051790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:31:15.051790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:31:15.051790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:15.052279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:15.052279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:31:15.052279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:31:15.052279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:31:15.052279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:31:15.052279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:15.052279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:15.052279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:15.052279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:31:15.052279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:15.052279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:15.052279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:15.052279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:31:15.052279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:15.052279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:31:15.052279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:31:15.052279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:15.052279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:31:15.052279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:15.053021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:31:15.053021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:31:15.053021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:15.053021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:31:15.053021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:15.053021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:31:15.218774 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:31:15.219359 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:31:15.219578 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:31:15.220737 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:31:15.221140 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:31:15.221387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:15.222309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:31:15.222309 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:15.222309 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:31:15.222309 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:31:15.222309 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:15.222309 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:31:15.222309 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:15.222309 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:31:15.223816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:15.223816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:31:15.223816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:15.223816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:31:15.223816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:31:15.223816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:31:15.223816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:31:15.223816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:15.223816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:15.223816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:15.223816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:31:15.223816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:15.224469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:15.224469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:15.224469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:31:15.224469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:15.224469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:31:15.224469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:31:15.224469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:15.224469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:31:15.224469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:15.224469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:31:15.224469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:31:15.224469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:15.224469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:31:15.225073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:15.225073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:31:15.225486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:15.225486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:31:15.225486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:31:15.225486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:31:15.225486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:31:15.225486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:31:15.225486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:15.225486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:31:15.225486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:31:15.225486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:31:15.225486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:31:15.225486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:31:15.225486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:15.225486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:15.225486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:31:15.225486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:31:15.226180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:31:15.226180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:31:15.226180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:15.226180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:15.226180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:15.226180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:31:15.226180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:15.226180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:15.226180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:15.226180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:31:15.226180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:15.226180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:31:15.226180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:31:15.226180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:15.226180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:31:15.226180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:15.226180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:31:15.226180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:31:15.226180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:15.226995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:31:15.226995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:15.226995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:31:15.226995 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:31:15.226995 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:31:15.226995 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:31:15.226995 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:31:15.226995 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:31:15.227437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:15.228021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:31:15.228021 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:15.228021 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:31:15.228021 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:31:15.228021 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:15.228021 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:31:15.228021 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:15.228021 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:31:15.229203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:15.229203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:31:15.229203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:31:15.229203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:31:15.229203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:15.229203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:31:15.229203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:15.229203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:31:15.229203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:15.229203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:31:15.229203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:15.229203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:31:15.229203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:15.229203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:31:15.229203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:31:15.229953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:15.229953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:31:15.229953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:15.229953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:31:15.229953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:31:15.229953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:15.229953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:31:15.229953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:15.229953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:31:15.230423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:15.230423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:31:15.230423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:15.230423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:31:15.230423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:31:15.230423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:31:15.230423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:31:15.230423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:31:15.230423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:31:15.230423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:31:15.230423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:15.230423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:15.231109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:15.231109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:31:15.231109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:15.231109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:15.231109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:15.231109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:31:15.231109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:15.231109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:31:15.231109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:31:15.231109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:15.231109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:31:15.231109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:15.231109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:31:15.231109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:31:15.231766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:15.231766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:31:15.231766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:15.231766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:31:15.231766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:15.231766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:31:15.231766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:31:15.231766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:31:15.231766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:31:15.231766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:31:15.231766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:15.231766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:31:15.231766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:31:15.231766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:31:15.232397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:31:15.232397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:31:15.232397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:15.232397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:15.232397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:31:15.232397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:31:15.232397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:31:15.232397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:31:15.232397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:31:15.232397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:31:15.232397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:31:15.232397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:15.232397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:15.232397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:15.232397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:31:15.232397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:15.232397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:15.232397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:15.232397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:31:15.233328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:15.233328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:31:15.233328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:31:15.233328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:15.233328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:31:15.233328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:15.233328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:31:15.233328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:31:15.233328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:15.233328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:31:15.233328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:15.233328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:31:15.247537 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:31:15.247862 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:31:15.248029 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:31:15.248216 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:31:15.248722 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:31:15.249094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:15.249854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:31:15.249854 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:15.249854 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:31:15.249854 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:31:15.249854 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:15.249854 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:31:15.249854 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:15.249854 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:31:15.250978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:15.250978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:31:15.250978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:31:15.250978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:31:15.250978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:15.250978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:31:15.250978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:15.250978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:31:15.250978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:15.250978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:31:15.250978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:15.250978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:31:15.250978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:15.250978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:31:15.250978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:31:15.251661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:15.251661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:31:15.251661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:15.251661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:31:15.251661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:31:15.251661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:15.251661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:31:15.251661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:15.251661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:31:15.252312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:15.252312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:31:15.252312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:15.252312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:31:15.252312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:31:15.252312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:31:15.252312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:31:15.252312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:31:15.252312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:31:15.252312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:15.252312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:15.252857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:15.252857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:31:15.252857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:15.252857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:15.252857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:15.252857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:31:15.252857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:15.252857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:31:15.252857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:31:15.252857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:15.252857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:31:15.252857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:15.252857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:31:15.252857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:31:15.253511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:15.253511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:31:15.253511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:15.253511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:31:15.253793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:15.253793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:31:15.253793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:31:15.253793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:31:15.253793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:31:15.253793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:31:15.253793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:15.253793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:31:15.253793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:31:15.253793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:31:15.253793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:31:15.253793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:31:15.253793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:15.253793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:15.253793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:31:15.253793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:31:15.253793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:31:15.254662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:31:15.254662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:31:15.254662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:31:15.254662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:15.254662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:15.254662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:15.254662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:31:15.254662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:15.254662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:15.254662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:15.254662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:31:15.254662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:15.254662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:31:15.254662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:31:15.254662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:15.254662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:31:15.254662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:15.254662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:31:15.254662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:31:15.255539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:15.255539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:31:15.255539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:15.255539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:31:15.255753 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:31:15.255914 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:31:15.255989 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:31:15.256146 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:31:15.256653 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:31:15.257012 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:31:15.257256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:15.257940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:31:15.257940 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:15.257940 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:31:15.257940 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:31:15.257940 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:15.257940 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:31:15.257940 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:15.257940 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:31:15.258730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:15.258730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:31:15.258730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:15.258730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:31:15.258730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:31:15.258730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:31:15.258730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:31:15.258730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:15.258730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:15.258730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:15.258730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:31:15.258730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:15.259085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:15.259085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:15.259085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:31:15.259085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:15.259085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:31:15.259085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:31:15.259085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:15.259085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:31:15.259085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:15.259085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:31:15.259085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:31:15.259085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:15.259085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:31:15.259440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:15.259440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:31:15.259662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:15.259662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:31:15.259662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:31:15.259662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:31:15.259662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:31:15.259662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:31:15.259662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:15.259662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:31:15.259662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:31:15.259662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:31:15.259662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:31:15.259662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:31:15.259662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:15.259662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:15.259662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:31:15.259662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:31:15.260110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:31:15.260110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:31:15.260110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:15.260110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:15.260110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:15.260110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:31:15.260110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:15.260110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:15.260110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:15.260110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:31:15.260110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:15.260110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:31:15.260110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:31:15.260110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:15.260110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:31:15.260110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:15.260110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:31:15.260110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:31:15.260110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:15.260615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:31:15.260615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:15.260615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:31:20.012285 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:31:20.013392 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:31:20.013392 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:31:20.014459 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:31:20.046575 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:31:20.046575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:20.048656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:31:20.048656 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:20.048656 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:31:20.048656 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:31:20.048656 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:20.048656 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:31:20.048656 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:20.048656 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:31:20.048656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:20.048656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:31:20.048656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:20.048656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:31:20.048656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:31:20.048656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:31:20.048656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:31:20.049789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:20.049789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:20.049789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:20.049789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:31:20.049789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:20.049789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:20.049789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:20.049789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:31:20.049789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:20.049789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:31:20.049789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:31:20.049789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:20.049789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:31:20.049789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:20.050562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:31:20.050562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:31:20.050562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:20.050562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:31:20.050562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:20.050562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:31:20.050562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:20.050562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:31:20.050562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:31:20.050562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:31:20.050562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:31:20.050562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:31:20.050562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:20.050562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:31:20.050978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:31:20.050978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:31:20.050978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:31:20.050978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:31:20.050978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:20.050978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:20.050978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:31:20.050978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:31:20.050978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:31:20.050978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:31:20.050978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:20.050978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:20.050978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:20.050978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:31:20.050978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:20.050978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:20.050978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:20.050978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:31:20.050978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:20.050978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:31:20.050978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:31:20.052768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:20.052768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:31:20.052768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:20.052768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:31:20.052768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:31:20.052768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:20.052768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:31:20.052768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:20.052768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:31:20.219567 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:31:20.220113 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:31:20.220268 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:31:20.221689 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:31:20.222040 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:31:20.222326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:20.223201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:31:20.223201 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:20.223201 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:31:20.223201 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:31:20.223201 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:20.223201 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:31:20.223201 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:20.223201 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:31:20.224673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:20.224673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:31:20.224673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:20.224673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:31:20.224673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:31:20.224673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:31:20.224673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:31:20.224673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:20.224673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:20.224673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:20.224673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:31:20.224673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:20.225825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:20.225825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:20.225825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:31:20.225825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:20.225825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:31:20.225825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:31:20.225825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:20.225825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:31:20.225825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:20.225825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:31:20.225825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:31:20.225825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:20.225825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:31:20.227176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:20.227176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:31:20.227176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:20.227176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:31:20.227176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:31:20.227176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:31:20.227176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:31:20.227176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:31:20.227176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:20.227176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:31:20.227176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:31:20.227176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:31:20.227176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:31:20.227176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:31:20.227176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:20.228003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:20.228003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:31:20.228003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:31:20.228003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:31:20.228003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:31:20.228003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:20.228003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:20.228003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:20.228003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:31:20.228003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:20.228003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:20.228003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:20.228003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:31:20.228003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:20.228003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:31:20.228003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:31:20.228003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:20.228003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:31:20.228003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:20.228763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:31:20.228763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:31:20.228763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:20.228763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:31:20.228763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:20.228763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:31:20.228763 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:31:20.228763 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:31:20.228763 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:31:20.228763 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:31:20.229293 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:31:20.229293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:20.229293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:31:20.229293 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:20.229293 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:31:20.229293 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:31:20.229293 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:20.229293 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:31:20.229293 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:20.229293 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:31:20.230272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:20.230272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:31:20.230272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:31:20.230272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:31:20.230272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:20.230272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:31:20.230272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:20.230272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:31:20.230272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:20.230272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:31:20.230272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:20.230272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:31:20.230272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:20.230272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:31:20.230272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:31:20.231155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:20.231155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:31:20.231155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:20.231155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:31:20.231155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:31:20.231155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:20.231155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:31:20.231155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:20.231155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:31:20.231155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:20.231155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:31:20.231155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:20.231155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:31:20.231155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:31:20.231979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:31:20.231979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:31:20.231979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:31:20.231979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:31:20.231979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:31:20.231979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:20.231979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:20.231979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:20.231979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:31:20.231979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:20.231979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:20.231979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:20.231979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:31:20.232653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:20.232653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:31:20.232653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:31:20.232653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:20.232653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:31:20.232653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:20.232653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:31:20.232653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:31:20.232653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:20.232653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:31:20.232653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:20.232653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:31:20.233303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:20.233303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:31:20.233303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:31:20.233303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:31:20.233303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:31:20.233303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:31:20.233303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:20.233303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:31:20.233303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:31:20.233303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:31:20.233303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:31:20.233303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:31:20.233303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:20.233303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:20.233303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:31:20.233303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:31:20.234382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:31:20.234382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:31:20.234382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:31:20.234382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:31:20.234382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:31:20.234382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:20.234382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:20.234382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:20.234382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:31:20.234382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:20.234382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:20.234382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:20.234382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:31:20.234382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:20.234382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:31:20.234382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:31:20.234382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:20.234382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:31:20.235107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:20.235107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:31:20.235107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:31:20.235107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:20.235107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:31:20.235107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:20.235107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:31:20.247991 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:31:20.248340 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:31:20.248516 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:31:20.248716 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:31:20.249193 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:31:20.249585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:20.250263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:31:20.250263 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:20.250263 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:31:20.250263 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:31:20.250263 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:20.250263 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:31:20.250263 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:20.250263 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:31:20.251394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:20.251394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:31:20.251394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:31:20.251394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:31:20.251394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:20.251394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:31:20.251394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:20.251394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:31:20.251394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:20.251394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:31:20.251394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:20.251394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:31:20.251394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:20.251394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:31:20.251394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:31:20.252039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:20.252039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:31:20.252039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:20.252039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:31:20.252039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:31:20.252039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:20.252039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:31:20.252039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:20.252039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:31:20.252998 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:20.252998 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:31:20.252998 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:20.252998 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:31:20.252998 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:31:20.252998 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:31:20.252998 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:31:20.252998 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:31:20.252998 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:31:20.252998 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:20.252998 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:20.254212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:20.254212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:31:20.254212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:20.254212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:20.254212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:20.254212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:31:20.254212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:20.254212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:31:20.254212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:31:20.254212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:20.254212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:31:20.254212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:20.254212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:31:20.254212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:31:20.255112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:20.255112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:31:20.255112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:20.255112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:31:20.255112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:20.255112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:31:20.255112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:31:20.255112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:31:20.255112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:31:20.255112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:31:20.255112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:20.255112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:31:20.255112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:31:20.255112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:31:20.255541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:31:20.255541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:31:20.255541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:20.255541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:20.255541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:31:20.255541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:31:20.255541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:31:20.255541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:31:20.255541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:31:20.255541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:31:20.255541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:20.255541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:20.255541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:20.255541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:31:20.255541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:20.255541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:20.255541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:20.255541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:31:20.255541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:20.256006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:31:20.256006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:31:20.256006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:20.256006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:31:20.256006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:20.256006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:31:20.256006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:31:20.256006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:20.256006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:31:20.256006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:20.256006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:31:20.256267 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:31:20.256417 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:31:20.256865 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:31:20.256865 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:31:20.257128 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:31:20.257512 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:31:20.257731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:20.258396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:31:20.258396 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:20.258396 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:31:20.258396 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:31:20.258396 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:20.258396 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:31:20.258396 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:20.258396 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:31:20.259240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:20.259240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:31:20.259240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:20.259240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:31:20.259240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:31:20.259240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:31:20.259240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:31:20.259240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:20.259240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:20.259240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:20.259240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:31:20.259240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:20.259944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:20.259944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:20.259944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:31:20.259944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:20.259944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:31:20.259944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:31:20.259944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:20.259944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:31:20.259944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:20.259944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:31:20.259944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:31:20.259944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:20.259944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:31:20.260349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:20.260349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:31:20.260349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:20.260349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:31:20.260349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:31:20.260349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:31:20.260349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:31:20.260349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:31:20.260349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:20.260349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:31:20.260349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:31:20.260349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:31:20.260349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:31:20.260349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:31:20.260349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:20.260742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:20.260742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:31:20.260742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:31:20.260742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:31:20.260742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:31:20.260742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:20.260742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:20.260742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:20.260742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:31:20.260742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:20.260742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:20.260742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:20.260742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:31:20.260742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:20.260742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:31:20.260742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:31:20.260742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:20.260742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:31:20.260742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:20.261198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:31:20.261198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:31:20.261198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:20.261198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:31:20.261198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:20.261198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:31:25.013187 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:31:25.014156 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:31:25.014264 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:31:25.015617 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:31:25.017695 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG dbcounter [-] [105581] Writing DB stats neutron:SELECT=240 {{(pid=105581) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 14:31:25.050545 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:31:25.050878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:25.051915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:31:25.051915 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:25.051915 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:31:25.051915 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:31:25.051915 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:25.051915 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:31:25.051915 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:25.051915 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:31:25.052654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:25.052654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:31:25.052654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:25.052654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:31:25.052654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:31:25.052654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:31:25.052654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:31:25.052654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:25.052654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:25.052654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:25.052654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:31:25.052654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:25.053329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:25.053329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:25.053329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:31:25.053329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:25.053329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:31:25.053329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:31:25.053329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:25.053329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:31:25.053329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:25.053329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:31:25.053329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:31:25.053329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:25.053329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:31:25.054515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:25.054515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:31:25.054515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:25.054515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:31:25.054515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:31:25.054515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:31:25.054515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:31:25.054515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:31:25.054515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:25.054515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:31:25.054515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:31:25.054515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:31:25.054515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:31:25.054515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:31:25.054515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:25.055285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:25.055285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:31:25.055285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:31:25.055285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:31:25.055285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:31:25.055285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:25.055285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:25.055285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:25.055285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:31:25.055285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:25.055285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:25.055285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:25.055285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:31:25.055285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:25.055285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:31:25.055285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:31:25.055285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:25.055285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:31:25.055285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:25.056297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:31:25.056297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:31:25.056297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:25.056297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:31:25.056297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:25.056297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:31:25.220597 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:31:25.221336 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:31:25.221457 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:31:25.223240 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:31:25.223500 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:31:25.223655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:25.224271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:31:25.224271 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:25.224271 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:31:25.224271 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:31:25.224271 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:25.224271 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:31:25.224271 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:25.224271 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:31:25.225282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:25.225282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:31:25.225282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:25.225282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:31:25.225282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:31:25.225282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:31:25.225282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:31:25.225282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:25.225282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:25.225282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:25.225282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:31:25.225282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:25.225969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:25.225969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:25.225969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:31:25.225969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:25.225969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:31:25.225969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:31:25.225969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:25.225969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:31:25.225969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:25.225969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:31:25.225969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:31:25.225969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:25.225969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:31:25.226588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:25.226588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:31:25.226588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:25.226588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:31:25.226588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:31:25.226588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:31:25.226588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:31:25.226588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:31:25.226588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:25.226588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:31:25.226588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:31:25.226588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:31:25.226588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:31:25.226588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:31:25.226588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:25.227277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:25.227277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:31:25.227277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:31:25.227277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:31:25.227277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:31:25.227277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:25.227277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:25.227277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:25.227277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:31:25.227277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:25.227277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:25.227277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:25.227277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:31:25.227277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:25.227277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:31:25.227277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:31:25.227277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:25.227277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:31:25.227277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:25.228178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:31:25.228178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:31:25.228178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:25.228178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:31:25.228178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:25.228178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:31:25.228178 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:31:25.228178 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:31:25.228178 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:31:25.228178 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:31:25.228724 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:31:25.228724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:25.230844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:31:25.230844 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:25.230844 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:31:25.230844 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:31:25.230844 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:25.230844 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:31:25.230844 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:25.230844 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:31:25.231627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:25.231627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:31:25.231627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:31:25.231627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:31:25.231627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:25.231627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:31:25.231627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:25.231627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:31:25.231627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:25.231627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:31:25.231627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:25.231627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:31:25.231627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:25.231627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:31:25.231627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:31:25.232115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:25.232115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:31:25.232115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:25.232115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:31:25.232115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:31:25.232115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:25.232115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:31:25.232115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:25.232115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:31:25.232497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:25.232497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:31:25.232497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:25.232497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:31:25.232497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:31:25.232497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:31:25.232497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:31:25.232497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:31:25.232497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:31:25.232497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:31:25.232497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:25.232497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:25.232875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:25.232875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:31:25.232875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:25.232875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:25.232875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:25.232875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:31:25.232875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:25.232875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:31:25.232875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:31:25.232875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:25.232875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:31:25.232875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:25.232875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:31:25.232875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:31:25.233318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:25.233318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:31:25.233318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:25.233318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:31:25.233572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:25.233572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:31:25.233572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:31:25.233572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:31:25.233572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:31:25.233572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:31:25.233572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:25.233572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:31:25.233572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:31:25.233572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:31:25.233572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:31:25.233572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:31:25.233572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:25.233572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:25.233572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:31:25.233572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:31:25.234386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:31:25.234386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:31:25.234386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:31:25.234386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:31:25.234386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:31:25.234386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:25.234386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:25.234386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:25.234386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:31:25.234386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:25.234386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:25.234386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:25.234386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:31:25.234386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:25.234386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:31:25.234386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:31:25.234386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:25.234386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:31:25.235137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:25.235137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:31:25.235137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:31:25.235137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:25.235137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:31:25.235137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:25.235137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:31:25.248510 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:31:25.249201 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:31:25.249326 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:31:25.249499 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:31:25.250095 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:31:25.250323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:25.250894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:31:25.250894 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:25.250894 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:31:25.250894 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:31:25.250894 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:25.250894 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:31:25.250894 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:25.250894 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:31:25.252075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:25.252075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:31:25.252075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:31:25.252075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:31:25.252075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:25.252075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:31:25.252075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:25.252075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:31:25.252075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:25.252075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:31:25.252075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:25.252075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:31:25.252075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:25.252075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:31:25.252075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:31:25.252520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:25.252520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:31:25.252520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:25.252520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:31:25.252520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:31:25.252520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:25.252520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:31:25.252520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:25.252520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:31:25.253485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:25.253485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:31:25.253485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:25.253485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:31:25.253485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:31:25.253485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:31:25.253485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:31:25.253485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:31:25.253485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:31:25.253485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:25.253485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:25.253820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:25.253820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:31:25.253820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:25.253820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:25.253820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:25.253820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:31:25.253820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:25.253820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:31:25.253820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:31:25.253820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:25.253820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:31:25.253820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:25.253820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:31:25.253820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:31:25.254211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:25.254211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:31:25.254211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:25.254211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:31:25.255286 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:25.255286 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:31:25.255286 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:31:25.255286 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:31:25.255286 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:31:25.255286 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:31:25.255286 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:25.255286 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:31:25.255286 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:31:25.255286 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:31:25.255286 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:31:25.255286 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:31:25.255286 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:25.255286 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:25.255286 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:31:25.255286 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:31:25.255286 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:31:25.255850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:31:25.255850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:31:25.255850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:31:25.255850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:25.255850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:25.255850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:25.255850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:31:25.255850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:25.255850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:25.255850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:25.255850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:31:25.255850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:25.255850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:31:25.255850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:31:25.255850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:25.255850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:31:25.255850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:25.255850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:31:25.255850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:31:25.256348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:25.256348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:31:25.256348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:25.256348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:31:25.256545 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:31:25.256926 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:31:25.257096 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:31:25.257328 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:31:25.257706 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:31:25.257925 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:31:25.258071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:25.258668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:31:25.258668 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:25.258668 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:31:25.258668 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:31:25.258668 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:25.258668 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:31:25.258668 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:25.258668 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:31:25.259987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:25.259987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:31:25.259987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:25.259987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:31:25.259987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:31:25.259987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:31:25.259987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:31:25.259987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:25.259987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:25.259987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:25.259987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:31:25.259987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:25.260340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:25.260340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:25.260340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:31:25.260340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:25.260340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:31:25.260340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:31:25.260340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:25.260340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:31:25.260340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:25.260340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:31:25.260340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:31:25.260340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:25.260340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:31:25.260702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:25.260702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:31:25.261645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:25.261645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:31:25.261645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:31:25.261645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:31:25.261645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:31:25.261645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:31:25.261645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:25.261645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:31:25.261645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:31:25.261645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:31:25.261645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:31:25.261645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:31:25.261645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:25.261645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:25.261645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:31:25.261645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:31:25.262080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:31:25.262080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:31:25.262080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:25.262080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:25.262080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:25.262080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:31:25.262080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:25.262080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:25.262080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:25.262080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:31:25.262080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:25.262080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:31:25.262080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:31:25.262080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:25.262080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:31:25.262080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:25.262080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:31:25.262080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:31:25.262080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:25.262623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:31:25.262623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:25.262623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:31:30.013902 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:31:30.014573 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:31:30.014573 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:31:30.015300 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:31:30.047140 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:31:30.047748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:30.048908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:31:30.048908 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:30.048908 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:31:30.048908 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:31:30.048908 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:30.048908 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:31:30.048908 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:30.048908 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:31:30.049516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:30.049516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:31:30.049516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:30.049516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:31:30.049516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:31:30.049516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:31:30.049516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:31:30.049516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:30.049516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:30.049516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:30.049516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:31:30.049516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:30.050100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:30.050100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:30.050100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:31:30.050100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:30.050100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:31:30.050100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:31:30.050100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:30.050100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:31:30.050100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:30.050100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:31:30.050100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:31:30.050100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:30.050100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:31:30.050546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:30.050546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:31:30.050546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:30.050546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:31:30.050546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:31:30.050546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:31:30.050546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:31:30.050546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:31:30.050546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:30.050546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:31:30.050546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:31:30.050546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:31:30.050546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:31:30.050546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:31:30.050546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:30.051039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:30.051039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:31:30.051039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:31:30.051039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:31:30.051039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:31:30.051039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:30.051039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:30.051039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:30.051039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:31:30.051039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:30.051039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:30.051039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:30.051039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:31:30.051039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:30.051039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:31:30.051039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:31:30.051039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:30.051039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:31:30.051039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:30.051689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:31:30.051689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:31:30.051689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:30.051689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:31:30.051689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:30.051689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:31:30.221326 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:31:30.221857 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:31:30.222051 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:31:30.223021 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:31:30.223563 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:31:30.223756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:30.224415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:31:30.224415 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:30.224415 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:31:30.224415 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:31:30.224415 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:30.224415 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:31:30.224415 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:30.224415 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:31:30.225467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:30.225467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:31:30.225467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:30.225467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:31:30.225467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:31:30.225467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:31:30.225467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:31:30.225467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:30.225467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:30.225467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:30.225467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:31:30.225467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:30.225875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:30.225875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:30.225875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:31:30.225875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:30.225875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:31:30.225875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:31:30.225875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:30.225875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:31:30.225875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:30.225875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:31:30.225875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:31:30.225875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:30.225875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:31:30.226237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:30.226237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:31:30.226870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:30.226870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:31:30.226870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:31:30.226870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:31:30.226870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:31:30.226870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:31:30.226870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:30.226870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:31:30.226870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:31:30.226870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:31:30.226870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:31:30.226870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:31:30.226870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:30.226870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:30.226870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:31:30.226870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:31:30.227555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:31:30.227555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:31:30.227555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:30.227555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:30.227555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:30.227555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:31:30.227555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:30.227555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:30.227555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:30.227555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:31:30.227555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:30.227555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:31:30.227555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:31:30.227555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:30.227555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:31:30.227555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:30.227555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:31:30.227555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:31:30.227555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:30.228363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:31:30.228363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:30.228363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:31:30.228363 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:31:30.228363 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:31:30.228363 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:31:30.228363 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:31:30.228754 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:31:30.228754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:30.229655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:31:30.229655 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:30.229655 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:31:30.229655 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:31:30.229655 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:30.229655 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:31:30.229655 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:30.229655 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:31:30.230428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:30.230428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:31:30.230428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:31:30.230428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:31:30.230428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:30.230428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:31:30.230428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:30.230428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:31:30.230428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:30.230428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:31:30.230428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:30.230428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:31:30.230428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:30.230428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:31:30.230428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:31:30.231306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:30.231306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:31:30.231306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:30.231306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:31:30.231306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:31:30.231306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:30.231306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:31:30.231306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:30.231306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:31:30.231306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:30.231306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:31:30.231306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:30.231306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:31:30.231306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:31:30.232044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:31:30.232044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:31:30.232044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:31:30.232044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:31:30.232044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:31:30.232044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:30.232044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:30.232044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:30.232044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:31:30.232044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:30.232044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:30.232044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:30.232044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:31:30.232642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:30.232642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:31:30.232642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:31:30.232642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:30.232642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:31:30.232642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:30.232642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:31:30.232642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:31:30.232642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:30.232642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:31:30.232642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:30.232642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:31:30.233159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:30.233159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:31:30.233159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:31:30.233159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:31:30.233159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:31:30.233159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:31:30.233159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:30.233159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:31:30.233159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:31:30.233159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:31:30.233159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:31:30.233159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:31:30.233159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:30.233159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:30.233159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:31:30.233159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:31:30.233864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:31:30.233864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:31:30.233864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:31:30.233864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:31:30.233864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:31:30.233864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:30.233864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:30.233864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:30.233864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:31:30.233864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:30.233864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:30.233864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:30.233864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:31:30.233864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:30.233864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:31:30.233864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:31:30.233864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:30.233864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:31:30.234659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:30.234659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:31:30.234659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:31:30.234659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:30.234659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:31:30.234659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:30.234659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:31:30.249755 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:31:30.250121 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:31:30.250310 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:31:30.250547 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:31:30.251050 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:31:30.251424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:30.252187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:31:30.252187 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:30.252187 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:31:30.252187 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:31:30.252187 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:30.252187 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:31:30.252187 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:30.252187 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:31:30.253422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:30.253422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:31:30.253422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:31:30.253422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:31:30.253422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:30.253422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:31:30.253422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:30.253422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:31:30.253422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:30.253422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:31:30.253422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:30.253422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:31:30.253422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:30.253422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:31:30.253422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:31:30.254077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:30.254077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:31:30.254077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:30.254077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:31:30.254077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:31:30.254077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:30.254077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:31:30.254077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:30.254077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:31:30.254906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:30.254906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:31:30.254906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:30.254906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:31:30.254906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:31:30.254906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:31:30.254906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:31:30.254906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:31:30.254906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:31:30.254906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:30.254906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:30.255467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:30.255467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:31:30.255467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:30.255467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:30.255467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:30.255467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:31:30.255467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:30.255467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:31:30.255467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:31:30.255467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:30.255467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:31:30.255467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:30.255467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:31:30.255467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:31:30.256089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:30.256089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:31:30.256089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:30.256089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:31:30.256708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:30.256708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:31:30.256708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:31:30.256708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:31:30.256708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:31:30.256708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:31:30.256708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:30.256708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:31:30.256708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:31:30.256708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:31:30.256708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:31:30.256708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:31:30.256708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:30.256708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:30.256708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:31:30.256708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:31:30.256708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:31:30.257495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:31:30.257495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:31:30.257495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:31:30.257495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:30.257495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:30.257495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:30.257495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:31:30.257495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:30.257495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:30.257495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:30.257495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:31:30.257495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:30.257495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:31:30.257495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:31:30.257495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:30.257495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:31:30.257495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:30.257495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:31:30.257495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:31:30.258292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:30.258292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:31:30.258292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:30.258292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:31:30.258292 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:31:30.258292 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:31:30.258292 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:31:30.258292 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:31:30.258713 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:31:30.258713 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:31:30.258713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:30.259435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:31:30.259435 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:30.259435 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:31:30.259435 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:31:30.259435 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:30.259435 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:31:30.259435 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:30.259435 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:31:30.260251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:30.260251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:31:30.260251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:30.260251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:31:30.260251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:31:30.260251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:31:30.260251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:31:30.260251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:30.260251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:30.260251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:30.260251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:31:30.260251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:30.260851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:30.260851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:30.260851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:31:30.260851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:30.260851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:31:30.260851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:31:30.260851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:30.260851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:31:30.260851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:30.260851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:31:30.260851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:31:30.260851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:30.260851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:31:30.261226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:30.261226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:31:30.261226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:30.261226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:31:30.261226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:31:30.261226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:31:30.261226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:31:30.261226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:31:30.261226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:30.261226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:31:30.261226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:31:30.261226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:31:30.261226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:31:30.261226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:31:30.261226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:30.261652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:30.261652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:31:30.261652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:31:30.261652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:31:30.261652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:31:30.261652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:30.261652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:30.261652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:30.261652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:31:30.261652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:30.261652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:30.261652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:30.261652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:31:30.261652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:30.261652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:31:30.261652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:31:30.261652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:30.261652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:31:30.261652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:30.262178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:31:30.262178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:31:30.262178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:30.262178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:31:30.262178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:30.262178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:31:35.014582 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:31:35.016132 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:31:35.016327 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:31:35.017575 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:31:35.051734 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:31:35.051969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:35.052597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:31:35.052597 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:35.052597 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:31:35.052597 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:31:35.052597 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:35.052597 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:31:35.052597 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:35.052597 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:31:35.053314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:35.053314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:31:35.053314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:35.053314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:31:35.053314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:31:35.053314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:31:35.053314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:31:35.053314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:35.053314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:35.053314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:35.053314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:31:35.053314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:35.053784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:35.053784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:35.053784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:31:35.053784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:35.053784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:31:35.053784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:31:35.053784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:35.053784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:31:35.053784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:35.053784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:31:35.053784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:31:35.053784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:35.053784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:31:35.054464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:35.054464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:31:35.054464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:35.054464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:31:35.054464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:31:35.054464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:31:35.054464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:31:35.054464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:31:35.054464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:35.054464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:31:35.054464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:31:35.054464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:31:35.054464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:31:35.054464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:31:35.054464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:35.055347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:35.055347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:31:35.055347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:31:35.055347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:31:35.055347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:31:35.055347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:35.055347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:35.055347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:35.055347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:31:35.055347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:35.055347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:35.055347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:35.055347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:31:35.055347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:35.055347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:31:35.055347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:31:35.055347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:35.055347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:31:35.055347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:35.056938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:31:35.056938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:31:35.056938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:35.056938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:31:35.056938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:35.056938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:31:35.221915 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:31:35.222667 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:31:35.222870 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:31:35.223977 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:31:35.224239 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:31:35.224379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:35.226297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:31:35.226297 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:35.226297 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:31:35.226297 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:31:35.226297 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:35.226297 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:31:35.226297 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:35.226297 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:31:35.227311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:35.227311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:31:35.227311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:35.227311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:31:35.227311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:31:35.227311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:31:35.227311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:31:35.227311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:35.227311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:35.227311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:35.227311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:31:35.227311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:35.228071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:35.228071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:35.228071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:31:35.228071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:35.228071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:31:35.228071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:31:35.228071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:35.228071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:31:35.228071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:35.228071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:31:35.228071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:31:35.228071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:35.228071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:31:35.228656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:35.228656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:31:35.229033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:35.229033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:31:35.229033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:31:35.229033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:31:35.229033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:31:35.229033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:31:35.229033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:35.229033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:31:35.229033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:31:35.229033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:31:35.229033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:31:35.229033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:31:35.229033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:35.229033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:35.229033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:31:35.229033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:31:35.229928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:31:35.229928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:31:35.229928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:35.229928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:35.229928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:35.229928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:31:35.229928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:35.229928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:35.229928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:35.229928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:31:35.229928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:35.229928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:31:35.229928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:31:35.229928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:35.229928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:31:35.229928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:35.229928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:31:35.229928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:31:35.229928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:35.230755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:31:35.230755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:35.230755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:31:35.230755 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:31:35.230755 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:31:35.230755 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:31:35.230755 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:31:35.231634 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:31:35.231634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:35.232818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:31:35.232818 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:35.232818 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:31:35.232818 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:31:35.232818 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:35.232818 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:31:35.232818 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:35.232818 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:31:35.233434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:35.233434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:31:35.233434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:31:35.233434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:31:35.233434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:35.233434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:31:35.233434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:35.233434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:31:35.233434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:35.233434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:31:35.233434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:35.233434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:31:35.233434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:35.233434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:31:35.233434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:31:35.233984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:35.233984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:31:35.233984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:35.233984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:31:35.233984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:31:35.233984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:35.233984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:31:35.233984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:35.233984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:31:35.233984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:35.233984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:31:35.233984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:35.233984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:31:35.233984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:31:35.234432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:31:35.234432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:31:35.234432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:31:35.234432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:31:35.234432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:31:35.234432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:35.234432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:35.234432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:35.234432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:31:35.234432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:35.234432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:35.234432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:35.234432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:31:35.234791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:35.234791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:31:35.234791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:31:35.234791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:35.234791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:31:35.234791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:35.234791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:31:35.234791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:31:35.234791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:35.234791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:31:35.234791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:35.234791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:31:35.235125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:35.235125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:31:35.235125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:31:35.235125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:31:35.235125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:31:35.235125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:31:35.235125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:35.235125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:31:35.235125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:31:35.235125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:31:35.235125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:31:35.235125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:31:35.235125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:35.235125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:35.235125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:31:35.235125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:31:35.235606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:31:35.235606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:31:35.235606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:31:35.235606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:31:35.235606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:31:35.235606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:35.235606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:35.235606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:35.235606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:31:35.235606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:35.235606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:35.235606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:35.235606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:31:35.235606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:35.235606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:31:35.235606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:31:35.235606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:35.235606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:31:35.236066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:35.236066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:31:35.236066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:31:35.236066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:35.236066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:31:35.236066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:35.236066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:31:35.250220 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:31:35.250501 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:31:35.250637 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:31:35.250637 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:31:35.251187 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:31:35.251448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:35.251978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:31:35.251978 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:35.251978 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:31:35.251978 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:31:35.251978 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:35.251978 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:31:35.251978 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:35.251978 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:31:35.252555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:35.252555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:31:35.252555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:31:35.252555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:31:35.252555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:35.252555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:31:35.252555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:35.252555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:31:35.252555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:35.252555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:31:35.252555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:35.252555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:31:35.252555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:35.252555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:31:35.252555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:31:35.253572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:35.253572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:31:35.253572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:35.253572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:31:35.253572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:31:35.253572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:35.253572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:31:35.253572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:35.253572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:31:35.253572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:35.253572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:31:35.253572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:35.253572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:31:35.253572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:31:35.254563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:31:35.254563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:31:35.254563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:31:35.254563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:31:35.254563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:35.254563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:35.254563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:35.254563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:31:35.254563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:35.254563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:35.254563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:35.254563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:31:35.254563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:35.255219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:31:35.255219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:31:35.255219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:35.255219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:31:35.255219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:35.255219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:31:35.255219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:31:35.255219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:35.255219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:31:35.255219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:35.255219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:31:35.255219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:35.255461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:31:35.255461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:31:35.255461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:31:35.255461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:31:35.255461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:31:35.255461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:35.255461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:31:35.255461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:31:35.255461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:31:35.255461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:31:35.255461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:31:35.255461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:35.255461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:35.255461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:31:35.255461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:31:35.255461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:31:35.255461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:31:35.255461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:31:35.255461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:31:35.255834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:35.255834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:35.255834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:35.255834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:31:35.255834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:35.255834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:35.255834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:35.255834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:31:35.255834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:35.255834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:31:35.255834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:31:35.255834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:35.255834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:31:35.255834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:35.255834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:31:35.255834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:31:35.255834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:35.255834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:31:35.255834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:35.255834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:31:35.257259 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:31:35.257521 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:31:35.257629 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:31:35.257774 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:31:35.258225 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:31:35.258618 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:31:35.258847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:35.259680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:31:35.259680 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:35.259680 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:31:35.259680 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:31:35.259680 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:35.259680 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:31:35.259680 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:35.259680 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:31:35.260512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:35.260512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:31:35.260512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:35.260512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:31:35.260512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:31:35.260512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:31:35.260512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:31:35.260512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:35.260512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:35.260512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:35.260512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:31:35.260512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:35.260869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:35.260869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:35.260869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:31:35.260869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:35.260869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:31:35.260869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:31:35.260869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:35.260869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:31:35.260869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:35.260869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:31:35.260869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:31:35.260869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:35.260869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:31:35.261182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:35.261182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:31:35.261476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:35.261476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:31:35.261476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:31:35.261476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:31:35.261476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:31:35.261476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:31:35.261476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:35.261476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:31:35.261476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:31:35.261476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:31:35.261476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:31:35.261476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:31:35.261476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:35.261476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:35.261476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:31:35.261476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:31:35.261921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:31:35.261921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:31:35.261921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:35.261921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:35.261921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:35.261921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:31:35.261921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:35.261921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:35.261921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:35.261921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:31:35.261921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:35.261921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:31:35.261921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:31:35.261921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:35.261921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:31:35.261921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:35.261921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:31:35.261921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:31:35.261921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:35.262381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:31:35.262381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:35.262381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:31:40.016224 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:31:40.017040 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:31:40.017040 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:31:40.018277 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:31:40.055156 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:31:40.055594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:40.056508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:31:40.056508 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:40.056508 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:31:40.056508 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:31:40.056508 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:40.056508 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:31:40.056508 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:40.056508 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:31:40.057865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:40.057865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:31:40.057865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:40.057865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:31:40.057865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:31:40.057865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:31:40.057865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:31:40.057865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:40.057865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:40.057865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:40.057865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:31:40.057865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:40.058561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:40.058561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:40.058561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:31:40.058561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:40.058561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:31:40.058561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:31:40.058561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:40.058561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:31:40.058561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:40.058561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:31:40.058561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:31:40.058561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:40.058561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:31:40.059228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:40.059228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:31:40.059355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:40.059355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:31:40.059355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:31:40.059355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:31:40.059355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:31:40.059355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:31:40.059355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:40.059355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:31:40.059355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:31:40.059355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:31:40.059355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:31:40.059355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:31:40.059355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:40.059355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:40.059355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:31:40.059355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:31:40.060194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:31:40.060194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:31:40.060194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:40.060194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:40.060194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:40.060194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:31:40.060194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:40.060194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:40.060194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:40.060194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:31:40.060194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:40.060194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:31:40.060194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:31:40.060194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:40.060194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:31:40.060194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:40.060194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:31:40.060194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:31:40.060194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:40.061140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:31:40.061140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:40.061140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:31:40.222808 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:31:40.223376 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:31:40.223599 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:31:40.224624 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:31:40.224879 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:31:40.225040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:40.225734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:31:40.225734 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:40.225734 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:31:40.225734 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:31:40.225734 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:40.225734 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:31:40.225734 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:40.225734 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:31:40.227292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:40.227292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:31:40.227292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:40.227292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:31:40.227292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:31:40.227292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:31:40.227292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:31:40.227292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:40.227292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:40.227292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:40.227292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:31:40.227292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:40.227619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:40.227619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:40.227619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:31:40.227619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:40.227619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:31:40.227619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:31:40.227619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:40.227619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:31:40.227619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:40.227619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:31:40.227619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:31:40.227619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:40.227619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:31:40.227878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:40.227878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:31:40.229263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:40.229263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:31:40.229263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:31:40.229263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:31:40.229263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:31:40.229263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:31:40.229263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:40.229263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:31:40.229263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:31:40.229263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:31:40.229263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:31:40.229263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:31:40.229263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:40.229263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:40.229263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:31:40.229263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:31:40.229774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:31:40.229774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:31:40.229774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:40.229774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:40.229774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:40.229774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:31:40.229774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:40.229774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:40.229774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:40.229774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:31:40.229774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:40.229774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:31:40.229774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:31:40.229774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:40.229774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:31:40.229774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:40.229774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:31:40.229774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:31:40.229774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:40.230330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:31:40.230330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:40.230330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:31:40.230330 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:31:40.230330 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:31:40.230330 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:31:40.230578 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:31:40.230985 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:31:40.231210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:40.231957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:31:40.231957 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:40.231957 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:31:40.231957 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:31:40.231957 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:40.231957 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:31:40.231957 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:40.231957 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:31:40.232819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:40.232819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:31:40.232819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:31:40.232819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:31:40.232819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:40.232819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:31:40.232819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:40.232819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:31:40.232819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:40.232819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:31:40.232819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:40.232819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:31:40.232819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:40.232819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:31:40.232819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:31:40.233462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:40.233462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:31:40.233462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:40.233462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:31:40.233462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:31:40.233462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:40.233462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:31:40.233462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:40.233462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:31:40.233744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:40.233744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:31:40.233744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:40.233744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:31:40.233744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:31:40.233744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:31:40.233744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:31:40.233744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:31:40.233744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:31:40.233744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:31:40.233744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:40.233744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:40.234082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:40.234082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:31:40.234082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:40.234082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:40.234082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:40.234082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:31:40.234082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:40.234082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:31:40.234082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:31:40.234082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:40.234082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:31:40.234082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:40.234082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:31:40.234082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:31:40.234439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:40.234439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:31:40.234439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:40.234439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:31:40.234564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:40.234564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:31:40.234564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:31:40.234564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:31:40.234564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:31:40.234564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:31:40.234564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:40.234564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:31:40.234564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:31:40.234564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:31:40.234564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:31:40.234564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:31:40.234564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:40.234564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:40.234564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:31:40.234564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:31:40.235070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:31:40.235070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:31:40.235070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:31:40.235070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:31:40.235070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:31:40.235070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:40.235070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:40.235070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:40.235070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:31:40.235070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:40.235070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:40.235070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:40.235070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:31:40.235070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:40.235070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:31:40.235070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:31:40.235070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:40.235070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:31:40.235566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:40.235566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:31:40.235566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:31:40.235566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:40.235566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:31:40.235566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:40.235566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:31:40.250807 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:31:40.251175 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:31:40.251367 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:31:40.251599 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:31:40.252899 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:31:40.253275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:40.254015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:31:40.254015 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:40.254015 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:31:40.254015 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:31:40.254015 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:40.254015 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:31:40.254015 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:40.254015 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:31:40.255324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:40.255324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:31:40.255324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:31:40.255324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:31:40.255324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:40.255324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:31:40.255324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:40.255324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:31:40.255324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:40.255324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:31:40.255324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:40.255324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:31:40.255324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:40.255324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:31:40.255324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:31:40.255765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:40.255765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:31:40.255765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:40.255765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:31:40.255765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:31:40.255765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:40.255765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:31:40.255765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:40.255765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:31:40.256753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:40.256753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:31:40.256753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:40.256753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:31:40.256753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:31:40.256753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:31:40.256753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:31:40.256753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:31:40.256753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:31:40.256753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:40.256753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:40.257108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:40.257108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:31:40.257108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:40.257108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:40.257108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:40.257108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:31:40.257108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:40.257108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:31:40.257108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:31:40.257108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:40.257108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:31:40.257108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:40.257108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:31:40.257108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:31:40.257513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:40.257513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:31:40.257513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:40.257513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:31:40.258704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:40.258704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:31:40.258704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:31:40.258704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:31:40.258704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:31:40.258704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:31:40.258704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:40.258704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:31:40.258704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:31:40.258704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:31:40.258704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:31:40.258704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:31:40.258704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:40.258704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:40.258704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:31:40.258704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:31:40.258704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:31:40.259248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:31:40.259248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:31:40.259248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:31:40.259248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:40.259248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:40.259248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:40.259248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:31:40.259248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:40.259248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:40.259248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:40.259248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:31:40.259248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:40.259248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:31:40.259248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:31:40.259248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:40.259248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:31:40.259248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:40.259248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:31:40.259248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:31:40.259826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:40.259826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:31:40.259826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:40.259826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:31:40.259826 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:31:40.259826 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:31:40.259826 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:31:40.259826 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:31:40.260076 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:31:40.260076 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:31:40.260222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:40.260729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:31:40.260729 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:40.260729 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:31:40.260729 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:31:40.260729 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:40.260729 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:31:40.260729 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:40.260729 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:31:40.261805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:40.261805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:31:40.261805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:40.261805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:31:40.261805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:31:40.261805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:31:40.261805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:31:40.261805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:40.261805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:40.261805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:40.261805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:31:40.261805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:40.262145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:40.262145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:40.262145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:31:40.262145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:40.262145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:31:40.262145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:31:40.262145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:40.262145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:31:40.262145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:40.262145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:31:40.262145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:31:40.262145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:40.262145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:31:40.262609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:40.262609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:31:40.262837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:40.262837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:31:40.262837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:31:40.262837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:31:40.262837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:31:40.262837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:31:40.262837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:40.262837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:31:40.262837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:31:40.262837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:31:40.262837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:31:40.262837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:31:40.262837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:40.262837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:40.262837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:31:40.262837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:31:40.263282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:31:40.263282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:31:40.263282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:40.263282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:40.263282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:40.263282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:31:40.263282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:40.263282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:40.263282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:40.263282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:31:40.263282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:40.263282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:31:40.263282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:31:40.263282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:40.263282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:31:40.263282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:40.263282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:31:40.263282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:31:40.263282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:40.263795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:31:40.263795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:40.263795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:31:45.016910 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:31:45.017791 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:31:45.017865 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:31:45.019072 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:31:45.053726 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:31:45.054025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:45.054905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:31:45.054905 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:45.054905 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:31:45.054905 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:31:45.054905 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:45.054905 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:31:45.054905 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:45.054905 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:31:45.055916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:45.055916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:31:45.055916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:45.055916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:31:45.055916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:31:45.055916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:31:45.055916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:31:45.055916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:45.055916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:45.055916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:45.055916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:31:45.055916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:45.056537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:45.056537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:45.056537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:31:45.056537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:45.056537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:31:45.056537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:31:45.056537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:45.056537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:31:45.056537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:45.056537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:31:45.056537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:31:45.056537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:45.056537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:31:45.057120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:45.057120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:31:45.057120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:45.057120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:31:45.057120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:31:45.057120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:31:45.057120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:31:45.057120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:31:45.057120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:45.057120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:31:45.057120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:31:45.057120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:31:45.057120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:31:45.057120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:31:45.057120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:45.057864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:45.057864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:31:45.057864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:31:45.057864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:31:45.057864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:31:45.057864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:45.057864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:45.057864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:45.057864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:31:45.057864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:45.057864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:45.057864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:45.057864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:31:45.057864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:45.057864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:31:45.057864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:31:45.057864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:45.057864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:31:45.057864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:45.058631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:31:45.058631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:31:45.058631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:45.058631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:31:45.058631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:45.058631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:31:45.223788 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:31:45.224083 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:31:45.224263 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:31:45.225340 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:31:45.225724 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:31:45.225983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:45.227654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:31:45.227654 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:45.227654 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:31:45.227654 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:31:45.227654 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:45.227654 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:31:45.227654 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:45.227654 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:31:45.229255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:45.229255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:31:45.229255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:45.229255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:31:45.229255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:31:45.229255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:31:45.229255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:31:45.229255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:45.229255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:45.229255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:45.229255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:31:45.229255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:45.229612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:45.229612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:45.229612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:31:45.229612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:45.229612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:31:45.229612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:31:45.229612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:45.229612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:31:45.229612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:45.229612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:31:45.229612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:31:45.229612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:45.229612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:31:45.229906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:45.229906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:31:45.230911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:45.230911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:31:45.230911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:31:45.230911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:31:45.230911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:31:45.230911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:31:45.230911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:45.230911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:31:45.230911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:31:45.230911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:31:45.230911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:31:45.230911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:31:45.230911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:45.230911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:45.230911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:31:45.230911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:31:45.231321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:31:45.231321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:31:45.231321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:45.231321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:45.231321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:45.231321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:31:45.231321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:45.231321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:45.231321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:45.231321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:31:45.231321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:45.231321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:31:45.231321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:31:45.231321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:45.231321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:31:45.231321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:45.231321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:31:45.231321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:31:45.231321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:45.231765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:31:45.231765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:45.231765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:31:45.231765 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:31:45.231765 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:31:45.231765 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:31:45.231765 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:31:45.232379 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:31:45.232984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:45.233738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:31:45.233738 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:45.233738 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:31:45.233738 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:31:45.233738 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:45.233738 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:31:45.233738 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:45.233738 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:31:45.234493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:45.234493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:31:45.234493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:31:45.234493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:31:45.234493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:45.234493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:31:45.234493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:45.234493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:31:45.234493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:45.234493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:31:45.234493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:45.234493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:31:45.234493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:45.234493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:31:45.234493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:31:45.234992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:45.234992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:31:45.234992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:45.234992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:31:45.234992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:31:45.234992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:45.234992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:31:45.234992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:45.234992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:31:45.235280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:45.235280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:31:45.235280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:45.235280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:31:45.235280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:31:45.235280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:31:45.235280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:31:45.235280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:31:45.235280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:31:45.235280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:31:45.235280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:45.235280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:45.235684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:45.235684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:31:45.235684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:45.235684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:45.235684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:45.235684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:31:45.235684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:45.235684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:31:45.235684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:31:45.235684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:45.235684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:31:45.235684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:45.235684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:31:45.235684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:31:45.236062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:45.236062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:31:45.236062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:45.236062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:31:45.236264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:45.236264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:31:45.236264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:31:45.236264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:31:45.236264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:31:45.236264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:31:45.236264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:45.236264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:31:45.236264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:31:45.236264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:31:45.236264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:31:45.236264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:31:45.236264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:45.236264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:45.236264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:31:45.236264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:31:45.236768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:31:45.236768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:31:45.236768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:31:45.236768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:31:45.236768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:31:45.236768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:45.236768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:45.236768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:45.236768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:31:45.236768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:45.236768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:45.236768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:45.236768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:31:45.236768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:45.236768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:31:45.236768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:31:45.236768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:45.236768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:31:45.237306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:45.237306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:31:45.237306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:31:45.237306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:45.237306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:31:45.237306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:45.237306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:31:45.251324 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:31:45.251607 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:31:45.251715 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:31:45.251807 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:31:45.252326 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:31:45.252611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:45.253191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:31:45.253191 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:45.253191 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:31:45.253191 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:31:45.253191 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:45.253191 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:31:45.253191 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:45.253191 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:31:45.253961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:45.253961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:31:45.253961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:31:45.253961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:31:45.253961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:45.253961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:31:45.253961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:45.253961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:31:45.253961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:45.253961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:31:45.253961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:45.253961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:31:45.253961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:45.253961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:31:45.253961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:31:45.255022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:45.255022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:31:45.255022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:45.255022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:31:45.255022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:31:45.255022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:45.255022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:31:45.255022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:45.255022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:31:45.255022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:45.255022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:31:45.255022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:45.255022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:31:45.255022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:31:45.255612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:31:45.255612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:31:45.255612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:31:45.255612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:31:45.255612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:45.255612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:45.255612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:45.255612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:31:45.255612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:45.255612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:45.255612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:45.255612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:31:45.255612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:45.255876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:31:45.255876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:31:45.255876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:45.255876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:31:45.255876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:45.255876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:31:45.255876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:31:45.255876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:45.255876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:31:45.255876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:45.255876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:31:45.256101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:45.256101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:31:45.256101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:31:45.256101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:31:45.256101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:31:45.256101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:31:45.256101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:45.256101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:31:45.256101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:31:45.256101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:31:45.256101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:31:45.256101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:31:45.256101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:45.256101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:45.256101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:31:45.256101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:31:45.256101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:31:45.256455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:31:45.256455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:31:45.256455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:31:45.256455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:45.256455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:45.256455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:45.256455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:31:45.256455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:45.256455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:45.256455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:45.256455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:31:45.256455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:45.256455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:31:45.256455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:31:45.256455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:45.256455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:31:45.256455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:45.256455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:31:45.256455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:31:45.256793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:45.256793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:31:45.256793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:45.256793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:31:45.259392 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:31:45.259721 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:31:45.259826 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:31:45.259984 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:31:45.260325 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:31:45.260571 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:31:45.260715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:45.261263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:31:45.261263 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:45.261263 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:31:45.261263 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:31:45.261263 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:45.261263 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:31:45.261263 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:45.261263 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:31:45.262565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:45.262565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:31:45.262565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:45.262565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:31:45.262565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:31:45.262565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:31:45.262565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:31:45.262565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:45.262565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:45.262565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:45.262565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:31:45.262565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:45.262846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:45.262846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:45.262846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:31:45.262846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:45.262846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:31:45.262846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:31:45.262846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:45.262846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:31:45.262846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:45.262846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:31:45.262846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:31:45.262846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:45.262846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:31:45.263094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:45.263094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:31:45.263660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:45.263660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:31:45.263660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:31:45.263660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:31:45.263660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:31:45.263660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:31:45.263660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:45.263660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:31:45.263660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:31:45.263660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:31:45.263660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:31:45.263660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:31:45.263660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:45.263660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:45.263660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:31:45.263660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:31:45.264020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:31:45.264020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:31:45.264020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:45.264020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:45.264020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:45.264020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:31:45.264020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:45.264020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:45.264020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:45.264020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:31:45.264020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:45.264020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:31:45.264020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:31:45.264020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:45.264020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:31:45.264020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:45.264020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:31:45.264020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:31:45.264020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:45.264430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:31:45.264430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:45.264430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:31:50.017652 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:31:50.018717 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:31:50.018717 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:31:50.019728 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:31:50.050496 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:31:50.050924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:50.051787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:31:50.051787 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:50.051787 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:31:50.051787 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:31:50.051787 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:50.051787 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:31:50.051787 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:50.051787 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:31:50.052536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:50.052536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:31:50.052536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:50.052536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:31:50.052536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:31:50.052536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:31:50.052536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:31:50.052536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:50.052536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:50.052536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:50.052536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:31:50.052536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:50.053306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:50.053306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:50.053306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:31:50.053306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:50.053306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:31:50.053306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:31:50.053306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:50.053306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:31:50.053306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:50.053306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:31:50.053306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:31:50.053306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:50.053306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:31:50.054041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:50.054041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:31:50.054041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:50.054041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:31:50.054041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:31:50.054041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:31:50.054041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:31:50.054041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:31:50.054041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:50.054041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:31:50.054041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:31:50.054041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:31:50.054041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:31:50.054041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:31:50.054041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:50.054745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:50.054745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:31:50.054745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:31:50.054745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:31:50.054745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:31:50.054745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:50.054745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:50.054745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:50.054745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:31:50.054745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:50.054745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:50.054745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:50.054745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:31:50.054745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:50.054745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:31:50.054745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:31:50.054745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:50.054745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:31:50.054745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:50.055229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:31:50.055229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:31:50.055229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:50.055229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:31:50.055229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:50.055229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:31:50.224207 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:31:50.224645 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:31:50.224840 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:31:50.225486 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:31:50.225675 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:31:50.225790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:50.226294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:31:50.226294 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:50.226294 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:31:50.226294 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:31:50.226294 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:50.226294 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:31:50.226294 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:50.226294 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:31:50.227039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:50.227039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:31:50.227039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:50.227039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:31:50.227039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:31:50.227039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:31:50.227039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:31:50.227039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:50.227039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:50.227039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:50.227039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:31:50.227039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:50.227425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:50.227425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:50.227425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:31:50.227425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:50.227425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:31:50.227425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:31:50.227425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:50.227425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:31:50.227425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:50.227425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:31:50.227425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:31:50.227425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:50.227425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:31:50.227802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:50.227802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:31:50.227802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:50.227802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:31:50.227802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:31:50.227802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:31:50.227802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:31:50.227802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:31:50.227802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:50.227802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:31:50.227802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:31:50.227802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:31:50.227802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:31:50.227802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:31:50.227802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:50.228252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:50.228252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:31:50.228252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:31:50.228252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:31:50.228252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:31:50.228252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:50.228252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:50.228252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:50.228252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:31:50.228252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:50.228252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:50.228252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:50.228252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:31:50.228252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:50.228252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:31:50.228252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:31:50.228252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:50.228252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:31:50.228252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:50.228766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:31:50.228766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:31:50.228766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:50.228766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:31:50.228766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:50.228766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:31:50.231591 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:31:50.231821 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:31:50.231821 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:31:50.232066 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:31:50.232181 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:31:50.232349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:50.232715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:31:50.232715 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:50.232715 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:31:50.232715 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:31:50.232715 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:50.232715 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:31:50.232715 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:50.232715 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:31:50.233284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:50.233284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:31:50.233284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:31:50.233284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:31:50.233284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:50.233284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:31:50.233284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:50.233284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:31:50.233284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:50.233284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:31:50.233284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:50.233284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:31:50.233284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:50.233284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:31:50.233284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:31:50.234311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:50.234311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:31:50.234311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:50.234311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:31:50.234311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:31:50.234311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:50.234311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:31:50.234311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:50.234311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:31:50.234311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:50.234311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:31:50.234311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:50.234311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:31:50.234311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:31:50.235349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:31:50.235349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:31:50.235349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:31:50.235349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:31:50.235349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:31:50.235349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:50.235349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:50.235349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:50.235349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:31:50.235349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:50.235349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:50.235349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:50.235349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:31:50.235908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:50.235908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:31:50.235908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:31:50.235908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:50.235908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:31:50.235908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:50.235908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:31:50.235908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:31:50.235908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:50.235908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:31:50.235908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:50.235908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:31:50.236389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:50.236389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:31:50.236389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:31:50.236389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:31:50.236389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:31:50.236389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:31:50.236389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:50.236389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:31:50.236389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:31:50.236389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:31:50.236389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:31:50.236389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:31:50.236389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:50.236389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:50.236389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:31:50.236389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:31:50.237082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:31:50.237082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:31:50.237082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:31:50.237082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:31:50.237082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:31:50.237082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:50.237082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:50.237082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:50.237082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:31:50.237082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:50.237082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:50.237082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:50.237082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:31:50.237082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:50.237082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:31:50.237082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:31:50.237082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:50.237082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:31:50.237857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:50.237857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:31:50.237857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:31:50.237857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:50.237857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:31:50.237857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:50.237857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:31:50.251657 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:31:50.251875 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:31:50.251941 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:31:50.252037 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:31:50.252296 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:31:50.252491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:50.252855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:31:50.252855 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:50.252855 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:31:50.252855 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:31:50.252855 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:50.252855 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:31:50.252855 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:50.252855 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:31:50.253365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:50.253365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:31:50.253365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:31:50.253365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:31:50.253365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:50.253365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:31:50.253365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:50.253365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:31:50.253365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:50.253365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:31:50.253365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:50.253365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:31:50.253365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:50.253365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:31:50.253365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:31:50.254125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:50.254125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:31:50.254125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:50.254125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:31:50.254125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:31:50.254125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:50.254125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:31:50.254125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:50.254125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:31:50.254125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:50.254125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:31:50.254125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:50.254125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:31:50.254125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:31:50.255064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:31:50.255064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:31:50.255064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:31:50.255064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:31:50.255064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:50.255064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:50.255064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:50.255064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:31:50.255064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:50.255064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:50.255064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:50.255064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:31:50.255064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:50.255744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:31:50.255744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:31:50.255744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:50.255744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:31:50.255744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:50.255744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:31:50.255744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:31:50.255744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:50.255744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:31:50.255744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:50.255744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:31:50.255744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:50.256268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:31:50.256268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:31:50.256268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:31:50.256268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:31:50.256268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:31:50.256268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:50.256268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:31:50.256268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:31:50.256268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:31:50.256268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:31:50.256268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:31:50.256268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:50.256268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:50.256268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:31:50.256268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:31:50.256268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:31:50.256268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:31:50.256268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:31:50.256268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:31:50.257245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:50.257245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:50.257245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:50.257245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:31:50.257245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:50.257245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:50.257245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:50.257245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:31:50.257245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:50.257245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:31:50.257245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:31:50.257245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:50.257245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:31:50.257245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:50.257245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:31:50.257245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:31:50.257245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:50.257245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:31:50.257245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:50.257245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:31:50.259830 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:31:50.260111 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:31:50.260249 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:31:50.260435 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:31:50.260766 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:31:50.260935 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:31:50.261038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:50.261441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:31:50.261441 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:50.261441 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:31:50.261441 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:31:50.261441 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:50.261441 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:31:50.261441 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:50.261441 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:31:50.262038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:50.262038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:31:50.262038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:50.262038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:31:50.262038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:31:50.262038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:31:50.262038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:31:50.262038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:50.262038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:50.262038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:50.262038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:31:50.262038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:50.262636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:50.262636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:50.262636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:31:50.262636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:50.262636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:31:50.262636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:31:50.262636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:50.262636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:31:50.262636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:50.262636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:31:50.262636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:31:50.262636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:50.262636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:31:50.262980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:50.262980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:31:50.262980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:50.262980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:31:50.262980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:31:50.262980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:31:50.262980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:31:50.262980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:31:50.262980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:50.262980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:31:50.262980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:31:50.262980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:31:50.262980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:31:50.262980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:31:50.262980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:50.263433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:50.263433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:31:50.263433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:31:50.263433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:31:50.263433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:31:50.263433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:50.263433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:50.263433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:50.263433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:31:50.263433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:50.263433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:50.263433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:50.263433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:31:50.263433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:50.263433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:31:50.263433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:31:50.263433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:50.263433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:31:50.263433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:50.263948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:31:50.263948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:31:50.263948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:50.263948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:31:50.263948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:50.263948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:31:55.018372 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:31:55.019264 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:31:55.019264 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:31:55.019681 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:31:55.053432 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:31:55.053811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:55.054568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:31:55.054568 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:55.054568 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:31:55.054568 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:31:55.054568 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:55.054568 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:31:55.054568 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:55.054568 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:31:55.055577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:55.055577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:31:55.055577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:55.055577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:31:55.055577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:31:55.055577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:31:55.055577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:31:55.055577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:55.055577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:55.055577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:55.055577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:31:55.055577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:55.056012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:55.056012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:55.056012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:31:55.056012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:55.056012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:31:55.056012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:31:55.056012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:55.056012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:31:55.056012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:55.056012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:31:55.056012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:31:55.056012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:55.056012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:31:55.056599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:55.056599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:31:55.056599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:55.056599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:31:55.056599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:31:55.056599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:31:55.056599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:31:55.056599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:31:55.056599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:55.056599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:31:55.056599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:31:55.056599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:31:55.056599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:31:55.056599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:31:55.056599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:55.057065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:55.057065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:31:55.057065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:31:55.057065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:31:55.057065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:31:55.057065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:55.057065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:55.057065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:55.057065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:31:55.057065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:55.057065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:55.057065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:55.057065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:31:55.057065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:55.057065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:31:55.057065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:31:55.057065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:55.057065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:31:55.057065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:55.057692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:31:55.057692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:31:55.057692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:55.057692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:31:55.057692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:55.057692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:31:55.225067 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:31:55.225366 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:31:55.225674 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:31:55.226836 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:31:55.227425 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:31:55.227681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:55.228855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:31:55.228855 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:55.228855 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:31:55.228855 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:31:55.228855 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:55.228855 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:31:55.228855 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:55.228855 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:31:55.230632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:55.230632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:31:55.230632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:55.230632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:31:55.230632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:31:55.230632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:31:55.230632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:31:55.230632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:55.230632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:55.230632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:55.230632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:31:55.230632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:55.231021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:55.231021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:55.231021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:31:55.231021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:55.231021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:31:55.231021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:31:55.231021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:55.231021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:31:55.231021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:55.231021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:31:55.231021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:31:55.231021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:55.231021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:31:55.231351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:55.231351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:31:55.232585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:55.232585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:31:55.232585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:31:55.232585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:31:55.232585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:31:55.232585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:31:55.232585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:55.232585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:31:55.232585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:31:55.232585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:31:55.232585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:31:55.232585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:31:55.232585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:55.232585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:55.232585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:31:55.232585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:31:55.232946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:31:55.232946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:31:55.232946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:55.232946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:55.232946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:55.232946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:31:55.232946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:55.232946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:55.232946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:55.232946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:31:55.232946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:55.232946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:31:55.232946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:31:55.232946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:55.232946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:31:55.232946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:55.232946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:31:55.232946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:31:55.232946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:55.233295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:31:55.233295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:55.233295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:31:55.233295 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:31:55.233295 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:31:55.234178 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:31:55.234178 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:31:55.234178 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:31:55.234326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:55.234852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:31:55.234852 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:55.234852 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:31:55.234852 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:31:55.234852 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:55.234852 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:31:55.234852 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:55.234852 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:31:55.235715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:55.235715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:31:55.235715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:31:55.235715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:31:55.235715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:55.235715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:31:55.235715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:55.235715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:31:55.235715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:55.235715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:31:55.235715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:55.235715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:31:55.235715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:55.235715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:31:55.235715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:31:55.236072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:55.236072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:31:55.236072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:55.236072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:31:55.236072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:31:55.236072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:55.236072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:31:55.236072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:55.236072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:31:55.236810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:55.236810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:31:55.236810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:55.236810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:31:55.236810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:31:55.236810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:31:55.236810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:31:55.236810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:31:55.236810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:31:55.236810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:31:55.236810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:55.236810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:55.237081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:55.237081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:31:55.237081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:55.237081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:55.237081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:55.237081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:31:55.237081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:55.237081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:31:55.237081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:31:55.237081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:55.237081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:31:55.237081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:55.237081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:31:55.237081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:31:55.237338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:55.237338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:31:55.237338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:55.237338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:31:55.238062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:55.238062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:31:55.238062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:31:55.238062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:31:55.238062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:31:55.238062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:31:55.238062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:55.238062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:31:55.238062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:31:55.238062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:31:55.238062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:31:55.238062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:31:55.238062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:55.238062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:55.238062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:31:55.238062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:31:55.238666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:31:55.238666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:31:55.238666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:31:55.238666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:31:55.238666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:31:55.238666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:55.238666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:55.238666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:55.238666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:31:55.238666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:55.238666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:55.238666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:55.238666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:31:55.238666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:55.238666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:31:55.238666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:31:55.238666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:55.238666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:31:55.239421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:55.239421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:31:55.239421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:31:55.239421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:55.239421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:31:55.239421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:55.239421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:31:55.252065 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:31:55.252537 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:31:55.252850 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:31:55.253136 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:31:55.253679 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:31:55.254012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:55.254859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:31:55.254859 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:55.254859 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:31:55.254859 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:31:55.254859 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:55.254859 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:31:55.254859 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:55.254859 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:31:55.256447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:55.256447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:31:55.256447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:31:55.256447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:31:55.256447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:55.256447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:31:55.256447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:55.256447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:31:55.256447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:55.256447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:31:55.256447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:55.256447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:31:55.256447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:55.256447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:31:55.256447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:31:55.257449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:55.257449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:31:55.257449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:55.257449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:31:55.257449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:31:55.257449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:55.257449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:31:55.257449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:55.257449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:31:55.258550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:55.258550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:31:55.258550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:55.258550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:31:55.258550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:31:55.258550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:31:55.258550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:31:55.258550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:31:55.258550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:31:55.258550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:55.258550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:55.259341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:55.259341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:31:55.259341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:55.259341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:55.259341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:55.259341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:31:55.259341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:55.259341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:31:55.259341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:31:55.259341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:55.259341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:31:55.259341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:55.259341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:31:55.259341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:31:55.260284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:55.260284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:31:55.260284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:55.260284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:31:55.261037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:55.261037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:31:55.261037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:31:55.261037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:31:55.261037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:31:55.261037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:31:55.261037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:55.261037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:31:55.261037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:31:55.261037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:31:55.261037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:31:55.261037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:31:55.261037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:55.261037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:55.261037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:31:55.261037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:31:55.261037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:31:55.261991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:31:55.261991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:31:55.261991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:31:55.261991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:55.261991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:55.261991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:55.261991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:31:55.261991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:55.261991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:55.261991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:55.261991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:31:55.261991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:55.261991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:31:55.261991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:31:55.261991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:55.261991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:31:55.261991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:55.261991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:31:55.261991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:31:55.262586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:55.262586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:31:55.262586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:55.262586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:31:55.262586 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:31:55.262586 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:31:55.262586 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:31:55.262586 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:31:55.262809 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:31:55.262839 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:31:55.263060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:55.263714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:31:55.263714 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:55.263714 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:31:55.263714 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:31:55.263714 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:55.263714 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:31:55.263714 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:55.263714 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:31:55.264394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:55.264394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:31:55.264394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:55.264394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:31:55.264394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:31:55.264394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:31:55.264394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:31:55.264394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:55.264394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:55.264394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:55.264394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:31:55.264394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:55.264791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:31:55.264791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:55.264791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:31:55.264791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:55.264791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:31:55.264791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:31:55.264791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:55.264791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:31:55.264791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:55.264791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:31:55.264791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:31:55.264791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:55.264791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:31:55.265254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:55.265254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:31:55.265418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:55.265418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:31:55.265418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:31:55.265418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:31:55.265418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:31:55.265418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:31:55.265418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:55.265418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:31:55.265418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:31:55.265418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:31:55.265418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:31:55.265418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:31:55.265418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:31:55.265418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:31:55.265418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:31:55.265418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:31:55.265769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:31:55.265769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:31:55.265769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:55.265769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:55.265769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:31:55.265769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:31:55.265769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:31:55.265769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:31:55.265769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:31:55.265769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:31:55.265769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:31:55.265769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:31:55.265769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:31:55.265769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:31:55.265769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:31:55.265769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:31:55.265769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:31:55.265769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:31:55.265769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:31:55.266128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:31:55.266128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:31:55.266128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:32:00.019077 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:32:00.021004 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:32:00.021004 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:32:00.021539 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:32:00.060143 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:32:00.060521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:00.061353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:32:00.061353 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:00.061353 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:32:00.061353 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:32:00.061353 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:00.061353 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:32:00.061353 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:00.061353 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:32:00.062238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:00.062238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:32:00.062238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:00.062238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:32:00.062238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:32:00.062238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:32:00.062238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:32:00.062238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:00.062238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:00.062238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:00.062238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:32:00.062238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:00.062741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:00.062741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:00.062741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:32:00.062741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:00.062741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:32:00.062741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:32:00.062741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:00.062741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:32:00.062741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:00.062741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:32:00.062741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:32:00.062741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:00.062741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:32:00.063242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:00.063242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:32:00.063242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:00.063242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:32:00.063242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:32:00.063242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:32:00.063242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:32:00.063242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:32:00.063242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:00.063242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:32:00.063242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:32:00.063242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:32:00.063242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:32:00.063242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:32:00.063242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:00.063899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:00.063899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:32:00.063899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:32:00.063899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:32:00.063899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:32:00.063899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:00.063899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:00.063899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:00.063899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:32:00.063899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:00.063899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:00.063899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:00.063899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:32:00.063899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:00.063899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:32:00.063899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:32:00.063899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:00.063899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:32:00.063899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:00.064779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:32:00.064779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:32:00.064779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:00.064779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:32:00.064779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:00.064779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:32:00.225711 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:32:00.226199 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:32:00.226392 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:32:00.227602 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:32:00.227949 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:32:00.228196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:00.229049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:32:00.229049 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:00.229049 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:32:00.229049 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:32:00.229049 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:00.229049 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:32:00.229049 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:00.229049 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:32:00.230664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:00.230664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:32:00.230664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:00.230664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:32:00.230664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:32:00.230664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:32:00.230664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:32:00.230664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:00.230664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:00.230664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:00.230664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:32:00.230664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:00.231140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:00.231140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:00.231140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:32:00.231140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:00.231140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:32:00.231140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:32:00.231140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:00.231140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:32:00.231140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:00.231140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:32:00.231140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:32:00.231140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:00.231140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:32:00.231704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:00.231704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:32:00.232420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:00.232420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:32:00.232420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:32:00.232420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:32:00.232420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:32:00.232420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:32:00.232420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:00.232420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:32:00.232420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:32:00.232420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:32:00.232420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:32:00.232420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:32:00.232420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:00.232420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:00.232420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:32:00.232420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:32:00.233102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:32:00.233102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:32:00.233102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:00.233102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:00.233102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:00.233102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:32:00.233102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:00.233102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:00.233102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:00.233102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:32:00.233102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:00.233102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:32:00.233102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:32:00.233102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:00.233102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:32:00.233102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:00.233102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:32:00.233102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:32:00.233102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:00.233810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:32:00.233810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:00.233810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:32:00.233810 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:32:00.233978 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:32:00.234036 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:32:00.234192 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:32:00.234647 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:32:00.234929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:00.235641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:32:00.235641 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:00.235641 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:32:00.235641 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:32:00.235641 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:00.235641 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:32:00.235641 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:00.235641 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:32:00.236360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:00.236360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:32:00.236360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:32:00.236360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:32:00.236360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:00.236360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:32:00.236360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:00.236360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:32:00.236360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:00.236360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:32:00.236360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:00.236360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:32:00.236360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:00.236360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:32:00.236360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:32:00.237124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:00.237124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:32:00.237124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:00.237124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:32:00.237124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:32:00.237124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:00.237124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:32:00.237124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:00.237124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:32:00.237124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:00.237124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:32:00.237124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:00.237124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:32:00.237124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:32:00.237849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:32:00.237849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:32:00.237849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:32:00.237849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:32:00.237849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:32:00.237849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:00.237849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:00.237849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:00.237849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:32:00.237849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:00.237849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:00.237849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:00.237849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:32:00.238253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:00.238253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:32:00.238253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:32:00.238253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:00.238253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:32:00.238253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:00.238253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:32:00.238253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:32:00.238253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:00.238253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:32:00.238253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:00.238253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:32:00.238638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:00.238638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:32:00.238638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:32:00.238638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:32:00.238638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:32:00.238638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:32:00.238638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:00.238638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:32:00.238638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:32:00.238638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:32:00.238638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:32:00.238638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:32:00.238638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:00.238638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:00.238638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:32:00.238638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:32:00.239332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:32:00.239332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:32:00.239332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:32:00.239332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:32:00.239332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:32:00.239332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:00.239332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:00.239332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:00.239332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:32:00.239332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:00.239332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:00.239332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:00.239332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:32:00.239332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:00.239332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:32:00.239332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:32:00.239332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:00.239332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:32:00.240148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:00.240148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:32:00.240148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:32:00.240148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:00.240148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:32:00.240148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:00.240148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:32:00.252554 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:32:00.252899 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:32:00.252993 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:32:00.253142 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:32:00.253707 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:32:00.253974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:00.254703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:32:00.254703 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:00.254703 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:32:00.254703 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:32:00.254703 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:00.254703 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:32:00.254703 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:00.254703 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:32:00.255812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:00.255812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:32:00.255812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:32:00.255812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:32:00.255812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:00.255812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:32:00.255812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:00.255812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:32:00.255812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:00.255812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:32:00.255812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:00.255812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:32:00.255812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:00.255812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:32:00.255812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:32:00.256626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:00.256626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:32:00.256626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:00.256626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:32:00.256626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:32:00.256626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:00.256626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:32:00.256626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:00.256626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:32:00.257125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:00.257125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:32:00.257125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:00.257125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:32:00.257125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:32:00.257125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:32:00.257125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:32:00.257125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:32:00.257125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:32:00.257125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:00.257125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:00.257760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:00.257760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:32:00.257760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:00.257760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:00.257760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:00.257760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:32:00.257760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:00.257760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:32:00.257760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:32:00.257760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:00.257760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:32:00.257760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:00.257760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:32:00.257760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:32:00.258461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:00.258461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:32:00.258461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:00.258461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:32:00.259082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:00.259082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:32:00.259082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:32:00.259082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:32:00.259082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:32:00.259082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:32:00.259082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:00.259082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:32:00.259082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:32:00.259082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:32:00.259082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:32:00.259082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:32:00.259082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:00.259082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:00.259082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:32:00.259082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:32:00.259082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:32:00.259975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:32:00.259975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:32:00.259975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:32:00.259975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:00.259975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:00.259975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:00.259975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:32:00.259975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:00.259975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:00.259975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:00.259975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:32:00.259975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:00.259975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:32:00.259975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:32:00.259975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:00.259975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:32:00.259975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:00.259975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:32:00.259975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:32:00.260530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:00.260530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:32:00.260530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:00.260530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:32:00.261764 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:32:00.262108 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:32:00.262252 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:32:00.262476 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:32:00.262946 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:32:00.263345 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:32:00.263604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:00.264319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:32:00.264319 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:00.264319 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:32:00.264319 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:32:00.264319 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:00.264319 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:32:00.264319 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:00.264319 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:32:00.265488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:00.265488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:32:00.265488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:00.265488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:32:00.265488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:32:00.265488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:32:00.265488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:32:00.265488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:00.265488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:00.265488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:00.265488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:32:00.265488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:00.265894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:00.265894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:00.265894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:32:00.265894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:00.265894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:32:00.265894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:32:00.265894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:00.265894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:32:00.265894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:00.265894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:32:00.265894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:32:00.265894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:00.265894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:32:00.266283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:00.266283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:32:00.266419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:00.266419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:32:00.266419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:32:00.266419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:32:00.266419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:32:00.266419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:32:00.266419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:00.266419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:32:00.266419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:32:00.266419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:32:00.266419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:32:00.266419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:32:00.266419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:00.266419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:00.266419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:32:00.266419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:32:00.267030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:32:00.267030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:32:00.267030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:00.267030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:00.267030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:00.267030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:32:00.267030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:00.267030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:00.267030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:00.267030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:32:00.267030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:00.267030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:32:00.267030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:32:00.267030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:00.267030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:32:00.267030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:00.267030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:32:00.267030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:32:00.267030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:00.267918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:32:00.267918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:00.267918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:32:05.020854 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:32:05.021999 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:32:05.021999 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:32:05.023511 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:32:05.058600 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:32:05.059026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:05.060009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:32:05.060009 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:05.060009 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:32:05.060009 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:32:05.060009 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:05.060009 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:32:05.060009 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:05.060009 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:32:05.061111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:05.061111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:32:05.061111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:05.061111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:32:05.061111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:32:05.061111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:32:05.061111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:32:05.061111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:05.061111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:05.061111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:05.061111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:32:05.061111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:05.061656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:05.061656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:05.061656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:32:05.061656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:05.061656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:32:05.061656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:32:05.061656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:05.061656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:32:05.061656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:05.061656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:32:05.061656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:32:05.061656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:05.061656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:32:05.062263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:05.062263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:32:05.062263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:05.062263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:32:05.062263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:32:05.062263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:32:05.062263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:32:05.062263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:32:05.062263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:05.062263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:32:05.062263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:32:05.062263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:32:05.062263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:32:05.062263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:32:05.062263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:05.062909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:05.062909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:32:05.062909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:32:05.062909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:32:05.062909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:32:05.062909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:05.062909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:05.062909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:05.062909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:32:05.062909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:05.062909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:05.062909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:05.062909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:32:05.062909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:05.062909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:32:05.062909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:32:05.062909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:05.062909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:32:05.062909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:05.063874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:32:05.063874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:32:05.063874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:05.063874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:32:05.063874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:05.063874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:32:05.226347 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:32:05.226822 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:32:05.227002 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:32:05.227993 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:32:05.228330 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:32:05.228644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:05.229528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:32:05.229528 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:05.229528 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:32:05.229528 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:32:05.229528 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:05.229528 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:32:05.229528 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:05.229528 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:32:05.231031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:05.231031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:32:05.231031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:05.231031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:32:05.231031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:32:05.231031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:32:05.231031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:32:05.231031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:05.231031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:05.231031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:05.231031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:32:05.231031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:05.231603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:05.231603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:05.231603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:32:05.231603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:05.231603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:32:05.231603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:32:05.231603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:05.231603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:32:05.231603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:05.231603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:32:05.231603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:32:05.231603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:05.231603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:32:05.232161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:05.232161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:32:05.232651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:05.232651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:32:05.232651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:32:05.232651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:32:05.232651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:32:05.232651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:32:05.232651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:05.232651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:32:05.232651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:32:05.232651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:32:05.232651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:32:05.232651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:32:05.232651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:05.232651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:05.232651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:32:05.232651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:32:05.233255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:32:05.233255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:32:05.233255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:05.233255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:05.233255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:05.233255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:32:05.233255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:05.233255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:05.233255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:05.233255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:32:05.233255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:05.233255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:32:05.233255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:32:05.233255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:05.233255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:32:05.233255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:05.233255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:32:05.233255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:32:05.233255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:05.233811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:32:05.233811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:05.233811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:32:05.234131 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:32:05.234492 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:32:05.234660 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:32:05.234835 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:32:05.235260 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:32:05.235508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:05.236124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:32:05.236124 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:05.236124 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:32:05.236124 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:32:05.236124 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:05.236124 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:32:05.236124 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:05.236124 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:32:05.237248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:05.237248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:32:05.237248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:32:05.237248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:32:05.237248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:05.237248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:32:05.237248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:05.237248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:32:05.237248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:05.237248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:32:05.237248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:05.237248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:32:05.237248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:05.237248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:32:05.237248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:32:05.237763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:05.237763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:32:05.237763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:05.237763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:32:05.237763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:32:05.237763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:05.237763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:32:05.237763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:05.237763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:32:05.238430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:05.238430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:32:05.238430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:05.238430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:32:05.238430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:32:05.238430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:32:05.238430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:32:05.238430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:32:05.238430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:32:05.238430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:32:05.238430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:05.238430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:05.238825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:05.238825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:32:05.238825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:05.238825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:05.238825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:05.238825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:32:05.238825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:05.238825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:32:05.238825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:32:05.238825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:05.238825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:32:05.238825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:05.238825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:32:05.238825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:32:05.239214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:05.239214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:32:05.239214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:05.239214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:32:05.239483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:05.239483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:32:05.239483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:32:05.239483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:32:05.239483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:32:05.239483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:32:05.239483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:05.239483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:32:05.239483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:32:05.239483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:32:05.239483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:32:05.239483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:32:05.239483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:05.239483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:05.239483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:32:05.239483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:32:05.239926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:32:05.239926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:32:05.239926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:32:05.239926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:32:05.239926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:32:05.239926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:05.239926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:05.239926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:05.239926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:32:05.239926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:05.239926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:05.239926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:05.239926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:32:05.239926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:05.239926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:32:05.239926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:32:05.239926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:05.239926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:32:05.240444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:05.240444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:32:05.240444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:32:05.240444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:05.240444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:32:05.240444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:05.240444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:32:05.253152 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:32:05.253563 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:32:05.253739 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:32:05.254317 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:32:05.254923 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:32:05.255313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:05.256272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:32:05.256272 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:05.256272 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:32:05.256272 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:32:05.256272 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:05.256272 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:32:05.256272 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:05.256272 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:32:05.257719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:05.257719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:32:05.257719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:32:05.257719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:32:05.257719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:05.257719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:32:05.257719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:05.257719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:32:05.257719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:05.257719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:32:05.257719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:05.257719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:32:05.257719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:05.257719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:32:05.257719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:32:05.258748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:05.258748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:32:05.258748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:05.258748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:32:05.258748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:32:05.258748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:05.258748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:32:05.258748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:05.258748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:32:05.259778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:05.259778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:32:05.259778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:05.259778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:32:05.259778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:32:05.259778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:32:05.259778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:32:05.259778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:32:05.259778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:32:05.259778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:05.259778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:05.260573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:05.260573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:32:05.260573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:05.260573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:05.260573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:05.260573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:32:05.260573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:05.260573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:32:05.260573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:32:05.260573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:05.260573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:32:05.260573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:05.260573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:32:05.260573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:32:05.261461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:05.261461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:32:05.261461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:05.261461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:32:05.262141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:05.262141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:32:05.262141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:32:05.262141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:32:05.262141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:32:05.262141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:32:05.262141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:05.262141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:32:05.262141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:32:05.262141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:32:05.262141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:32:05.262141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:32:05.262141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:05.262141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:05.262141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:32:05.262141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:32:05.262141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:32:05.262549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:32:05.262549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:32:05.262549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:32:05.262549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:05.262549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:05.262549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:05.262549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:32:05.262549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:05.262549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:05.262549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:05.262549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:32:05.262549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:05.262549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:32:05.262549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:32:05.262549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:05.262549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:32:05.262549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:05.262549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:32:05.262549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:32:05.263034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:05.263034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:32:05.263034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:05.263034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:32:05.263034 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:32:05.263034 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:32:05.263034 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:32:05.263254 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:32:05.263620 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:32:05.263896 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:32:05.264053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:05.264660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:32:05.264660 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:05.264660 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:32:05.264660 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:32:05.264660 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:05.264660 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:32:05.264660 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:05.264660 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:32:05.265429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:05.265429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:32:05.265429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:05.265429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:32:05.265429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:32:05.265429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:32:05.265429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:32:05.265429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:05.265429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:05.265429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:05.265429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:32:05.265429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:05.265834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:05.265834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:05.265834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:32:05.265834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:05.265834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:32:05.265834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:32:05.265834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:05.265834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:32:05.265834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:05.265834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:32:05.265834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:32:05.265834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:05.265834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:32:05.266287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:05.266287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:32:05.266490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:05.266490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:32:05.266490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:32:05.266490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:32:05.266490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:32:05.266490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:32:05.266490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:05.266490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:32:05.266490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:32:05.266490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:32:05.266490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:32:05.266490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:32:05.266490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:05.266490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:05.266490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:32:05.266490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:32:05.266825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:32:05.266825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:32:05.266825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:05.266825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:05.266825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:05.266825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:32:05.266825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:05.266825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:05.266825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:05.266825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:32:05.266825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:05.266825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:32:05.266825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:32:05.266825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:05.266825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:32:05.266825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:05.266825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:32:05.266825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:32:05.266825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:05.267177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:32:05.267177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:05.267177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:32:10.021440 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:32:10.021949 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:32:10.022043 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:32:10.022798 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:32:10.062059 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:32:10.062413 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:10.063262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:32:10.063262 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:10.063262 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:32:10.063262 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:32:10.063262 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:10.063262 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:32:10.063262 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:10.063262 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:32:10.064260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:10.064260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:32:10.064260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:10.064260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:32:10.064260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:32:10.064260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:32:10.064260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:32:10.064260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:10.064260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:10.064260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:10.064260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:32:10.064260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:10.064713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:10.064713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:10.064713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:32:10.064713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:10.064713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:32:10.064713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:32:10.064713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:10.064713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:32:10.064713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:10.064713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:32:10.064713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:32:10.064713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:10.064713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:32:10.065092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:10.065092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:32:10.065459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:10.065459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:32:10.065459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:32:10.065459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:32:10.065459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:32:10.065459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:32:10.065459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:10.065459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:32:10.065459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:32:10.065459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:32:10.065459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:32:10.065459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:32:10.065459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:10.065459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:10.065459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:32:10.065459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:32:10.066579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:32:10.066579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:32:10.066579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:10.066579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:10.066579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:10.066579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:32:10.066579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:10.066579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:10.066579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:10.066579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:32:10.066579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:10.066579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:32:10.066579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:32:10.066579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:10.066579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:32:10.066579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:10.066579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:32:10.066579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:32:10.066579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:10.067939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:32:10.067939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:10.067939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:32:10.226853 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:32:10.227324 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:32:10.227520 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:32:10.228363 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:32:10.228838 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:32:10.229046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:10.229777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:32:10.229777 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:10.229777 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:32:10.229777 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:32:10.229777 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:10.229777 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:32:10.229777 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:10.229777 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:32:10.230779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:10.230779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:32:10.230779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:10.230779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:32:10.230779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:32:10.230779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:32:10.230779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:32:10.230779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:10.230779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:10.230779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:10.230779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:32:10.230779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:10.231317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:10.231317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:10.231317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:32:10.231317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:10.231317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:32:10.231317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:32:10.231317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:10.231317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:32:10.231317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:10.231317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:32:10.231317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:32:10.231317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:10.231317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:32:10.232716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:10.232716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:32:10.232716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:10.232716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:32:10.232716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:32:10.232716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:32:10.232716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:32:10.232716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:32:10.232716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:10.232716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:32:10.232716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:32:10.232716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:32:10.232716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:32:10.232716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:32:10.232716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:10.233929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:10.233929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:32:10.233929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:32:10.233929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:32:10.233929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:32:10.233929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:10.233929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:10.233929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:10.233929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:32:10.233929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:10.233929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:10.233929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:10.233929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:32:10.233929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:10.233929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:32:10.233929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:32:10.233929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:10.233929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:32:10.233929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:10.234762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:32:10.234762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:32:10.234762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:10.234762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:32:10.234762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:10.234762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:32:10.234762 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:32:10.235095 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:32:10.235095 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:32:10.235095 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:32:10.235572 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:32:10.235897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:10.236615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:32:10.236615 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:10.236615 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:32:10.236615 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:32:10.236615 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:10.236615 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:32:10.236615 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:10.236615 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:32:10.237322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:10.237322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:32:10.237322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:32:10.237322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:32:10.237322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:10.237322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:32:10.237322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:10.237322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:32:10.237322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:10.237322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:32:10.237322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:10.237322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:32:10.237322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:10.237322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:32:10.237322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:32:10.237796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:10.237796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:32:10.237796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:10.237796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:32:10.237796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:32:10.237796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:10.237796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:32:10.237796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:10.237796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:32:10.238153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:10.238153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:32:10.238153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:10.238153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:32:10.238153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:32:10.238153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:32:10.238153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:32:10.238153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:32:10.238153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:32:10.238153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:32:10.238153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:10.238153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:10.238525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:10.238525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:32:10.238525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:10.238525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:10.238525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:10.238525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:32:10.238525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:10.238525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:32:10.238525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:32:10.238525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:10.238525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:32:10.238525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:10.238525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:32:10.238525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:32:10.238870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:10.238870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:32:10.238870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:10.238870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:32:10.239158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:10.239158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:32:10.239158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:32:10.239158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:32:10.239158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:32:10.239158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:32:10.239158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:10.239158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:32:10.239158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:32:10.239158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:32:10.239158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:32:10.239158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:32:10.239158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:10.239158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:10.239158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:32:10.239158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:32:10.239633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:32:10.239633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:32:10.239633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:32:10.239633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:32:10.239633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:32:10.239633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:10.239633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:10.239633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:10.239633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:32:10.239633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:10.239633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:10.239633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:10.239633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:32:10.239633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:10.239633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:32:10.239633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:32:10.239633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:10.239633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:32:10.240139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:10.240139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:32:10.240139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:32:10.240139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:10.240139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:32:10.240139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:10.240139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:32:10.253758 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:32:10.254159 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:32:10.254344 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:32:10.254566 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:32:10.255099 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:32:10.255637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:10.256673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:32:10.256673 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:10.256673 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:32:10.256673 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:32:10.256673 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:10.256673 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:32:10.256673 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:10.256673 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:32:10.258013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:10.258013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:32:10.258013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:32:10.258013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:32:10.258013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:10.258013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:32:10.258013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:10.258013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:32:10.258013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:10.258013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:32:10.258013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:10.258013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:32:10.258013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:10.258013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:32:10.258013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:32:10.258802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:10.258802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:32:10.258802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:10.258802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:32:10.258802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:32:10.258802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:10.258802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:32:10.258802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:10.258802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:32:10.259338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:10.259338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:32:10.259338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:10.259338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:32:10.259338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:32:10.259338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:32:10.259338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:32:10.259338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:32:10.259338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:32:10.259338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:10.259338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:10.259955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:10.259955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:32:10.259955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:10.259955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:10.259955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:10.259955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:32:10.259955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:10.259955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:32:10.259955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:32:10.259955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:10.259955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:32:10.259955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:10.259955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:32:10.259955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:32:10.260640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:10.260640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:32:10.260640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:10.260640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:32:10.261600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:10.261600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:32:10.261600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:32:10.261600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:32:10.261600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:32:10.261600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:32:10.261600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:10.261600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:32:10.261600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:32:10.261600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:32:10.261600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:32:10.261600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:32:10.261600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:10.261600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:10.261600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:32:10.261600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:32:10.261600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:32:10.262453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:32:10.262453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:32:10.262453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:32:10.262453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:10.262453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:10.262453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:10.262453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:32:10.262453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:10.262453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:10.262453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:10.262453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:32:10.262453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:10.262453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:32:10.262453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:32:10.262453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:10.262453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:32:10.262453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:10.262453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:32:10.262453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:32:10.263356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:10.263356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:32:10.263356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:10.263356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:32:10.263356 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:32:10.263356 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:32:10.263356 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:32:10.263765 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:32:10.264077 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:32:10.264630 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:32:10.264880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:10.265552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:32:10.265552 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:10.265552 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:32:10.265552 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:32:10.265552 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:10.265552 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:32:10.265552 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:10.265552 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:32:10.266626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:10.266626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:32:10.266626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:10.266626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:32:10.266626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:32:10.266626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:32:10.266626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:32:10.266626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:10.266626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:10.266626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:10.266626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:32:10.266626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:10.266966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:10.266966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:10.266966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:32:10.266966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:10.266966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:32:10.266966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:32:10.266966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:10.266966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:32:10.266966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:10.266966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:32:10.266966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:32:10.266966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:10.266966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:32:10.267311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:10.267311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:32:10.268041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:10.268041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:32:10.268041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:32:10.268041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:32:10.268041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:32:10.268041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:32:10.268041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:10.268041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:32:10.268041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:32:10.268041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:32:10.268041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:32:10.268041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:32:10.268041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:10.268041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:10.268041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:32:10.268041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:32:10.268490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:32:10.268490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:32:10.268490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:10.268490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:10.268490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:10.268490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:32:10.268490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:10.268490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:10.268490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:10.268490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:32:10.268490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:10.268490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:32:10.268490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:32:10.268490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:10.268490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:32:10.268490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:10.268490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:32:10.268490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:32:10.268490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:10.268958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:32:10.268958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:10.268958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:32:15.022110 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:32:15.023024 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:32:15.023024 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:32:15.024073 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:32:15.055693 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:32:15.056303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:15.057231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:32:15.057231 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:15.057231 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:32:15.057231 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:32:15.057231 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:15.057231 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:32:15.057231 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:15.057231 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:32:15.058423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:15.058423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:32:15.058423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:15.058423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:32:15.058423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:32:15.058423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:32:15.058423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:32:15.058423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:15.058423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:15.058423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:15.058423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:32:15.058423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:15.059325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:15.059325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:15.059325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:32:15.059325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:15.059325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:32:15.059325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:32:15.059325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:15.059325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:32:15.059325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:15.059325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:32:15.059325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:32:15.059325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:15.059325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:32:15.060134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:15.060134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:32:15.060134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:15.060134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:32:15.060134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:32:15.060134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:32:15.060134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:32:15.060134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:32:15.060134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:15.060134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:32:15.060134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:32:15.060134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:32:15.060134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:32:15.060134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:32:15.060134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:15.061243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:15.061243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:32:15.061243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:32:15.061243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:32:15.061243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:32:15.061243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:15.061243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:15.061243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:15.061243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:32:15.061243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:15.061243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:15.061243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:15.061243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:32:15.061243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:15.061243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:32:15.061243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:32:15.061243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:15.061243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:32:15.061243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:15.062596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:32:15.062596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:32:15.062596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:15.062596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:32:15.062596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:15.062596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:32:15.227553 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:32:15.227926 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:32:15.228100 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:32:15.229022 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:32:15.229198 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:32:15.229314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:15.229981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:32:15.229981 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:15.229981 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:32:15.229981 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:32:15.229981 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:15.229981 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:32:15.229981 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:15.229981 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:32:15.231516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:15.231516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:32:15.231516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:15.231516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:32:15.231516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:32:15.231516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:32:15.231516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:32:15.231516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:15.231516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:15.231516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:15.231516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:32:15.231516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:15.232396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:15.232396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:15.232396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:32:15.232396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:15.232396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:32:15.232396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:32:15.232396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:15.232396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:32:15.232396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:15.232396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:32:15.232396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:32:15.232396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:15.232396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:32:15.233172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:15.233172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:32:15.233172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:15.233172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:32:15.233172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:32:15.233172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:32:15.233172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:32:15.233172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:32:15.233172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:15.233172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:32:15.233172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:32:15.233172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:32:15.233172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:32:15.233172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:32:15.233172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:15.234133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:15.234133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:32:15.234133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:32:15.234133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:32:15.234133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:32:15.234133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:15.234133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:15.234133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:15.234133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:32:15.234133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:15.234133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:15.234133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:15.234133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:32:15.234133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:15.234133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:32:15.234133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:32:15.234133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:15.234133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:32:15.234133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:15.235923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:32:15.235923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:32:15.235923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:15.235923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:32:15.235923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:15.235923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:32:15.235923 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:32:15.235923 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:32:15.235923 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:32:15.235923 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:32:15.236866 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:32:15.236866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:15.236866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:32:15.236866 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:15.236866 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:32:15.236866 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:32:15.236866 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:15.236866 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:32:15.236866 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:15.236866 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:32:15.237986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:15.237986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:32:15.237986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:32:15.237986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:32:15.237986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:15.237986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:32:15.237986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:15.237986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:32:15.237986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:15.237986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:32:15.237986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:15.237986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:32:15.237986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:15.237986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:32:15.237986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:32:15.238817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:15.238817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:32:15.238817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:15.238817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:32:15.238817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:32:15.238817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:15.238817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:32:15.238817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:15.238817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:32:15.239262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:15.239262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:32:15.239262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:15.239262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:32:15.239262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:32:15.239262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:32:15.239262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:32:15.239262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:32:15.239262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:32:15.239262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:32:15.239262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:15.239262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:15.239637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:15.239637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:32:15.239637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:15.239637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:15.239637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:15.239637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:32:15.239637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:15.239637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:32:15.239637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:32:15.239637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:15.239637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:32:15.239637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:15.239637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:32:15.239637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:32:15.240078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:15.240078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:32:15.240078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:15.240078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:32:15.240327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:15.240327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:32:15.240327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:32:15.240327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:32:15.240327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:32:15.240327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:32:15.240327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:15.240327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:32:15.240327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:32:15.240327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:32:15.240327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:32:15.240327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:32:15.240327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:15.240327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:15.240327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:32:15.240327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:32:15.240770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:32:15.240770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:32:15.240770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:32:15.240770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:32:15.240770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:32:15.240770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:15.240770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:15.240770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:15.240770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:32:15.240770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:15.240770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:15.240770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:15.240770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:32:15.240770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:15.240770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:32:15.240770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:32:15.240770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:15.240770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:32:15.241246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:15.241246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:32:15.241246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:32:15.241246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:15.241246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:32:15.241246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:15.241246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:32:15.254226 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:32:15.254589 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:32:15.254744 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:32:15.254950 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:32:15.255486 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:32:15.255486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:15.255922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:32:15.255922 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:15.255922 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:32:15.255922 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:32:15.255922 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:15.255922 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:32:15.255922 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:15.255922 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:32:15.257115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:15.257115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:32:15.257115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:32:15.257115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:32:15.257115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:15.257115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:32:15.257115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:15.257115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:32:15.257115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:15.257115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:32:15.257115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:15.257115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:32:15.257115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:15.257115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:32:15.257115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:32:15.258080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:15.258080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:32:15.258080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:15.258080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:32:15.258080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:32:15.258080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:15.258080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:32:15.258080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:15.258080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:32:15.258628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:15.258628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:32:15.258628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:15.258628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:32:15.258628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:32:15.258628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:32:15.258628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:32:15.258628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:32:15.258628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:32:15.258628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:15.258628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:15.259357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:15.259357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:32:15.259357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:15.259357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:15.259357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:15.259357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:32:15.259357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:15.259357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:32:15.259357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:32:15.259357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:15.259357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:32:15.259357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:15.259357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:32:15.259357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:32:15.259878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:15.259878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:32:15.259878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:15.259878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:32:15.260141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:15.260141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:32:15.260141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:32:15.260141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:32:15.260141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:32:15.260141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:32:15.260141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:15.260141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:32:15.260141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:32:15.260141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:32:15.260141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:32:15.260141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:32:15.260141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:15.260141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:15.260141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:32:15.260141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:32:15.260141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:32:15.260827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:32:15.260827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:32:15.260827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:32:15.260827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:15.260827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:15.260827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:15.260827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:32:15.260827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:15.260827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:15.260827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:15.260827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:32:15.260827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:15.260827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:32:15.260827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:32:15.260827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:15.260827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:32:15.260827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:15.260827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:32:15.260827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:32:15.261519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:15.261519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:32:15.261519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:15.261519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:32:15.263208 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:32:15.263381 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:32:15.263507 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:32:15.263660 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:32:15.263904 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:32:15.264069 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:32:15.264164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:15.264584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:32:15.264584 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:15.264584 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:32:15.264584 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:32:15.264584 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:15.264584 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:32:15.264584 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:15.264584 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:32:15.265389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:15.265389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:32:15.265389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:15.265389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:32:15.265389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:32:15.265389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:32:15.265389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:32:15.265389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:15.265389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:15.265389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:15.265389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:32:15.265389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:15.265899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:15.265899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:15.265899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:32:15.265899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:15.265899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:32:15.265899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:32:15.265899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:15.265899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:32:15.265899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:15.265899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:32:15.265899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:32:15.265899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:15.265899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:32:15.266355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:15.266355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:32:15.266355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:15.266355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:32:15.266355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:32:15.266355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:32:15.266355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:32:15.266355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:32:15.266355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:15.266355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:32:15.266355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:32:15.266355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:32:15.266355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:32:15.266355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:32:15.266355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:15.266953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:15.266953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:32:15.266953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:32:15.266953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:32:15.266953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:32:15.266953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:15.266953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:15.266953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:15.266953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:32:15.266953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:15.266953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:15.266953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:15.266953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:32:15.266953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:15.266953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:32:15.266953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:32:15.266953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:15.266953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:32:15.266953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:15.267670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:32:15.267670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:32:15.267670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:15.267670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:32:15.267670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:15.267670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:32:20.022768 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:32:20.023742 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:32:20.023742 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:32:20.024579 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:32:20.057776 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:32:20.058163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:20.059063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:32:20.059063 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:20.059063 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:32:20.059063 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:32:20.059063 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:20.059063 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:32:20.059063 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:20.059063 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:32:20.059849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:20.059849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:32:20.059849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:20.059849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:32:20.059849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:32:20.059849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:32:20.059849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:32:20.059849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:20.059849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:20.059849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:20.059849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:32:20.059849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:20.060251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:20.060251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:20.060251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:32:20.060251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:20.060251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:32:20.060251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:32:20.060251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:20.060251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:32:20.060251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:20.060251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:32:20.060251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:32:20.060251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:20.060251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:32:20.060792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:20.060792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:32:20.060792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:20.060792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:32:20.060792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:32:20.060792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:32:20.060792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:32:20.060792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:32:20.060792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:20.060792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:32:20.060792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:32:20.060792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:32:20.060792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:32:20.060792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:32:20.060792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:20.061978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:20.061978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:32:20.061978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:32:20.061978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:32:20.061978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:32:20.061978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:20.061978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:20.061978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:20.061978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:32:20.061978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:20.061978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:20.061978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:20.061978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:32:20.061978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:20.061978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:32:20.061978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:32:20.061978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:20.061978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:32:20.061978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:20.062953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:32:20.062953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:32:20.062953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:20.062953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:32:20.062953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:20.062953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:32:20.228156 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:32:20.228971 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:32:20.229088 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:32:20.230149 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:32:20.230558 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:32:20.230846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:20.232541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:32:20.232541 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:20.232541 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:32:20.232541 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:32:20.232541 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:20.232541 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:32:20.232541 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:20.232541 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:32:20.233576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:20.233576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:32:20.233576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:20.233576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:32:20.233576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:32:20.233576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:32:20.233576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:32:20.233576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:20.233576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:20.233576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:20.233576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:32:20.233576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:20.234001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:20.234001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:20.234001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:32:20.234001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:20.234001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:32:20.234001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:32:20.234001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:20.234001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:32:20.234001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:20.234001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:32:20.234001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:32:20.234001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:20.234001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:32:20.234258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:20.234258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:32:20.234653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:20.234653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:32:20.234653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:32:20.234653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:32:20.234653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:32:20.234653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:32:20.234653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:20.234653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:32:20.234653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:32:20.234653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:32:20.234653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:32:20.234653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:32:20.234653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:20.234653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:20.234653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:32:20.234653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:32:20.235010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:32:20.235010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:32:20.235010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:20.235010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:20.235010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:20.235010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:32:20.235010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:20.235010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:20.235010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:20.235010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:32:20.235010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:20.235010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:32:20.235010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:32:20.235010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:20.235010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:32:20.235010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:20.235010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:32:20.235010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:32:20.235010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:20.235389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:32:20.235389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:20.235389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:32:20.235632 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:32:20.235721 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:32:20.235786 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:32:20.235845 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:32:20.236310 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:32:20.236638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:20.237344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:32:20.237344 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:20.237344 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:32:20.237344 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:32:20.237344 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:20.237344 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:32:20.237344 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:20.237344 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:32:20.237946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:20.237946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:32:20.237946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:32:20.237946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:32:20.237946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:20.237946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:32:20.237946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:20.237946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:32:20.237946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:20.237946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:32:20.237946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:20.237946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:32:20.237946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:20.237946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:32:20.237946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:32:20.238322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:20.238322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:32:20.238322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:20.238322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:32:20.238322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:32:20.238322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:20.238322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:32:20.238322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:20.238322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:32:20.238542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:20.238542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:32:20.238542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:20.238542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:32:20.238542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:32:20.238542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:32:20.238542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:32:20.238542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:32:20.238542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:32:20.238542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:32:20.238542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:20.238542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:20.238798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:20.238798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:32:20.238798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:20.238798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:20.238798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:20.238798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:32:20.238798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:20.238798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:32:20.238798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:32:20.238798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:20.238798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:32:20.238798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:20.238798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:32:20.238798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:32:20.239052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:20.239052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:32:20.239052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:20.239052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:32:20.239225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:20.239225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:32:20.239225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:32:20.239225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:32:20.239225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:32:20.239225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:32:20.239225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:20.239225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:32:20.239225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:32:20.239225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:32:20.239225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:32:20.239225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:32:20.239225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:20.239225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:20.239225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:32:20.239225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:32:20.239555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:32:20.239555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:32:20.239555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:32:20.239555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:32:20.239555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:32:20.239555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:20.239555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:20.239555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:20.239555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:32:20.239555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:20.239555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:20.239555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:20.239555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:32:20.239555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:20.239555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:32:20.239555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:32:20.239555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:20.239555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:32:20.239888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:20.239888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:32:20.239888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:32:20.239888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:20.239888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:32:20.239888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:20.239888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:32:20.254610 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:32:20.254911 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:32:20.255078 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:32:20.255270 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:32:20.255815 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:32:20.256199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:20.256970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:32:20.256970 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:20.256970 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:32:20.256970 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:32:20.256970 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:20.256970 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:32:20.256970 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:20.256970 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:32:20.258164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:20.258164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:32:20.258164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:32:20.258164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:32:20.258164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:20.258164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:32:20.258164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:20.258164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:32:20.258164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:20.258164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:32:20.258164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:20.258164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:32:20.258164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:20.258164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:32:20.258164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:32:20.258463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:20.258463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:32:20.258463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:20.258463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:32:20.258463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:32:20.258463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:20.258463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:32:20.258463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:20.258463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:32:20.259611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:20.259611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:32:20.259611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:20.259611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:32:20.259611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:32:20.259611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:32:20.259611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:32:20.259611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:32:20.259611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:32:20.259611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:20.259611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:20.259842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:20.259842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:32:20.259842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:20.259842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:20.259842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:20.259842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:32:20.259842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:20.259842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:32:20.259842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:32:20.259842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:20.259842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:32:20.259842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:20.259842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:32:20.259842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:32:20.260092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:20.260092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:32:20.260092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:20.260092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:32:20.261290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:20.261290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:32:20.261290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:32:20.261290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:32:20.261290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:32:20.261290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:32:20.261290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:20.261290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:32:20.261290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:32:20.261290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:32:20.261290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:32:20.261290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:32:20.261290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:20.261290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:20.261290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:32:20.261290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:32:20.261290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:32:20.261655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:32:20.261655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:32:20.261655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:32:20.261655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:20.261655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:20.261655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:20.261655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:32:20.261655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:20.261655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:20.261655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:20.261655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:32:20.261655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:20.261655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:32:20.261655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:32:20.261655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:20.261655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:32:20.261655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:20.261655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:32:20.261655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:32:20.262019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:20.262019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:32:20.262019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:20.262019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:32:20.263466 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:32:20.263653 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:32:20.263771 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:32:20.263914 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:32:20.264358 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:32:20.264756 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:32:20.265033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:20.265766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:32:20.265766 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:20.265766 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:32:20.265766 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:32:20.265766 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:20.265766 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:32:20.265766 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:20.265766 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:32:20.266568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:20.266568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:32:20.266568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:20.266568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:32:20.266568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:32:20.266568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:32:20.266568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:32:20.266568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:20.266568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:20.266568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:20.266568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:32:20.266568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:20.266872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:20.266872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:20.266872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:32:20.266872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:20.266872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:32:20.266872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:32:20.266872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:20.266872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:32:20.266872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:20.266872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:32:20.266872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:32:20.266872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:20.266872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:32:20.267116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:20.267116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:32:20.267542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:20.267542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:32:20.267542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:32:20.267542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:32:20.267542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:32:20.267542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:32:20.267542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:20.267542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:32:20.267542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:32:20.267542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:32:20.267542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:32:20.267542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:32:20.267542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:20.267542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:20.267542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:32:20.267542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:32:20.267986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:32:20.267986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:32:20.267986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:20.267986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:20.267986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:20.267986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:32:20.267986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:20.267986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:20.267986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:20.267986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:32:20.267986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:20.267986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:32:20.267986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:32:20.267986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:20.267986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:32:20.267986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:20.267986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:32:20.267986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:32:20.267986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:20.268344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:32:20.268344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:20.268344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:32:25.023456 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:32:25.024505 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:32:25.024505 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:32:25.025462 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:32:25.027205 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG dbcounter [-] [105581] Writing DB stats neutron:SELECT=240 {{(pid=105581) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 14:32:25.057521 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:32:25.057901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:25.058648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:32:25.058648 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:25.058648 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:32:25.058648 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:32:25.058648 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:25.058648 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:32:25.058648 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:25.058648 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:32:25.059281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:25.059281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:32:25.059281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:25.059281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:32:25.059281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:32:25.059281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:32:25.059281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:32:25.059281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:25.059281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:25.059281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:25.059281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:32:25.059281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:25.059969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:25.059969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:25.059969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:32:25.059969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:25.059969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:32:25.059969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:32:25.059969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:25.059969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:32:25.059969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:25.059969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:32:25.059969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:32:25.059969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:25.059969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:32:25.060743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:25.060743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:32:25.060743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:25.060743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:32:25.060743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:32:25.060743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:32:25.060743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:32:25.060743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:32:25.060743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:25.060743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:32:25.060743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:32:25.060743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:32:25.060743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:32:25.060743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:32:25.060743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:25.061611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:25.061611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:32:25.061611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:32:25.061611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:32:25.061611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:32:25.061611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:25.061611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:25.061611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:25.061611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:32:25.061611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:25.061611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:25.061611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:25.061611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:32:25.061611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:25.061611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:32:25.061611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:32:25.061611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:25.061611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:32:25.061611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:25.062632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:32:25.062632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:32:25.062632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:25.062632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:32:25.062632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:25.062632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:32:25.253998 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:32:25.253998 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:32:25.253998 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:32:25.253998 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:32:25.254743 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:32:25.255033 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:32:25.255442 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:32:25.255633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:25.256517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:32:25.256517 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:25.256517 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:32:25.256517 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:32:25.256517 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:25.256517 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:32:25.256517 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:25.256517 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:32:25.257918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:25.257918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:32:25.257918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:25.257918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:32:25.257918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:32:25.257918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:32:25.257918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:32:25.257918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:25.257918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:25.257918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:25.257918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:32:25.257918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:25.258777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:25.258777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:25.258777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:32:25.258777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:25.258777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:32:25.258777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:32:25.258777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:25.258777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:32:25.258777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:25.258777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:32:25.258777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:32:25.258777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:25.258777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:32:25.259617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:25.259617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:32:25.259825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:25.259825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:32:25.259825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:32:25.259825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:32:25.259825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:32:25.259825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:32:25.259825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:25.259825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:32:25.259825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:32:25.259825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:32:25.259825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:32:25.259825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:32:25.259825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:25.259825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:25.259825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:32:25.259825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:32:25.260805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:32:25.260805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:32:25.260805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:25.260805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:25.260805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:25.260805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:32:25.260805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:25.260805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:25.260805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:25.260805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:32:25.260805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:25.260805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:32:25.260805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:32:25.260805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:25.260805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:32:25.260805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:25.260805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:32:25.260805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:32:25.260805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:25.261605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:32:25.261605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:25.261605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:32:25.261605 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:32:25.261605 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:32:25.261605 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:32:25.261605 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:32:25.261605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:25.262005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:32:25.262005 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:25.262005 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:32:25.262005 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:32:25.262005 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:25.262005 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:32:25.262005 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:25.262005 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:32:25.262657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:25.262657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:32:25.262657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:32:25.262657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:32:25.262657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:25.262657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:32:25.262657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:25.262657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:32:25.262657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:25.262657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:32:25.262657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:25.262657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:32:25.262657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:25.262657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:32:25.262657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:32:25.263393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:25.263393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:32:25.263393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:25.263393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:32:25.263393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:32:25.263393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:25.263393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:32:25.263393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:25.263393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:32:25.263849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:25.263849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:32:25.263849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:25.263849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:32:25.263849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:32:25.263849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:32:25.263849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:32:25.263849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:32:25.263849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:32:25.263849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:32:25.263849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:25.263849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:25.264295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:25.264295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:32:25.264295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:25.264295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:25.264295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:25.264295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:32:25.264295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:25.264295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:32:25.264295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:32:25.264295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:25.264295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:32:25.264295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:25.264295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:32:25.264295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:32:25.264837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:25.264837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:32:25.264837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:25.264837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:32:25.264837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:25.264837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:32:25.264837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:32:25.264837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:32:25.264837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:32:25.264837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:32:25.264837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:25.264837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:32:25.264837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:32:25.264837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:32:25.265390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:32:25.265390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:32:25.265390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:25.265390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:25.265390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:32:25.265390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:32:25.265390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:32:25.265390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:32:25.265390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:32:25.265390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:32:25.265390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:32:25.265390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:25.265390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:25.265390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:25.265390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:32:25.265390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:25.265390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:25.265390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:25.265390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:32:25.266206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:25.266206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:32:25.266206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:32:25.266206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:25.266206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:32:25.266206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:25.266206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:32:25.266206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:32:25.266206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:25.266206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:32:25.266206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:25.266206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:32:25.266206 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:32:25.266206 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:32:25.266749 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:32:25.266749 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:32:25.266749 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:32:25.266749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:25.266749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:32:25.266749 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:25.266749 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:32:25.266749 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:32:25.266749 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:25.266749 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:32:25.266749 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:25.266749 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:32:25.267126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:25.267126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:32:25.267126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:32:25.267126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:32:25.267126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:25.267126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:32:25.267126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:25.267126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:32:25.267126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:25.267126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:32:25.267126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:25.267126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:32:25.267126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:25.267126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:32:25.267126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:32:25.267702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:25.267702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:32:25.267702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:25.267702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:32:25.267702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:32:25.267702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:25.267702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:32:25.267702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:25.267702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:32:25.267702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:25.267702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:32:25.267702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:25.267702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:32:25.267702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:32:25.268349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:32:25.268349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:32:25.268349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:32:25.268349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:32:25.268349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:25.268349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:25.268349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:25.268349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:32:25.268349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:25.268349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:25.268349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:25.268349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:32:25.268349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:25.269145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:32:25.269145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:32:25.269145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:25.269145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:32:25.269145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:25.269145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:32:25.269145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:32:25.269145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:25.269145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:32:25.269145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:25.269145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:32:25.269145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:25.269641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:32:25.269641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:32:25.269641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:32:25.269641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:32:25.269641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:32:25.269641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:25.269641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:32:25.269641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:32:25.269641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:32:25.269641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:32:25.269641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:32:25.269641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:25.269641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:25.269641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:32:25.269641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:32:25.269641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:32:25.269641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:32:25.269641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:32:25.269641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:32:25.270205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:25.270205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:25.270205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:25.270205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:32:25.270205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:25.270205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:25.270205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:25.270205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:32:25.270205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:25.270205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:32:25.270205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:32:25.270205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:25.270205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:32:25.270205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:25.270205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:32:25.270205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:32:25.270205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:25.270205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:32:25.270205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:25.270205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:32:25.270802 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:32:25.270802 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:32:25.270802 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:32:25.270802 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:32:25.270802 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:32:25.270802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:25.271008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:32:25.271008 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:25.271008 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:32:25.271008 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:32:25.271008 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:25.271008 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:32:25.271008 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:25.271008 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:32:25.271240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:25.271240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:32:25.271240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:25.271240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:32:25.271240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:32:25.271240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:32:25.271240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:32:25.271240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:25.271240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:25.271240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:25.271240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:32:25.271240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:25.271636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:25.271636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:25.271636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:32:25.271636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:25.271636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:32:25.271636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:32:25.271636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:25.271636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:32:25.271636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:25.271636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:32:25.271636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:32:25.271636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:25.271636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:32:25.272196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:25.272196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:32:25.272269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:25.272269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:32:25.272269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:32:25.272269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:32:25.272269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:32:25.272269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:32:25.272269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:25.272269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:32:25.272269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:32:25.272269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:32:25.272269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:32:25.272269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:32:25.272269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:25.272269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:25.272269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:32:25.272269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:32:25.272873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:32:25.272873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:32:25.272873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:25.272873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:25.272873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:25.272873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:32:25.272873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:25.272873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:25.272873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:25.272873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:32:25.272873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:25.272873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:32:25.272873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:32:25.272873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:25.272873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:32:25.272873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:25.272873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:32:25.272873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:32:25.272873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:25.273298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:32:25.273298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:25.273298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:32:30.024293 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:32:30.025190 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:32:30.025190 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:32:30.026498 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:32:30.064457 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:32:30.064678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:30.065254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:32:30.065254 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:30.065254 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:32:30.065254 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:32:30.065254 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:30.065254 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:32:30.065254 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:30.065254 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:32:30.066368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:30.066368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:32:30.066368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:30.066368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:32:30.066368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:32:30.066368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:32:30.066368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:32:30.066368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:30.066368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:30.066368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:30.066368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:32:30.066368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:30.067112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:30.067112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:30.067112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:32:30.067112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:30.067112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:32:30.067112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:32:30.067112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:30.067112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:32:30.067112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:30.067112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:32:30.067112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:32:30.067112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:30.067112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:32:30.068096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:30.068096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:32:30.068096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:30.068096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:32:30.068096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:32:30.068096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:32:30.068096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:32:30.068096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:32:30.068096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:30.068096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:32:30.068096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:32:30.068096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:32:30.068096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:32:30.068096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:32:30.068096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:30.068979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:30.068979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:32:30.068979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:32:30.068979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:32:30.068979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:32:30.068979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:30.068979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:30.068979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:30.068979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:32:30.068979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:30.068979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:30.068979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:30.068979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:32:30.068979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:30.068979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:32:30.068979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:32:30.068979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:30.068979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:32:30.068979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:30.070109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:32:30.070109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:32:30.070109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:30.070109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:32:30.070109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:30.070109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:32:30.253580 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:32:30.254134 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:32:30.254317 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:32:30.255337 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:32:30.255734 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:32:30.255992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:30.256928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:32:30.256928 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:30.256928 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:32:30.256928 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:32:30.256928 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:30.256928 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:32:30.256928 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:30.256928 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:32:30.258549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:30.258549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:32:30.258549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:30.258549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:32:30.258549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:32:30.258549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:32:30.258549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:32:30.258549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:30.258549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:30.258549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:30.258549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:32:30.258549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:30.259220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:30.259220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:30.259220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:32:30.259220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:30.259220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:32:30.259220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:32:30.259220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:30.259220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:32:30.259220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:30.259220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:32:30.259220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:32:30.259220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:30.259220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:32:30.259877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:30.259877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:32:30.260567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:30.260567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:32:30.260567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:32:30.260567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:32:30.260567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:32:30.260567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:32:30.260567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:30.260567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:32:30.260567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:32:30.260567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:32:30.260567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:32:30.260567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:32:30.260567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:30.260567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:30.260567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:32:30.260567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:32:30.261478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:32:30.261478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:32:30.261478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:30.261478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:30.261478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:30.261478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:32:30.261478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:30.261478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:30.261478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:30.261478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:32:30.261478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:30.261478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:32:30.261478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:32:30.261478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:30.261478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:32:30.261478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:30.261478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:32:30.261478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:32:30.261478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:30.262197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:32:30.262197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:30.262197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:32:30.262197 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:32:30.262197 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:32:30.262197 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:32:30.262197 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:32:30.262197 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:32:30.262601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:30.262601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:32:30.262601 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:30.262601 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:32:30.262601 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:32:30.262601 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:30.262601 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:32:30.262601 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:30.262601 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:32:30.263675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:30.263675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:32:30.263675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:32:30.263675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:32:30.263675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:30.263675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:32:30.263675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:30.263675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:32:30.263675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:30.263675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:32:30.263675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:30.263675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:32:30.263675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:30.263675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:32:30.263675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:32:30.264387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:30.264387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:32:30.264387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:30.264387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:32:30.264387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:32:30.264387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:30.264387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:32:30.264387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:30.264387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:32:30.264907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:30.264907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:32:30.264907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:30.264907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:32:30.264907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:32:30.264907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:32:30.264907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:32:30.264907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:32:30.264907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:32:30.264907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:32:30.264907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:30.264907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:30.265337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:30.265337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:32:30.265337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:30.265337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:30.265337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:30.265337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:32:30.265337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:30.265337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:32:30.265337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:32:30.265337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:30.265337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:32:30.265337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:30.265337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:32:30.265337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:32:30.265909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:30.265909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:32:30.265909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:30.265909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:32:30.265909 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:32:30.266759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:30.266759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:32:30.266759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:32:30.266759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:32:30.266759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:32:30.266759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:32:30.266759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:30.266759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:32:30.266759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:32:30.266759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:32:30.266759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:32:30.266759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:32:30.266759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:30.266759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:30.266759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:32:30.266759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:32:30.267479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:32:30.267479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:32:30.267479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:32:30.267479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:32:30.267479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:32:30.267479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:30.267479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:30.267479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:30.267479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:32:30.267479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:30.267479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:30.267479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:30.267479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:32:30.267479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:30.267479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:32:30.267479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:32:30.267479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:30.267479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:32:30.268106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:30.268106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:32:30.268106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:32:30.268106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:30.268106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:32:30.268106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:30.268106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:32:30.268106 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:32:30.268106 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:32:30.268106 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:32:30.268576 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:32:30.268576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:30.268576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:32:30.268576 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:30.268576 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:32:30.268576 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:32:30.268576 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:30.268576 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:32:30.268576 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:30.268576 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:32:30.269568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:30.269568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:32:30.269568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:32:30.269568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:32:30.269568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:30.269568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:32:30.269568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:30.269568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:32:30.269568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:30.269568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:32:30.269568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:30.269568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:32:30.269568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:30.269568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:32:30.269568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:32:30.270145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:30.270145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:32:30.270145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:30.270145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:32:30.270145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:32:30.270145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:30.270145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:32:30.270145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:30.270145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:32:30.270145 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:32:30.270999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:30.270999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:32:30.270999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:30.270999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:32:30.270999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:32:30.270999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:32:30.270999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:32:30.270999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:32:30.270999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:32:30.270999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:30.270999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:30.271487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:30.271487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:32:30.271487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:30.271487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:30.271487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:30.271487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:32:30.271487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:30.271487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:32:30.271487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:32:30.271487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:30.271487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:32:30.271487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:30.271487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:32:30.271487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:32:30.272054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:30.272054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:32:30.272054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:30.272054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:32:30.272519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:30.272519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:32:30.272519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:32:30.272519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:32:30.272519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:32:30.272519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:32:30.272519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:30.272519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:32:30.272519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:32:30.272519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:32:30.272519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:32:30.272519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:32:30.272519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:30.272519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:30.272519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:32:30.272519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:32:30.272519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:32:30.273203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:32:30.273203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:32:30.273203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:32:30.273203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:30.273203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:30.273203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:30.273203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:32:30.273203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:30.273203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:30.273203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:30.273203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:32:30.273203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:30.273203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:32:30.273203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:32:30.273203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:30.273203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:32:30.273203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:30.273203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:32:30.273203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:32:30.273575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:30.273575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:32:30.273575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:30.273575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:32:30.273575 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:32:30.273575 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:32:30.273575 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:32:30.273575 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:32:30.273920 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:32:30.274122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:30.274801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:32:30.274801 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:30.274801 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:32:30.274801 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:32:30.274801 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:30.274801 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:32:30.274801 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:30.274801 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:32:30.275464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:30.275464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:32:30.275464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:30.275464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:32:30.275464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:32:30.275464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:32:30.275464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:32:30.275464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:30.275464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:30.275464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:30.275464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:32:30.275464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:30.275849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:30.275849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:30.275849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:32:30.275849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:30.275849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:32:30.275849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:32:30.275849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:30.275849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:32:30.275849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:30.275849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:32:30.275849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:32:30.275849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:30.275849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:32:30.276266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:30.276266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:32:30.276476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:30.276476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:32:30.276476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:32:30.276476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:32:30.276476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:32:30.276476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:32:30.276476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:30.276476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:32:30.276476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:32:30.276476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:32:30.276476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:32:30.276476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:32:30.276476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:30.276476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:30.276476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:32:30.276476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:32:30.276850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:32:30.276850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:32:30.276850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:30.276850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:30.276850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:30.276850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:32:30.276850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:30.276850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:30.276850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:30.276850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:32:30.276850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:30.276850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:32:30.276850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:32:30.276850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:30.276850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:32:30.276850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:30.276850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:32:30.276850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:32:30.276850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:30.277207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:32:30.277207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:30.277207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:32:35.024980 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:32:35.025901 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:32:35.025901 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:32:35.026833 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:32:35.061660 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:32:35.062073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:35.062904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:32:35.062904 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:35.062904 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:32:35.062904 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:32:35.062904 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:35.062904 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:32:35.062904 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:35.062904 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:32:35.063685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:35.063685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:32:35.063685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:35.063685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:32:35.063685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:32:35.063685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:32:35.063685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:32:35.063685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:35.063685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:35.063685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:35.063685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:32:35.063685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:35.064251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:35.064251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:35.064251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:32:35.064251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:35.064251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:32:35.064251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:32:35.064251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:35.064251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:32:35.064251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:35.064251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:32:35.064251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:32:35.064251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:35.064251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:32:35.064851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:35.064851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:32:35.064851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:35.064851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:32:35.064851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:32:35.064851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:32:35.064851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:32:35.064851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:32:35.064851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:35.064851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:32:35.064851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:32:35.064851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:32:35.064851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:32:35.064851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:32:35.064851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:35.065551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:35.065551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:32:35.065551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:32:35.065551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:32:35.065551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:32:35.065551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:35.065551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:35.065551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:35.065551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:32:35.065551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:35.065551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:35.065551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:35.065551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:32:35.065551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:35.065551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:32:35.065551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:32:35.065551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:35.065551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:32:35.065551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:35.066374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:32:35.066374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:32:35.066374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:35.066374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:32:35.066374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:35.066374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:32:35.254214 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:32:35.254724 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:32:35.254912 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:32:35.256069 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:32:35.256395 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:32:35.256662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:35.257575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:32:35.257575 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:35.257575 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:32:35.257575 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:32:35.257575 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:35.257575 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:32:35.257575 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:35.257575 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:32:35.259154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:35.259154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:32:35.259154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:35.259154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:32:35.259154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:32:35.259154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:32:35.259154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:32:35.259154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:35.259154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:35.259154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:35.259154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:32:35.259154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:35.259781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:35.259781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:35.259781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:32:35.259781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:35.259781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:32:35.259781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:32:35.259781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:35.259781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:32:35.259781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:35.259781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:32:35.259781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:32:35.259781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:35.259781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:32:35.260366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:35.260366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:32:35.261230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:35.261230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:32:35.261230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:32:35.261230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:32:35.261230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:32:35.261230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:32:35.261230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:35.261230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:32:35.261230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:32:35.261230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:32:35.261230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:32:35.261230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:32:35.261230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:35.261230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:35.261230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:32:35.261230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:32:35.261958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:32:35.261958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:32:35.261958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:35.261958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:35.261958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:35.261958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:32:35.261958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:35.261958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:35.261958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:35.261958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:32:35.261958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:35.261958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:32:35.261958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:32:35.261958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:35.261958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:32:35.261958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:35.261958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:32:35.261958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:32:35.261958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:35.262795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:32:35.262795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:35.262795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:32:35.262795 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:32:35.262795 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:32:35.262795 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:32:35.262795 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:32:35.262795 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:32:35.263200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:35.263577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:32:35.263577 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:35.263577 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:32:35.263577 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:32:35.263577 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:35.263577 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:32:35.263577 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:35.263577 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:32:35.264317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:35.264317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:32:35.264317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:32:35.264317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:32:35.264317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:35.264317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:32:35.264317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:35.264317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:32:35.264317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:35.264317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:32:35.264317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:35.264317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:32:35.264317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:35.264317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:32:35.264317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:32:35.265036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:35.265036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:32:35.265036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:35.265036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:32:35.265036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:32:35.265036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:35.265036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:32:35.265036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:35.265036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:32:35.265304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:35.265304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:32:35.265304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:35.265304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:32:35.265304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:32:35.265304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:32:35.265304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:32:35.265304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:32:35.265304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:32:35.265304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:32:35.265304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:35.265304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:35.265668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:35.265668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:32:35.265668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:35.265668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:35.265668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:35.265668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:32:35.265668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:35.265668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:32:35.265668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:32:35.265668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:35.265668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:32:35.265668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:35.265668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:32:35.265668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:32:35.266056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:35.266056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:32:35.266056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:35.266056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:32:35.266388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:35.266388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:32:35.266388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:32:35.266388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:32:35.266388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:32:35.266388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:32:35.266388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:35.266388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:32:35.266388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:32:35.266388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:32:35.266388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:32:35.266388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:32:35.266388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:35.266388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:35.266388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:32:35.266388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:32:35.266857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:32:35.266857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:32:35.266857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:32:35.266857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:32:35.266857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:32:35.266857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:35.266857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:35.266857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:35.266857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:32:35.266857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:35.266857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:35.266857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:35.266857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:32:35.266857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:35.266857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:32:35.266857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:32:35.266857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:35.266857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:32:35.267314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:35.267314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:32:35.267314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:32:35.267314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:35.267314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:32:35.267314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:35.267314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:32:35.267314 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:32:35.267314 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:32:35.267564 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:32:35.267564 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:32:35.267796 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:32:35.267965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:35.268365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:32:35.268365 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:35.268365 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:32:35.268365 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:32:35.268365 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:35.268365 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:32:35.268365 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:35.268365 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:32:35.269086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:35.269086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:32:35.269086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:32:35.269086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:32:35.269086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:35.269086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:32:35.269086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:35.269086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:32:35.269086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:35.269086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:32:35.269086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:35.269086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:32:35.269086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:35.269086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:32:35.269086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:32:35.269532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:35.269532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:32:35.269532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:35.269532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:32:35.269532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:32:35.269532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:35.269532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:32:35.269532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:35.269532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:32:35.269924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:35.269924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:32:35.269924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:35.269924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:32:35.269924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:32:35.269924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:32:35.269924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:32:35.269924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:32:35.269924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:32:35.269924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:35.269924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:35.270213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:35.270213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:32:35.270213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:35.270213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:35.270213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:35.270213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:32:35.270213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:35.270213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:32:35.270213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:32:35.270213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:35.270213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:32:35.270213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:35.270213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:32:35.270213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:32:35.270569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:35.270569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:32:35.270569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:35.270569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:32:35.270927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:35.270927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:32:35.270927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:32:35.270927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:32:35.270927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:32:35.270927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:32:35.270927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:35.270927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:32:35.270927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:32:35.270927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:32:35.270927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:32:35.270927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:32:35.270927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:35.270927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:35.270927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:32:35.270927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:32:35.270927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:32:35.271443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:32:35.271443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:32:35.271443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:32:35.271443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:35.271443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:35.271443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:35.271443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:32:35.271443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:35.271443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:35.271443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:35.271443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:32:35.271443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:35.271443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:32:35.271443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:32:35.271443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:35.271443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:32:35.271443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:35.271443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:32:35.271443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:32:35.271922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:35.271922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:32:35.271922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:35.271922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:32:35.272752 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:32:35.272889 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:32:35.272960 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:32:35.273070 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:32:35.273525 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:32:35.273852 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:32:35.274064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:35.274717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:32:35.274717 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:35.274717 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:32:35.274717 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:32:35.274717 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:35.274717 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:32:35.274717 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:35.274717 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:32:35.275333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:35.275333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:32:35.275333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:35.275333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:32:35.275333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:32:35.275333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:32:35.275333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:32:35.275333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:35.275333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:35.275333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:35.275333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:32:35.275333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:35.275718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:35.275718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:35.275718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:32:35.275718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:35.275718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:32:35.275718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:32:35.275718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:35.275718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:32:35.275718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:35.275718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:32:35.275718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:32:35.275718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:35.275718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:32:35.276066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:35.276066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:32:35.276066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:35.276066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:32:35.276066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:32:35.276066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:32:35.276066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:32:35.276066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:32:35.276066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:35.276066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:32:35.276066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:32:35.276066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:32:35.276066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:32:35.276066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:32:35.276066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:35.276464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:35.276464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:32:35.276464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:32:35.276464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:32:35.276464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:32:35.276464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:35.276464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:35.276464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:35.276464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:32:35.276464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:35.276464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:35.276464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:35.276464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:32:35.276464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:35.276464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:32:35.276464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:32:35.276464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:35.276464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:32:35.276464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:35.276934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:32:35.276934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:32:35.276934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:35.276934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:32:35.276934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:35.276934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:32:40.025663 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:32:40.026632 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:32:40.026632 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:32:40.027784 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:32:40.065815 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:32:40.066078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:40.066739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:32:40.066739 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:40.066739 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:32:40.066739 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:32:40.066739 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:40.066739 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:32:40.066739 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:40.066739 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:32:40.067511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:40.067511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:32:40.067511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:40.067511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:32:40.067511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:32:40.067511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:32:40.067511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:32:40.067511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:40.067511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:40.067511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:40.067511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:32:40.067511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:40.068286 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:40.068286 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:40.068286 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:32:40.068286 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:40.068286 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:32:40.068286 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:32:40.068286 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:40.068286 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:32:40.068286 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:40.068286 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:32:40.068286 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:32:40.068286 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:40.068286 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:32:40.069077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:40.069077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:32:40.069077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:40.069077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:32:40.069077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:32:40.069077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:32:40.069077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:32:40.069077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:32:40.069077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:40.069077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:32:40.069077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:32:40.069077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:32:40.069077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:32:40.069077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:32:40.069077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:40.069885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:40.069885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:32:40.069885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:32:40.069885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:32:40.069885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:32:40.069885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:40.069885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:40.069885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:40.069885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:32:40.069885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:40.069885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:40.069885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:40.069885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:32:40.069885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:40.069885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:32:40.069885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:32:40.069885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:40.069885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:32:40.069885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:40.070917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:32:40.070917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:32:40.070917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:40.070917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:32:40.070917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:40.070917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:32:40.254947 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:32:40.255644 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:32:40.255830 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:32:40.257176 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:32:40.257838 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:32:40.258167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:40.259219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:32:40.259219 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:40.259219 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:32:40.259219 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:32:40.259219 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:40.259219 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:32:40.259219 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:40.259219 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:32:40.260785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:40.260785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:32:40.260785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:40.260785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:32:40.260785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:32:40.260785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:32:40.260785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:32:40.260785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:40.260785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:40.260785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:40.260785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:32:40.260785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:40.261169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:40.261169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:40.261169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:32:40.261169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:40.261169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:32:40.261169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:32:40.261169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:40.261169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:32:40.261169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:40.261169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:32:40.261169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:32:40.261169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:40.261169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:32:40.261542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:40.261542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:32:40.262418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:40.262418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:32:40.262418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:32:40.262418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:32:40.262418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:32:40.262418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:32:40.262418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:40.262418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:32:40.262418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:32:40.262418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:32:40.262418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:32:40.262418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:32:40.262418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:40.262418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:40.262418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:32:40.262418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:32:40.262853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:32:40.262853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:32:40.262853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:40.262853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:40.262853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:40.262853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:32:40.262853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:40.262853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:40.262853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:40.262853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:32:40.262853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:40.262853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:32:40.262853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:32:40.262853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:40.262853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:32:40.262853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:40.262853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:32:40.262853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:32:40.262853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:40.263380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:32:40.263380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:40.263380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:32:40.263380 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:32:40.263380 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:32:40.263380 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:32:40.263380 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:32:40.263802 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:32:40.264013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:40.264635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:32:40.264635 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:40.264635 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:32:40.264635 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:32:40.264635 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:40.264635 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:32:40.264635 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:40.264635 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:32:40.265989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:40.265989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:32:40.265989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:32:40.265989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:32:40.265989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:40.265989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:32:40.265989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:40.265989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:32:40.265989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:40.265989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:32:40.265989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:40.265989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:32:40.265989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:40.265989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:32:40.265989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:32:40.266487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:40.266487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:32:40.266487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:40.266487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:32:40.266487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:32:40.266487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:40.266487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:32:40.266487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:40.266487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:32:40.267612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:40.267612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:32:40.267612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:40.267612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:32:40.267612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:32:40.267612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:32:40.267612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:32:40.267612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:32:40.267612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:32:40.267612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:32:40.267612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:40.267612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:40.267999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:40.267999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:32:40.267999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:40.267999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:40.267999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:40.267999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:32:40.267999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:40.267999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:32:40.267999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:32:40.267999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:40.267999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:32:40.267999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:40.267999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:32:40.267999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:32:40.268398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:40.268398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:32:40.268398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:40.268398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:32:40.269382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:40.269382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:32:40.269382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:32:40.269382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:32:40.269382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:32:40.269382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:32:40.269382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:40.269382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:32:40.269382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:32:40.269382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:32:40.269382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:32:40.269382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:32:40.269382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:40.269382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:40.269382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:32:40.269382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:32:40.269957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:32:40.269957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:32:40.269957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:32:40.269957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:32:40.269957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:32:40.269957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:40.269957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:40.269957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:40.269957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:32:40.269957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:40.269957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:40.269957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:40.269957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:32:40.269957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:40.269957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:32:40.269957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:32:40.269957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:40.269957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:32:40.270494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:40.270494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:32:40.270494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:32:40.270494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:40.270494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:32:40.270494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:40.270494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:32:40.270494 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:32:40.270494 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:32:40.270494 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:32:40.270792 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:32:40.270792 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:32:40.270792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:40.271156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:32:40.271156 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:40.271156 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:32:40.271156 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:32:40.271156 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:40.271156 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:32:40.271156 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:40.271156 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:32:40.271923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:40.271923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:32:40.271923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:32:40.271923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:32:40.271923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:40.271923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:32:40.271923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:40.271923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:32:40.271923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:40.271923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:32:40.271923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:40.271923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:32:40.271923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:40.271923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:32:40.271923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:32:40.272358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:40.272358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:32:40.272358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:40.272358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:32:40.272358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:32:40.272358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:40.272358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:32:40.272358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:40.272358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:32:40.272804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:40.272804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:32:40.272804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:40.272804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:32:40.272804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:32:40.272804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:32:40.272804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:32:40.272804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:32:40.272804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:32:40.272804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:40.272804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:40.273141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:40.273141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:32:40.273141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:40.273141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:40.273141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:40.273141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:32:40.273141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:40.273141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:32:40.273141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:32:40.273141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:40.273141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:32:40.273141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:40.273141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:32:40.273141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:32:40.273532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:40.273532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:32:40.273532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:40.273532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:32:40.273868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:40.273868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:32:40.273868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:32:40.273868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:32:40.273868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:32:40.273868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:32:40.273868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:40.273868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:32:40.273868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:32:40.273868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:32:40.273868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:32:40.273868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:32:40.273868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:40.273868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:40.273868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:32:40.273868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:32:40.273868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:32:40.274351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:32:40.274351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:32:40.274351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:32:40.274351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:40.274351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:40.274351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:40.274351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:32:40.274351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:40.274351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:40.274351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:40.274351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:32:40.274351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:40.274351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:32:40.274351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:32:40.274351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:40.274351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:32:40.274351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:40.274351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:32:40.274351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:32:40.274873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:40.274873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:32:40.274873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:40.274873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:32:40.274873 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:32:40.274873 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:32:40.274873 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:32:40.274873 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:32:40.275391 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:32:40.275954 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:32:40.276338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:40.277087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:32:40.277087 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:40.277087 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:32:40.277087 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:32:40.277087 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:40.277087 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:32:40.277087 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:40.277087 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:32:40.277903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:40.277903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:32:40.277903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:40.277903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:32:40.277903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:32:40.277903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:32:40.277903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:32:40.277903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:40.277903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:40.277903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:40.277903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:32:40.277903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:40.278320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:40.278320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:40.278320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:32:40.278320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:40.278320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:32:40.278320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:32:40.278320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:40.278320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:32:40.278320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:40.278320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:32:40.278320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:32:40.278320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:40.278320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:32:40.278682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:40.278682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:32:40.278935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:40.278935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:32:40.278935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:32:40.278935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:32:40.278935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:32:40.278935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:32:40.278935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:40.278935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:32:40.278935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:32:40.278935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:32:40.278935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:32:40.278935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:32:40.278935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:40.278935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:40.278935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:32:40.278935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:32:40.279393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:32:40.279393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:32:40.279393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:40.279393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:40.279393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:40.279393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:32:40.279393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:40.279393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:40.279393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:40.279393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:32:40.279393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:40.279393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:32:40.279393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:32:40.279393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:40.279393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:32:40.279393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:40.279393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:32:40.279393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:32:40.279393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:40.279939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:32:40.279939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:40.279939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:32:45.026347 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:32:45.027369 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:32:45.027369 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:32:45.028588 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:32:45.067135 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:32:45.067315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:45.067920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:32:45.067920 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:45.067920 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:32:45.067920 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:32:45.067920 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:45.067920 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:32:45.067920 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:45.067920 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:32:45.068979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:45.068979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:32:45.068979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:45.068979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:32:45.068979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:32:45.068979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:32:45.068979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:32:45.068979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:45.068979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:45.068979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:45.068979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:32:45.068979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:45.069633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:45.069633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:45.069633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:32:45.069633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:45.069633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:32:45.069633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:32:45.069633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:45.069633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:32:45.069633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:45.069633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:32:45.069633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:32:45.069633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:45.069633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:32:45.070215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:45.070215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:32:45.070215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:45.070215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:32:45.070215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:32:45.070215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:32:45.070215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:32:45.070215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:32:45.070215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:45.070215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:32:45.070215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:32:45.070215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:32:45.070215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:32:45.070215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:32:45.070215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:45.071001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:45.071001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:32:45.071001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:32:45.071001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:32:45.071001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:32:45.071001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:45.071001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:45.071001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:45.071001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:32:45.071001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:45.071001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:45.071001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:45.071001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:32:45.071001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:45.071001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:32:45.071001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:32:45.071001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:45.071001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:32:45.071001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:45.071895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:32:45.071895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:32:45.071895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:45.071895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:32:45.071895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:45.071895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:32:45.255905 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:32:45.263149 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:32:45.270908 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:32:45.271172 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:32:45.273143 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:32:45.273832 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:32:45.274089 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:32:45.274244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:45.274988 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:32:45.275445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:32:45.275445 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:45.275445 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:32:45.275445 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:32:45.275445 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:45.275445 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:32:45.275445 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:45.275445 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:32:45.276547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:45.276547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:32:45.276547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:45.276547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:32:45.276547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:32:45.276547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:32:45.276547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:32:45.276547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:45.276547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:45.276547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:45.276547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:32:45.276547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:45.277446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:45.277446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:45.277446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:32:45.277446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:45.277446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:32:45.277446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:32:45.277446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:45.277446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:32:45.277446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:45.277446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:32:45.277446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:32:45.277446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:45.277446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:32:45.277961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:45.277961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:32:45.277961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:45.277961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:32:45.277961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:32:45.277961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:32:45.277961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:32:45.277961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:32:45.277961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:45.277961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:32:45.277961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:32:45.277961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:32:45.277961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:32:45.277961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:32:45.277961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:45.278584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:45.278584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:32:45.278584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:32:45.278584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:32:45.278584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:32:45.278584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:45.278584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:45.278584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:45.278584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:32:45.278584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:45.278584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:45.278584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:45.278584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:32:45.278584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:45.278584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:32:45.278584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:32:45.278584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:45.278584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:32:45.278584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:45.279320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:32:45.279320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:32:45.279320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:45.279320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:32:45.279320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:45.279320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:32:45.279320 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:32:45.279320 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:32:45.279320 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:32:45.279320 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:32:45.280034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:45.280034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:32:45.280034 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:45.280034 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:32:45.280034 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:32:45.280034 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:45.280034 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:32:45.280034 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:45.280034 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:32:45.280034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:45.280034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:32:45.280034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:32:45.280034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:32:45.280034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:45.280034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:32:45.280034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:45.280034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:32:45.280656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:45.280656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:32:45.280656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:45.280656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:32:45.280656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:45.280656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:32:45.280656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:32:45.280656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:45.280656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:32:45.280656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:45.280656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:32:45.280656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:32:45.280656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:45.280656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:32:45.280656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:45.280656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:32:45.281089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:45.281089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:32:45.281089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:45.281089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:32:45.281089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:32:45.281089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:32:45.281089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:32:45.281089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:32:45.281089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:32:45.281089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:32:45.281089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:45.281089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:45.281573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:45.281573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:32:45.281573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:45.281573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:45.281573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:45.281573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:32:45.281573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:45.281573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:32:45.281573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:32:45.281573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:45.281573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:32:45.281573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:45.281573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:32:45.281573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:32:45.282330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:45.282330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:32:45.282330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:45.282330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:32:45.282330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:45.282330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:32:45.282330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:32:45.282330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:32:45.282330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:32:45.282330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:32:45.282330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:45.282330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:32:45.282330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:32:45.282330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:32:45.283221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:32:45.283221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:32:45.283221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:45.283221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:45.283221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:32:45.283221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:32:45.283221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:32:45.283221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:32:45.283221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:32:45.283221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:32:45.283221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:32:45.283221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:45.283221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:45.283221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:45.283221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:32:45.283221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:45.283221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:45.283221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:45.283221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:32:45.284440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:45.284440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:32:45.284440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:32:45.284440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:45.284440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:32:45.284440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:45.284440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:32:45.284440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:32:45.284440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:45.284440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:32:45.284440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:45.284440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:32:45.284440 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:32:45.284440 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:32:45.285225 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:32:45.285225 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:32:45.285225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:45.285225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:32:45.285225 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:45.285225 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:32:45.285225 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:32:45.285225 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:45.285225 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:32:45.285225 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:45.285225 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:32:45.285225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:45.286023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:32:45.286023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:32:45.286023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:32:45.286023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:45.286023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:32:45.286023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:45.286023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:32:45.286023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:45.286023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:32:45.286023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:45.286023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:32:45.286023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:45.286023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:32:45.286023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:32:45.286023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:45.286023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:32:45.286023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:45.286957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:32:45.286957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:32:45.286957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:45.286957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:32:45.286957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:45.286957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:32:45.286957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:45.286957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:32:45.286957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:45.286957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:32:45.286957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:32:45.286957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:32:45.286957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:32:45.287364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:32:45.287364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:32:45.287364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:45.287364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:45.287364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:45.287364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:32:45.287364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:45.287364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:45.287364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:45.287364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:32:45.287364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:45.287364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:32:45.287364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:32:45.287726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:45.287726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:32:45.287726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:45.287726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:32:45.287726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:32:45.287726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:45.287726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:32:45.287726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:45.287726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:32:45.287726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:45.287726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:32:45.287726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:32:45.288258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:32:45.288258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:32:45.288258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:32:45.288258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:45.288258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:32:45.288258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:32:45.288258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:32:45.288258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:32:45.288258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:32:45.288258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:45.288258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:45.288258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:32:45.288258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:32:45.288258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:32:45.288258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:32:45.288258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:32:45.288258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:32:45.288258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:45.288258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:45.288781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:45.288781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:32:45.288781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:45.288781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:45.288781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:45.288781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:32:45.288781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:45.288781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:32:45.288781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:32:45.288781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:45.288781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:32:45.288781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:45.288781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:32:45.288781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:32:45.288781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:45.288781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:32:45.288781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:45.288781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:32:45.289395 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:32:45.289395 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:32:45.289395 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:32:45.289395 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:32:45.289395 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:32:45.289395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:45.289792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:32:45.289792 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:45.289792 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:32:45.289792 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:32:45.289792 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:45.289792 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:32:45.289792 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:45.289792 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:32:45.289792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:45.289792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:32:45.289792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:45.289792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:32:45.289792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:32:45.289792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:32:45.289792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:32:45.290510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:45.290510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:45.290510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:45.290510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:32:45.290510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:45.290510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:45.290510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:45.290510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:32:45.290510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:45.290510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:32:45.290510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:32:45.290510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:45.290510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:32:45.290510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:45.290851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:32:45.290851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:32:45.290851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:45.290851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:32:45.290851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:45.290851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:32:45.290851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:45.290851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:32:45.290851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:32:45.290851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:32:45.290851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:32:45.290851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:32:45.290851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:45.290851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:32:45.291206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:32:45.291206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:32:45.291206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:32:45.291206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:32:45.291206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:45.291206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:45.291206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:32:45.291206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:32:45.291206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:32:45.291206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:32:45.291206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:45.291206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:45.291206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:45.291206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:32:45.291206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:45.291206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:45.291206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:45.291206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:32:45.291206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:45.291206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:32:45.291951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:32:45.291951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:45.291951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:32:45.291951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:45.291951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:32:45.291951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:32:45.291951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:45.291951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:32:45.291951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:45.291951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:32:50.027192 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:32:50.028282 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:32:50.028282 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:32:50.029703 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:32:50.061872 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:32:50.062293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:50.063057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:32:50.063057 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:50.063057 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:32:50.063057 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:32:50.063057 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:50.063057 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:32:50.063057 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:50.063057 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:32:50.063813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:50.063813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:32:50.063813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:50.063813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:32:50.063813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:32:50.063813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:32:50.063813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:32:50.063813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:50.063813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:50.063813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:50.063813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:32:50.063813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:50.064363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:50.064363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:50.064363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:32:50.064363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:50.064363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:32:50.064363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:32:50.064363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:50.064363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:32:50.064363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:50.064363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:32:50.064363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:32:50.064363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:50.064363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:32:50.064903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:50.064903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:32:50.064903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:50.064903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:32:50.064903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:32:50.064903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:32:50.064903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:32:50.064903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:32:50.064903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:50.064903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:32:50.064903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:32:50.064903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:32:50.064903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:32:50.064903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:32:50.064903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:50.065613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:50.065613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:32:50.065613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:32:50.065613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:32:50.065613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:32:50.065613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:50.065613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:50.065613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:50.065613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:32:50.065613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:50.065613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:50.065613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:50.065613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:32:50.065613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:50.065613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:32:50.065613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:32:50.065613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:50.065613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:32:50.065613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:50.066470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:32:50.066470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:32:50.066470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:50.066470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:32:50.066470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:50.066470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:32:50.269711 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:32:50.270177 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:32:50.270310 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:32:50.271509 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:32:50.271879 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:32:50.272148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:50.273036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:32:50.273036 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:50.273036 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:32:50.273036 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:32:50.273036 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:50.273036 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:32:50.273036 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:50.273036 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:32:50.274881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:50.274881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:32:50.274881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:50.274881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:32:50.274881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:32:50.274881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:32:50.274881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:32:50.274881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:50.274881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:50.274881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:50.274881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:32:50.274881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:50.275644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:50.275644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:50.275644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:32:50.275644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:50.275644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:32:50.275644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:32:50.275644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:50.275644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:32:50.275644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:50.275644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:32:50.275644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:32:50.275644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:50.275644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:32:50.276152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:50.276152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:32:50.276798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:50.276798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:32:50.276798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:32:50.276798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:32:50.276798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:32:50.276798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:32:50.276798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:50.276798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:32:50.276798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:32:50.276798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:32:50.276798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:32:50.276798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:32:50.276798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:50.276798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:50.276798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:32:50.276798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:32:50.277550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:32:50.277550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:32:50.277550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:50.277550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:50.277550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:50.277550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:32:50.277550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:50.277550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:50.277550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:50.277550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:32:50.277550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:50.277550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:32:50.277550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:32:50.277550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:50.277550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:32:50.277550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:50.277550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:32:50.277550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:32:50.277550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:50.278137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:32:50.278137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:50.278137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:32:50.278137 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:32:50.278289 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:32:50.278339 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:32:50.278462 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:32:50.278800 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:32:50.278983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:50.279492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:32:50.279492 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:50.279492 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:32:50.279492 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:32:50.279492 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:50.279492 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:32:50.279492 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:50.279492 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:32:50.280386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:50.280386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:32:50.280386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:32:50.280386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:32:50.280386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:50.280386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:32:50.280386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:50.280386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:32:50.280386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:50.280386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:32:50.280386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:50.280386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:32:50.280386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:50.280386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:32:50.280386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:32:50.281083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:50.281083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:32:50.281083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:50.281083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:32:50.281083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:32:50.281083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:50.281083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:32:50.281083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:50.281083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:32:50.281486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:50.281486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:32:50.281486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:50.281486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:32:50.281486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:32:50.281486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:32:50.281486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:32:50.281486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:32:50.281486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:32:50.281486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:32:50.281486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:50.281486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:50.281992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:50.281992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:32:50.281992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:50.281992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:50.281992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:50.281992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:32:50.281992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:50.281992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:32:50.281992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:32:50.281992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:50.281992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:32:50.281992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:50.281992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:32:50.281992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:32:50.282548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:50.282548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:32:50.282548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:50.282548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:32:50.282548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:50.282548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:32:50.282548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:32:50.282548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:32:50.282548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:32:50.282548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:32:50.282548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:50.282548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:32:50.282548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:32:50.282548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:32:50.282970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:32:50.282970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:32:50.282970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:50.282970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:50.282970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:32:50.282970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:32:50.282970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:32:50.282970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:32:50.282970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:32:50.282970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:32:50.282970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:32:50.282970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:50.282970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:50.282970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:50.282970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:32:50.282970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:50.282970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:50.282970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:50.282970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:32:50.283552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:50.283552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:32:50.283552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:32:50.283552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:50.283552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:32:50.283552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:50.283552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:32:50.283552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:32:50.283552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:50.283552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:32:50.283552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:50.283552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:32:50.283552 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:32:50.283552 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:32:50.284098 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:32:50.284098 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:32:50.284098 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:32:50.284098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:50.284266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:32:50.284266 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:50.284266 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:32:50.284266 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:32:50.284266 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:50.284266 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:32:50.284266 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:50.284266 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:32:50.284904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:50.284904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:32:50.284904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:32:50.284904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:32:50.284904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:50.284904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:32:50.284904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:50.284904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:32:50.284904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:50.284904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:32:50.284904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:50.284904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:32:50.284904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:50.284904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:32:50.284904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:32:50.285464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:50.285464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:32:50.285464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:50.285464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:32:50.285464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:32:50.285464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:50.285464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:32:50.285464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:50.285464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:32:50.285782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:50.285782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:32:50.285782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:50.285782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:32:50.285782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:32:50.285782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:32:50.285782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:32:50.285782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:32:50.285782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:32:50.285782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:50.285782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:50.286094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:50.286094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:32:50.286094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:50.286094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:50.286094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:50.286094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:32:50.286094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:50.286094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:32:50.286094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:32:50.286094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:50.286094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:32:50.286094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:50.286094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:32:50.286094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:32:50.286462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:50.286462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:32:50.286462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:50.286462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:32:50.287004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:50.287004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:32:50.287004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:32:50.287004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:32:50.287004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:32:50.287004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:32:50.287004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:50.287004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:32:50.287004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:32:50.287004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:32:50.287004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:32:50.287004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:32:50.287004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:50.287004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:50.287004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:32:50.287004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:32:50.287004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:32:50.287491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:32:50.287491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:32:50.287491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:32:50.287491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:50.287491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:50.287491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:50.287491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:32:50.287491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:50.287491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:50.287491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:50.287491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:32:50.287491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:50.287491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:32:50.287491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:32:50.287491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:50.287491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:32:50.287491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:50.287491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:32:50.287491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:32:50.287948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:50.287948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:32:50.287948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:50.287948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:32:50.287948 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:32:50.287948 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:32:50.287948 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:32:50.287948 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:32:50.288148 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:32:50.288364 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:32:50.288600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:50.289283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:32:50.289283 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:50.289283 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:32:50.289283 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:32:50.289283 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:50.289283 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:32:50.289283 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:50.289283 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:32:50.290074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:50.290074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:32:50.290074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:50.290074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:32:50.290074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:32:50.290074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:32:50.290074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:32:50.290074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:50.290074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:50.290074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:50.290074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:32:50.290074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:50.290347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:50.290347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:50.290347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:32:50.290347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:50.290347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:32:50.290347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:32:50.290347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:50.290347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:32:50.290347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:50.290347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:32:50.290347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:32:50.290347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:50.290347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:32:50.290737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:50.290737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:32:50.291149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:50.291149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:32:50.291149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:32:50.291149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:32:50.291149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:32:50.291149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:32:50.291149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:50.291149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:32:50.291149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:32:50.291149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:32:50.291149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:32:50.291149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:32:50.291149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:50.291149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:50.291149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:32:50.291149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:32:50.291494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:32:50.291494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:32:50.291494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:50.291494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:50.291494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:50.291494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:32:50.291494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:50.291494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:50.291494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:50.291494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:32:50.291494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:50.291494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:32:50.291494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:32:50.291494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:50.291494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:32:50.291494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:50.291494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:32:50.291494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:32:50.291494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:50.291853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:32:50.291853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:50.291853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:32:55.028401 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:32:55.029179 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:32:55.029338 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:32:55.030284 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:32:55.064034 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:32:55.064560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:55.065524 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:32:55.065524 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:55.065524 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:32:55.065524 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:32:55.065524 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:55.065524 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:32:55.065524 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:55.065524 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:32:55.066739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:55.066739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:32:55.066739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:55.066739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:32:55.066739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:32:55.066739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:32:55.066739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:32:55.066739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:55.066739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:55.066739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:55.066739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:32:55.066739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:55.067131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:55.067131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:55.067131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:32:55.067131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:55.067131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:32:55.067131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:32:55.067131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:55.067131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:32:55.067131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:55.067131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:32:55.067131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:32:55.067131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:55.067131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:32:55.067490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:55.067490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:32:55.068042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:55.068042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:32:55.068042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:32:55.068042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:32:55.068042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:32:55.068042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:32:55.068042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:55.068042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:32:55.068042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:32:55.068042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:32:55.068042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:32:55.068042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:32:55.068042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:55.068042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:55.068042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:32:55.068042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:32:55.068511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:32:55.068511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:32:55.068511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:55.068511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:55.068511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:55.068511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:32:55.068511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:55.068511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:55.068511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:55.068511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:32:55.068511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:55.068511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:32:55.068511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:32:55.068511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:55.068511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:32:55.068511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:55.068511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:32:55.068511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:32:55.068511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:55.069046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:32:55.069046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:55.069046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:32:55.270702 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:32:55.271328 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:32:55.271608 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:32:55.272913 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:32:55.273259 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:32:55.273532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:55.274429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:32:55.274429 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:55.274429 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:32:55.274429 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:32:55.274429 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:55.274429 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:32:55.274429 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:55.274429 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:32:55.276129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:55.276129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:32:55.276129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:55.276129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:32:55.276129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:32:55.276129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:32:55.276129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:32:55.276129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:55.276129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:55.276129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:55.276129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:32:55.276129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:55.276890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:55.276890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:55.276890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:32:55.276890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:55.276890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:32:55.276890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:32:55.276890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:55.276890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:32:55.276890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:55.276890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:32:55.276890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:32:55.276890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:55.276890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:32:55.277542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:55.277542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:32:55.278384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:55.278384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:32:55.278384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:32:55.278384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:32:55.278384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:32:55.278384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:32:55.278384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:55.278384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:32:55.278384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:32:55.278384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:32:55.278384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:32:55.278384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:32:55.278384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:55.278384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:55.278384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:32:55.278384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:32:55.280315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:32:55.280315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:32:55.280315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:55.280315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:55.280315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:55.280315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:32:55.280315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:55.280315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:55.280315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:55.280315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:32:55.280315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:55.280315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:32:55.280315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:32:55.280315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:55.280315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:32:55.280315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:55.280315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:32:55.280315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:32:55.280315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:55.281189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:32:55.281189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:55.281189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:32:55.281189 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:32:55.281189 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:32:55.281189 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:32:55.281189 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:32:55.281189 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:32:55.281638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:55.281836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:32:55.281836 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:55.281836 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:32:55.281836 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:32:55.281836 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:55.281836 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:32:55.281836 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:55.281836 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:32:55.282772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:55.282772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:32:55.282772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:32:55.282772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:32:55.282772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:55.282772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:32:55.282772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:55.282772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:32:55.282772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:55.282772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:32:55.282772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:55.282772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:32:55.282772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:55.282772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:32:55.282772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:32:55.283496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:55.283496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:32:55.283496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:55.283496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:32:55.283496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:32:55.283496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:55.283496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:32:55.283496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:55.283496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:32:55.284063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:55.284063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:32:55.284063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:55.284063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:32:55.284063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:32:55.284063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:32:55.284063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:32:55.284063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:32:55.284063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:32:55.284063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:32:55.284063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:55.284063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:55.284707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:55.284707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:32:55.284707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:55.284707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:55.284707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:55.284707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:32:55.284707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:55.284707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:32:55.284707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:32:55.284707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:55.284707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:32:55.284707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:55.284707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:32:55.284707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:32:55.285335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:55.285335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:32:55.285335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:55.285335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:32:55.285335 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:32:55.285743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:55.285743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:32:55.285743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:32:55.285743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:32:55.285743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:32:55.285743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:32:55.285743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:55.285743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:32:55.285743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:32:55.285743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:32:55.285743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:32:55.285743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:32:55.285743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:55.285743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:55.285743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:32:55.285743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:32:55.286469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:32:55.286469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:32:55.286469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:32:55.286469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:32:55.286469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:32:55.286469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:55.286469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:55.286469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:55.286469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:32:55.286469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:55.286469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:55.286469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:55.286469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:32:55.286469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:55.286469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:32:55.286469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:32:55.286469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:55.286469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:32:55.287248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:55.287248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:32:55.287248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:32:55.287248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:55.287248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:32:55.287248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:55.287248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:32:55.287248 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:32:55.287248 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:32:55.287248 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:32:55.287747 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:32:55.287747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:55.287747 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:32:55.288191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:32:55.288191 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:55.288191 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:32:55.288191 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:32:55.288191 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:55.288191 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:32:55.288191 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:55.288191 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:32:55.289066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:55.289066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:32:55.289066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:32:55.289066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:32:55.289066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:55.289066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:32:55.289066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:55.289066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:32:55.289066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:55.289066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:32:55.289066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:55.289066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:32:55.289066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:55.289066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:32:55.289066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:32:55.289470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:55.289470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:32:55.289470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:55.289470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:32:55.289470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:32:55.289470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:55.289470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:32:55.289470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:55.289470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:32:55.289855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:55.289855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:32:55.289855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:55.289855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:32:55.289855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:32:55.289855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:32:55.289855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:32:55.289855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:32:55.289855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:32:55.289855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:55.289855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:55.290100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:55.290100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:32:55.290100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:55.290100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:55.290100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:55.290100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:32:55.290100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:55.290100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:32:55.290100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:32:55.290100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:55.290100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:32:55.290100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:55.290100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:32:55.290100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:32:55.290399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:55.290399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:32:55.290399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:55.290399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:32:55.290807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:55.290807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:32:55.290807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:32:55.290807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:32:55.290807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:32:55.290807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:32:55.290807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:55.290807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:32:55.290807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:32:55.290807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:32:55.290807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:32:55.290807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:32:55.290807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:55.290807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:55.290807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:32:55.290807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:32:55.290807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:32:55.291163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:32:55.291163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:32:55.291163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:32:55.291163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:55.291163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:55.291163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:55.291163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:32:55.291163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:55.291163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:55.291163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:55.291163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:32:55.291163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:55.291163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:32:55.291163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:32:55.291163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:55.291163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:32:55.291163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:55.291163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:32:55.291163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:32:55.291550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:55.291550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:32:55.291550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:55.291550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:32:55.291550 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:32:55.291550 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:32:55.291550 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:32:55.291719 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:32:55.291998 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:32:55.292216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:55.292937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:32:55.292937 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:55.292937 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:32:55.292937 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:32:55.292937 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:55.292937 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:32:55.292937 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:55.292937 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:32:55.293644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:55.293644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:32:55.293644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:55.293644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:32:55.293644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:32:55.293644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:32:55.293644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:32:55.293644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:55.293644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:55.293644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:55.293644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:32:55.293644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:55.293905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:32:55.293905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:55.293905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:32:55.293905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:55.293905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:32:55.293905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:32:55.293905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:55.293905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:32:55.293905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:55.293905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:32:55.293905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:32:55.293905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:55.293905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:32:55.294161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:55.294161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:32:55.294519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:55.294519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:32:55.294519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:32:55.294519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:32:55.294519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:32:55.294519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:32:55.294519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:55.294519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:32:55.294519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:32:55.294519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:32:55.294519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:32:55.294519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:32:55.294519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:32:55.294519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:32:55.294519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:32:55.294519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:32:55.294820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:32:55.294820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:32:55.294820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:55.294820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:55.294820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:32:55.294820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:32:55.294820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:32:55.294820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:32:55.294820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:32:55.294820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:32:55.294820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:32:55.294820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:32:55.294820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:32:55.294820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:32:55.294820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:32:55.294820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:32:55.294820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:32:55.294820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:32:55.294820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:32:55.295155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:32:55.295155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:32:55.295155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:33:00.029552 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:33:00.030498 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:33:00.031017 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:33:00.032005 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:33:00.070468 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:33:00.070968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:00.071856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:33:00.071856 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:00.071856 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:33:00.071856 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:33:00.071856 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:00.071856 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:33:00.071856 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:00.071856 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:33:00.072951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:00.072951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:33:00.072951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:00.072951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:33:00.072951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:33:00.072951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:33:00.072951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:33:00.072951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:00.072951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:00.072951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:00.072951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:33:00.072951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:00.073622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:00.073622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:00.073622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:33:00.073622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:00.073622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:33:00.073622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:33:00.073622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:00.073622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:33:00.073622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:00.073622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:33:00.073622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:33:00.073622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:00.073622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:33:00.074245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:00.074245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:33:00.074369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:00.074369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:33:00.074369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:33:00.074369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:33:00.074369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:33:00.074369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:33:00.074369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:00.074369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:33:00.074369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:33:00.074369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:33:00.074369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:33:00.074369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:33:00.074369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:00.074369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:00.074369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:33:00.074369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:33:00.075131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:33:00.075131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:33:00.075131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:00.075131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:00.075131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:00.075131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:33:00.075131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:00.075131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:00.075131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:00.075131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:33:00.075131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:00.075131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:33:00.075131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:33:00.075131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:00.075131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:33:00.075131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:00.075131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:33:00.075131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:33:00.075131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:00.075714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:33:00.075714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:00.075714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:33:00.271683 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:33:00.272446 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:33:00.272669 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:33:00.274892 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:33:00.275459 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:33:00.275915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:00.276794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:33:00.276794 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:00.276794 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:33:00.276794 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:33:00.276794 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:00.276794 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:33:00.276794 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:00.276794 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:33:00.278326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:00.278326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:33:00.278326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:00.278326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:33:00.278326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:33:00.278326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:33:00.278326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:33:00.278326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:00.278326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:00.278326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:00.278326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:33:00.278326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:00.279980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:00.279980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:00.279980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:33:00.279980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:00.279980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:33:00.279980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:33:00.279980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:00.279980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:33:00.279980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:00.279980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:33:00.279980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:33:00.279980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:00.279980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:33:00.280877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:00.280877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:33:00.280877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:00.280877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:33:00.280877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:33:00.280877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:33:00.280877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:33:00.280877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:33:00.280877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:00.280877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:33:00.280877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:33:00.280877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:33:00.280877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:33:00.280877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:33:00.280877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:00.281872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:00.281872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:33:00.281872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:33:00.281872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:33:00.281872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:33:00.281872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:00.281872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:00.281872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:00.281872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:33:00.281872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:00.281872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:00.281872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:00.281872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:33:00.281872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:00.281872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:33:00.281872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:33:00.281872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:00.281872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:33:00.281872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:00.283230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:33:00.283230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:33:00.283230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:00.283230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:33:00.283230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:00.283230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:33:00.283230 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:33:00.283230 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:33:00.283230 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:33:00.283230 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:33:00.283560 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:33:00.283560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:00.283560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:33:00.283560 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:00.283560 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:33:00.283560 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:33:00.283560 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:00.283560 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:33:00.283560 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:00.283560 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:33:00.283560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:00.283560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:33:00.283560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:33:00.283560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:33:00.284164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:00.284164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:33:00.284164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:00.284164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:33:00.284164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:00.284164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:33:00.284164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:00.284164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:33:00.284164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:00.284164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:33:00.284164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:33:00.284164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:00.284164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:33:00.284164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:00.284164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:33:00.284164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:33:00.284164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:00.284164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:33:00.284889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:00.284889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:33:00.284889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:00.284889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:33:00.284889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:00.284889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:33:00.284889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:33:00.284889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:33:00.284889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:33:00.284889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:33:00.284889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:33:00.284889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:33:00.285379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:00.285379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:00.285379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:00.285379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:33:00.285379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:00.285379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:00.285379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:00.285379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:33:00.285379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:00.285379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:33:00.285379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:33:00.285379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:00.285379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:33:00.285379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:00.285920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:33:00.285920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:33:00.285920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:00.285920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:33:00.285920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:00.285920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:33:00.285920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:00.285920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:33:00.285920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:33:00.285920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:33:00.285920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:33:00.285920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:33:00.285920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:00.285920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:33:00.286358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:33:00.286358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:33:00.286358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:33:00.286358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:33:00.286358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:00.286358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:00.286358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:33:00.286358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:33:00.286358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:33:00.286358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:33:00.286358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:33:00.286358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:33:00.286358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:33:00.286358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:00.286358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:00.286358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:00.286358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:33:00.286358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:00.286358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:00.287667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:00.287667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:33:00.287667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:00.287667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:33:00.287667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:33:00.287667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:00.287667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:33:00.287667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:00.287667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:33:00.287667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:33:00.287667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:00.287667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:33:00.287667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:00.287667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:33:00.287667 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:33:00.288125 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:33:00.288125 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:33:00.288125 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:33:00.288125 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:33:00.288125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:00.288294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:33:00.288294 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:00.288294 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:33:00.288294 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:33:00.288294 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:00.288294 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:33:00.288294 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:00.288294 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:33:00.288960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:00.288960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:33:00.288960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:33:00.288960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:33:00.288960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:00.288960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:33:00.288960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:00.288960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:33:00.288960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:00.288960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:33:00.288960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:00.288960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:33:00.288960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:00.288960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:33:00.288960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:33:00.289373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:00.289373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:33:00.289373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:00.289373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:33:00.289373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:33:00.289373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:00.289373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:33:00.289373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:00.289373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:33:00.289846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:00.289846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:33:00.289846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:00.289846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:33:00.289846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:33:00.289846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:33:00.289846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:33:00.289846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:33:00.289846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:33:00.289846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:00.289846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:00.290155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:00.290155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:33:00.290155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:00.290155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:00.290155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:00.290155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:33:00.290155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:00.290155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:33:00.290155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:33:00.290155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:00.290155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:33:00.290155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:00.290155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:33:00.290155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:33:00.290620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:00.290620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:33:00.290620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:00.290620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:33:00.290970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:00.290970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:33:00.290970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:33:00.290970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:33:00.290970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:33:00.290970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:33:00.290970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:00.290970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:33:00.290970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:33:00.290970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:33:00.290970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:33:00.290970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:33:00.290970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:00.290970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:00.290970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:33:00.290970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:33:00.290970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:33:00.291369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:33:00.291369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:33:00.291369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:33:00.291369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:00.291369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:00.291369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:00.291369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:33:00.291369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:00.291369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:00.291369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:00.291369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:33:00.291369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:00.291369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:33:00.291369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:33:00.291369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:00.291369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:33:00.291369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:00.291369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:33:00.291369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:33:00.291796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:00.291796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:33:00.291796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:00.291796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:33:00.291796 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:33:00.291796 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:33:00.292003 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:33:00.292003 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:33:00.292413 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:33:00.292832 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:33:00.293018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:00.293715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:33:00.293715 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:00.293715 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:33:00.293715 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:33:00.293715 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:00.293715 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:33:00.293715 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:00.293715 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:33:00.294569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:00.294569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:33:00.294569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:00.294569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:33:00.294569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:33:00.294569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:33:00.294569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:33:00.294569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:00.294569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:00.294569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:00.294569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:33:00.294569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:00.294933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:00.294933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:00.294933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:33:00.294933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:00.294933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:33:00.294933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:33:00.294933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:00.294933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:33:00.294933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:00.294933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:33:00.294933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:33:00.294933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:00.294933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:33:00.295362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:00.295362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:33:00.295558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:00.295558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:33:00.295558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:33:00.295558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:33:00.295558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:33:00.295558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:33:00.295558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:00.295558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:33:00.295558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:33:00.295558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:33:00.295558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:33:00.295558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:33:00.295558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:00.295558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:00.295558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:33:00.295558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:33:00.296010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:33:00.296010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:33:00.296010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:00.296010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:00.296010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:00.296010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:33:00.296010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:00.296010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:00.296010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:00.296010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:33:00.296010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:00.296010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:33:00.296010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:33:00.296010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:00.296010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:33:00.296010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:00.296010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:33:00.296010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:33:00.296010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:00.296537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:33:00.296537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:00.296537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:33:05.030673 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:33:05.031573 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:33:05.031775 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:33:05.033103 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:33:05.060443 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:33:05.060732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:05.061390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:33:05.061390 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:05.061390 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:33:05.061390 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:33:05.061390 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:05.061390 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:33:05.061390 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:05.061390 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:33:05.062136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.030 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:05.062136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:33:05.062136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:05.062136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:33:05.062136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:33:05.062136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:33:05.062136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:33:05.062136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:05.062136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:05.062136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:05.062136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:33:05.062136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:05.062803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:05.062803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:05.062803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:33:05.062803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:05.062803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:33:05.062803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:33:05.062803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:05.062803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:33:05.062803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:05.062803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:33:05.062803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:33:05.062803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:05.062803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:33:05.063476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:05.063476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:33:05.063476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:05.063476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:33:05.063476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:33:05.063476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:33:05.063476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:33:05.063476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:33:05.063476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:05.063476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:33:05.063476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:33:05.063476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:33:05.063476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:33:05.063476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:33:05.063476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:05.064303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:05.064303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:33:05.064303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:33:05.064303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:33:05.064303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:33:05.064303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:05.064303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:05.064303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:05.064303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:33:05.064303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:05.064303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:05.064303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:05.064303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:33:05.064303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:05.064303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:33:05.064303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:33:05.064303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:05.064303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:33:05.064303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:05.065499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:33:05.065499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:33:05.065499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:05.065499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:33:05.065499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:05.065499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:33:05.272584 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:33:05.273288 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:33:05.273563 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:33:05.275158 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:33:05.275495 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:33:05.275741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:05.276672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:33:05.276672 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:05.276672 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:33:05.276672 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:33:05.276672 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:05.276672 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:33:05.276672 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:05.276672 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:33:05.278542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:05.278542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:33:05.278542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:05.278542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:33:05.278542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:33:05.278542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:33:05.278542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:33:05.278542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:05.278542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:05.278542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:05.278542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:33:05.278542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:05.279491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:05.279491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:05.279491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:33:05.279491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:05.279491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:33:05.279491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:33:05.279491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:05.279491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:33:05.279491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:05.279491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:33:05.279491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:33:05.279491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:05.279491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:33:05.280454 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:05.280454 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:33:05.281159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:05.281159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:33:05.281159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:33:05.281159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:33:05.281159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:33:05.281159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:33:05.281159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:05.281159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:33:05.281159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:33:05.281159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:33:05.281159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:33:05.281159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:33:05.281159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:05.281159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:05.281159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:33:05.281159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:33:05.282035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:33:05.282035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:33:05.282035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:05.282035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:05.282035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:05.282035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:33:05.282035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:05.282035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:05.282035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:05.282035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:33:05.282035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:05.282035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:33:05.282035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:33:05.282035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:05.282035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:33:05.282035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:05.282035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:33:05.282035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:33:05.282035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:05.282618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:33:05.282618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:05.282618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:33:05.282618 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:33:05.282618 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:33:05.282618 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:33:05.282618 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:33:05.282618 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:33:05.282892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:05.282926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:33:05.282926 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:05.282926 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:33:05.282926 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:33:05.282926 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:05.282926 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:33:05.282926 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:05.282926 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:33:05.284186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:05.284186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:33:05.284186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:33:05.284186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:33:05.284186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:05.284186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:33:05.284186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:05.284186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:33:05.284186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:05.284186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:33:05.284186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:05.284186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:33:05.284186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:05.284186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:33:05.284186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:33:05.284713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:05.284713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:33:05.284713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:05.284713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:33:05.284713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:33:05.284713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:05.284713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:33:05.284713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:05.284713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:33:05.285836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:05.285836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:33:05.285836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:05.285836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:33:05.285836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:33:05.285836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:33:05.285836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:33:05.285836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:33:05.285836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:33:05.285836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:33:05.285836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:05.285836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:05.286215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:05.286215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:33:05.286215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:05.286215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:05.286215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:05.286215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:33:05.286215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:05.286215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:33:05.286215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:33:05.286215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:05.286215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:33:05.286215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:05.286215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:33:05.286215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:33:05.286677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:05.286677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:33:05.286677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:05.286677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:33:05.287727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:05.287727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:33:05.287727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:33:05.287727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:33:05.287727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:33:05.287727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:33:05.287727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:05.287727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:33:05.287727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:33:05.287727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:33:05.287727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:33:05.287727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:33:05.287727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:05.287727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:05.287727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:33:05.287727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:33:05.288158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:33:05.288158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:33:05.288158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:33:05.288158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:33:05.288158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:33:05.288158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:05.288158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:05.288158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:05.288158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:33:05.288158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:05.288158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:05.288158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:05.288158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:33:05.288158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:05.288158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:33:05.288158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:33:05.288158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:05.288158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:33:05.288705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:05.288705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:33:05.288705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:33:05.288705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:05.288705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:33:05.288705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:05.288705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:33:05.288705 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:33:05.288705 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:33:05.288705 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:33:05.289012 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:33:05.289223 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:33:05.289455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:05.289995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:33:05.289995 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:05.289995 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:33:05.289995 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:33:05.289995 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:05.289995 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:33:05.289995 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:05.289995 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:33:05.290782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:05.290782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:33:05.290782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:33:05.290782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:33:05.290782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:05.290782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:33:05.290782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:05.290782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:33:05.290782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:05.290782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:33:05.290782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:05.290782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:33:05.290782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:05.290782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:33:05.290782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:33:05.291193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:05.291193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:33:05.291193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:05.291193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:33:05.291193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:33:05.291193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:05.291193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:33:05.291193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:05.291193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:33:05.291625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:05.291625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:33:05.291625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:05.291625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:33:05.291625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:33:05.291625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:33:05.291625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:33:05.291625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:33:05.291625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:33:05.291625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:05.291625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:05.291949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:05.291949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:33:05.291949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:05.291949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:05.291949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:05.291949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:33:05.291949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:05.291949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:33:05.291949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:33:05.291949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:05.291949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:33:05.291949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:05.291949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:33:05.291949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:33:05.292324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:05.292324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:33:05.292324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:05.292324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:33:05.292880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:05.292880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:33:05.292880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:33:05.292880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:33:05.292880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:33:05.292880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:33:05.292880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:05.292880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:33:05.292880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:33:05.292880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:33:05.292880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:33:05.292880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:33:05.292880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:05.292880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:05.292880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:33:05.292880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:33:05.292880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:33:05.293391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:33:05.293391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:33:05.293391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:33:05.293391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:05.293391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:05.293391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:05.293391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:33:05.293391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:05.293391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:05.293391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:05.293391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:33:05.293391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:05.293391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:33:05.293391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:33:05.293391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:05.293391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:33:05.293391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:05.293391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:33:05.293391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:33:05.293929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:05.293929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:33:05.293929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:05.293929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:33:05.293929 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:33:05.293929 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:33:05.293929 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:33:05.293929 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:33:05.294173 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:33:05.294432 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:33:05.294664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:05.295748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:33:05.295748 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:05.295748 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:33:05.295748 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:33:05.295748 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:05.295748 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:33:05.295748 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:05.295748 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:33:05.296617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:05.296617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:33:05.296617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:05.296617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:33:05.296617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:33:05.296617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:33:05.296617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:33:05.296617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:05.296617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:05.296617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:05.296617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:33:05.296617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:05.296971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:05.296971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:05.296971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:33:05.296971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:05.296971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:33:05.296971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:33:05.296971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:05.296971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:33:05.296971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:05.296971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:33:05.296971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:33:05.296971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:05.296971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:33:05.297363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:05.297363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:33:05.297879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:05.297879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:33:05.297879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:33:05.297879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:33:05.297879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:33:05.297879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:33:05.297879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:05.297879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:33:05.297879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:33:05.297879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:33:05.297879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:33:05.297879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:33:05.297879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:05.297879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:05.297879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:33:05.297879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:33:05.298331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:33:05.298331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:33:05.298331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:05.298331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:05.298331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:05.298331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:33:05.298331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:05.298331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:05.298331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:05.298331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:33:05.298331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:05.298331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:33:05.298331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:33:05.298331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:05.298331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:33:05.298331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:05.298331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:33:05.298331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:33:05.298331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:05.298881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:33:05.298881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:05.298881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:33:10.031689 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:33:10.034595 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:33:10.034595 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:33:10.034595 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:33:10.064883 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:33:10.065270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:10.067833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:33:10.067833 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:10.067833 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:33:10.067833 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:33:10.067833 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:10.067833 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:33:10.067833 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:10.067833 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:33:10.067833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:10.067833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:33:10.067833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:10.067833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:33:10.067833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:33:10.067833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:33:10.067833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:33:10.068211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:10.068211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:10.068211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:10.068211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:33:10.068211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:10.068211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:10.068211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:10.068211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:33:10.068211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:10.068211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:33:10.068211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:33:10.068211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:10.068211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:33:10.068211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:10.068538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:33:10.068538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:33:10.068538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:10.068538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:33:10.068538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:10.068538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:33:10.068538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:10.068538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:33:10.068538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:33:10.068538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:33:10.068538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:33:10.068538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:33:10.068538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:10.068538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:33:10.068862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:33:10.068862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:33:10.068862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:33:10.068862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:33:10.068862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:10.068862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:10.068862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:33:10.068862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:33:10.068862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:33:10.068862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:33:10.068862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:10.068862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:10.068862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:10.068862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:33:10.068862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:10.068862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:10.068862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:10.068862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:33:10.068862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:10.068862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:33:10.068862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:33:10.069356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:10.069356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:33:10.069356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:10.069356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:33:10.069356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:33:10.069356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:10.069356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:33:10.069356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:10.069356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:33:10.273437 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:33:10.274570 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:33:10.274570 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:33:10.275741 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:33:10.276092 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:33:10.276376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:10.277323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:33:10.277323 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:10.277323 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:33:10.277323 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:33:10.277323 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:10.277323 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:33:10.277323 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:10.277323 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:33:10.278858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:10.278858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:33:10.278858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:10.278858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:33:10.278858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:33:10.278858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:33:10.278858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:33:10.278858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:10.278858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:10.278858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:10.278858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:33:10.278858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:10.279474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:10.279474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:10.279474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:33:10.279474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:10.279474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:33:10.279474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:33:10.279474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:10.279474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:33:10.279474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:10.279474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:33:10.279474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:33:10.279474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:10.279474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:33:10.279997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:10.279997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:33:10.280487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:10.280487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:33:10.280487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:33:10.280487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:33:10.280487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:33:10.280487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:33:10.280487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:10.280487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:33:10.280487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:33:10.280487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:33:10.280487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:33:10.280487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:33:10.280487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:10.280487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:10.280487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:33:10.280487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:33:10.281269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:33:10.281269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:33:10.281269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:10.281269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:10.281269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:10.281269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:33:10.281269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:10.281269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:10.281269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:10.281269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:33:10.281269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:10.281269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:33:10.281269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:33:10.281269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:10.281269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:33:10.281269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:10.281269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:33:10.281269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:33:10.281269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:10.282356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:33:10.282356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:10.282356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:33:10.282356 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:33:10.282356 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:33:10.282356 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:33:10.282356 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:33:10.282820 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:33:10.283245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:10.283397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:33:10.283397 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:10.283397 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:33:10.283397 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:33:10.283397 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:10.283397 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:33:10.283397 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:10.283397 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:33:10.284251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:10.284251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:33:10.284251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:33:10.284251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:33:10.284251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:10.284251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:33:10.284251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:10.284251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:33:10.284251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:10.284251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:33:10.284251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:10.284251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:33:10.284251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:10.284251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:33:10.284251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:33:10.285650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:10.285650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:33:10.285650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:10.285650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:33:10.285650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:33:10.285650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:10.285650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:33:10.285650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:10.285650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:33:10.285650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:10.285650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:33:10.285650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:10.285650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:33:10.285650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:33:10.286144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:33:10.286144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:33:10.286144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:33:10.286144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:33:10.286144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:33:10.286144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:10.286144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:10.286144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:10.286144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:33:10.286144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:10.286144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:10.286144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:10.286144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:33:10.286529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:10.286529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:33:10.286529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:33:10.286529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:10.286529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:33:10.286529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:10.286529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:33:10.286529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:33:10.286529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:10.286529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:33:10.286529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:10.286529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:33:10.286821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:10.286821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:33:10.286821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:33:10.286821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:33:10.286821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:33:10.286821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:33:10.286821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:10.286821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:33:10.286821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:33:10.286821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:33:10.286821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:33:10.286821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:33:10.286821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:10.286821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:10.286821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:33:10.286821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:33:10.287359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:33:10.287359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:33:10.287359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:33:10.287359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:33:10.287359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:33:10.287359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:10.287359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:10.287359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:10.287359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:33:10.287359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:10.287359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:10.287359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:10.287359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:33:10.287359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:10.287359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:33:10.287359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:33:10.287359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:10.287359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:33:10.287827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:10.287827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:33:10.287827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:33:10.287827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:10.287827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:33:10.287827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:10.287827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:33:10.288419 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:33:10.288720 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:33:10.288843 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:33:10.289057 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:33:10.289510 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:33:10.289903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:10.290617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:33:10.290617 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:10.290617 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:33:10.290617 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:33:10.290617 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:10.290617 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:33:10.290617 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:10.290617 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:33:10.291372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:10.291372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:33:10.291372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:33:10.291372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:33:10.291372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:10.291372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:33:10.291372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:10.291372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:33:10.291372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:10.291372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:33:10.291372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:10.291372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:33:10.291372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:10.291372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:33:10.291372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:33:10.291942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:10.291942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:33:10.291942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:10.291942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:33:10.291942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:33:10.291942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:10.291942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:33:10.291942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:10.291942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:33:10.292325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:10.292325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:33:10.292325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:10.292325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:33:10.292325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:33:10.292325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:33:10.292325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:33:10.292325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:33:10.292325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:33:10.292325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:10.292325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:10.293166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:10.293166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:33:10.293166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:10.293166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:10.293166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:10.293166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:33:10.293166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:10.293166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:33:10.293166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:33:10.293166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:10.293166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:33:10.293166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:10.293166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:33:10.293166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:33:10.293753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:10.293753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:33:10.293753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:10.293753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:33:10.293753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:10.293753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:33:10.293753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:33:10.293753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:33:10.293753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:33:10.293753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:33:10.293753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:10.293753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:33:10.293753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:33:10.293753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:33:10.294339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:33:10.294339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:33:10.294339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:10.294339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:10.294339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:33:10.294339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:33:10.294339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:33:10.294339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:33:10.294339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:33:10.294339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:33:10.294339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:10.294339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:10.294339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:10.294339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:33:10.294339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:10.294339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:10.294339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:10.294339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:33:10.294339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:10.295092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:33:10.295092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:33:10.295092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:10.295092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:33:10.295092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:10.295092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:33:10.295092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:33:10.295092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:10.295092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:33:10.295092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:10.295092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:33:10.295092 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:33:10.295092 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:33:10.295668 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:33:10.295668 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:33:10.295668 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:33:10.295668 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:33:10.295668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:10.296182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:33:10.296182 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:10.296182 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:33:10.296182 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:33:10.296182 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:10.296182 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:33:10.296182 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:10.296182 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:33:10.297131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:10.297131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:33:10.297131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:10.297131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:33:10.297131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:33:10.297131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:33:10.297131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:33:10.297131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:10.297131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:10.297131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:10.297131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:33:10.297131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:10.297768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:10.297768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:10.297768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:33:10.297768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:10.297768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:33:10.297768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:33:10.297768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:10.297768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:33:10.297768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:10.297768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:33:10.297768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:33:10.297768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:10.297768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:33:10.298213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:10.298213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:33:10.298213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:10.298213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:33:10.298213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:33:10.298213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:33:10.298213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:33:10.298213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:33:10.298213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:10.298213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:33:10.298213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:33:10.298213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:33:10.298213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:33:10.298213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:33:10.298213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:10.298828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:10.298828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:33:10.298828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:33:10.298828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:33:10.298828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:33:10.298828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:10.298828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:10.298828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:10.298828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:33:10.298828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:10.298828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:10.298828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:10.298828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:33:10.298828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:10.298828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:33:10.298828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:33:10.298828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:10.298828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:33:10.298828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:10.299572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:33:10.299572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:33:10.299572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:10.299572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:33:10.299572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:10.299572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:33:15.033246 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:33:15.033813 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:33:15.033924 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:33:15.035434 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:33:15.071912 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:33:15.072157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:15.072772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:33:15.072772 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:15.072772 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:33:15.072772 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:33:15.072772 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:15.072772 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:33:15.072772 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:15.072772 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:33:15.073935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:15.073935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:33:15.073935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:15.073935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:33:15.073935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:33:15.073935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:33:15.073935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:33:15.073935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:15.073935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:15.073935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:15.073935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:33:15.073935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:15.074272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:15.074272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:15.074272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:33:15.074272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:15.074272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:33:15.074272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:33:15.074272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:15.074272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:33:15.074272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:15.074272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:33:15.074272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:33:15.074272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:15.074272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:33:15.074608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:15.074608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:33:15.075011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:15.075011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:33:15.075011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:33:15.075011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:33:15.075011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:33:15.075011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:33:15.075011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:15.075011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:33:15.075011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:33:15.075011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:33:15.075011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:33:15.075011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:33:15.075011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:15.075011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:15.075011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:33:15.075011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:33:15.075373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:33:15.075373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:33:15.075373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:15.075373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:15.075373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:15.075373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:33:15.075373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:15.075373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:15.075373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:15.075373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:33:15.075373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:15.075373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:33:15.075373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:33:15.075373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:15.075373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:33:15.075373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:15.075373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:33:15.075373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:33:15.075373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:15.075807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:33:15.075807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:15.075807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:33:15.274396 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:33:15.274613 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:33:15.274931 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:33:15.276178 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:33:15.276359 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:33:15.277749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:15.277749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:33:15.277749 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:15.277749 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:33:15.277749 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:33:15.277749 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:15.277749 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:33:15.277749 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:15.277749 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:33:15.279198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:15.279198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:33:15.279198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:15.279198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:33:15.279198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:33:15.279198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:33:15.279198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:33:15.279198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:15.279198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:15.279198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:15.279198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:33:15.279198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:15.280049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:15.280049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:15.280049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:33:15.280049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:15.280049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:33:15.280049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:33:15.280049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:15.280049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:33:15.280049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:15.280049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:33:15.280049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:33:15.280049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:15.280049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:33:15.280975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:15.280975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:33:15.280975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:15.280975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:33:15.280975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:33:15.280975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:33:15.280975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:33:15.280975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:33:15.280975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:15.280975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:33:15.280975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:33:15.280975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:33:15.280975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:33:15.280975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:33:15.280975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:15.281741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:15.281741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:33:15.281741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:33:15.281741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:33:15.281741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:33:15.281741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:15.281741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:15.281741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:15.281741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:33:15.281741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:15.281741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:15.281741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:15.281741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:33:15.281741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:15.281741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:33:15.281741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:33:15.281741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:15.281741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:33:15.281741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:15.284611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:33:15.284611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:33:15.284611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:15.284611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:33:15.284611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:15.284611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:33:15.284611 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:33:15.284611 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:33:15.284611 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:33:15.284611 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:33:15.286004 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:33:15.286004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:15.286004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:33:15.286004 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:15.286004 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:33:15.286004 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:33:15.286004 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:15.286004 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:33:15.286004 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:15.286004 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:33:15.286004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:15.286004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:33:15.286004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:33:15.286004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:33:15.286497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:15.286497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:33:15.286497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:15.286497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:33:15.286497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:15.286497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:33:15.286497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:15.286497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:33:15.286497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:15.286497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:33:15.286497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:33:15.286497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:15.286497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:33:15.286497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:15.286497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:33:15.286497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:33:15.286497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:15.286497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:33:15.287074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:15.287074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:33:15.287074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:15.287074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:33:15.287074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:15.287074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:33:15.287074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:33:15.287074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:33:15.287074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:33:15.287074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:33:15.287074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:33:15.287074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:33:15.287462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:15.287462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:15.287462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:15.287462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:33:15.287462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:15.287462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:15.287462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:15.287462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:33:15.287462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:15.287462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:33:15.287462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:33:15.287462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:15.287462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:33:15.287462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:15.287853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:33:15.287853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:33:15.287853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:15.287853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:33:15.287853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:15.287853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:33:15.287853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:15.287853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:33:15.287853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:33:15.287853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:33:15.287853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:33:15.287853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:33:15.287853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:15.287853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:33:15.288245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:33:15.288245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:33:15.288245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:33:15.288245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:33:15.288245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:15.288245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:15.288245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:33:15.288245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:33:15.288245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:33:15.288245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:33:15.288245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:33:15.288245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:33:15.288245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:33:15.288245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:15.288245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:15.288245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:15.288245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:33:15.288245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:15.288245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:15.289070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:15.289070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:33:15.289070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:15.289070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:33:15.289070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:33:15.289070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:15.289070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:33:15.289070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:15.289070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:33:15.289070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:33:15.289070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:15.289070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:33:15.289070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:15.289070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:33:15.289070 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:33:15.289629 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:33:15.289629 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:33:15.289629 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:33:15.289629 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:33:15.289785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:15.290058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:33:15.290058 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:15.290058 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:33:15.290058 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:33:15.290058 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:15.290058 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:33:15.290058 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:15.290058 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:33:15.290788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:15.290788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:33:15.290788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:33:15.290788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:33:15.290788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:15.290788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:33:15.290788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:15.290788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:33:15.290788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:15.290788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:33:15.290788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:15.290788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:33:15.290788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:15.290788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:33:15.290788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:33:15.291064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:15.291064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:33:15.291064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:15.291064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:33:15.291064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:33:15.291064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:15.291064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:33:15.291064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:15.291064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:33:15.291742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:15.291742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:33:15.291742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:15.291742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:33:15.291742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:33:15.291742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:33:15.291742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:33:15.291742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:33:15.291742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:33:15.291742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:15.291742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:15.291967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:15.291967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:33:15.291967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:15.291967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:15.291967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:15.291967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:33:15.291967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:15.291967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:33:15.291967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:33:15.291967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:15.291967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:33:15.291967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:15.291967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:33:15.291967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:33:15.292225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:15.292225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:33:15.292225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:15.292225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:33:15.292922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:15.292922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:33:15.292922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:33:15.292922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:33:15.292922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:33:15.292922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:33:15.292922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:15.292922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:33:15.292922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:33:15.292922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:33:15.292922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:33:15.292922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:33:15.292922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:15.292922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:15.292922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:33:15.292922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:33:15.292922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:33:15.293263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:33:15.293263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:33:15.293263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:33:15.293263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:15.293263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:15.293263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:15.293263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:33:15.293263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:15.293263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:15.293263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:15.293263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:33:15.293263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:15.293263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:33:15.293263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:33:15.293263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:15.293263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:33:15.293263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:15.293263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:33:15.293263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:33:15.293644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:15.293644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:33:15.293644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:15.293644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:33:15.294219 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:33:15.294474 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:33:15.294588 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:33:15.294735 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:33:15.294967 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:33:15.295164 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:33:15.295293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:15.295706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:33:15.295706 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:15.295706 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:33:15.295706 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:33:15.295706 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:15.295706 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:33:15.295706 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:15.295706 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:33:15.296587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:15.296587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:33:15.296587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:15.296587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:33:15.296587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:33:15.296587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:33:15.296587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:33:15.296587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:15.296587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:15.296587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:15.296587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:33:15.296587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:15.296871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:15.296871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:15.296871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:33:15.296871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:15.296871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:33:15.296871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:33:15.296871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:15.296871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:33:15.296871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:15.296871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:33:15.296871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:33:15.296871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:15.296871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:33:15.297121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:15.297121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:33:15.297774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:15.297774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:33:15.297774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:33:15.297774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:33:15.297774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:33:15.297774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:33:15.297774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:15.297774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:33:15.297774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:33:15.297774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:33:15.297774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:33:15.297774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:33:15.297774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:15.297774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:15.297774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:33:15.297774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:33:15.298328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:33:15.298328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:33:15.298328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:15.298328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:15.298328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:15.298328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:33:15.298328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:15.298328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:15.298328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:15.298328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:33:15.298328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:15.298328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:33:15.298328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:33:15.298328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:15.298328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:33:15.298328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:15.298328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:33:15.298328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:33:15.298328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:15.298881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:33:15.298881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:15.298881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:33:20.033879 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:33:20.034778 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:33:20.034778 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:33:20.036058 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:33:20.071640 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:33:20.071953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:20.072649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:33:20.072649 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:20.072649 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:33:20.072649 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:33:20.072649 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:20.072649 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:33:20.072649 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:20.072649 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:33:20.073599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:20.073599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:33:20.073599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:20.073599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:33:20.073599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:33:20.073599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:33:20.073599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:33:20.073599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:20.073599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:20.073599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:20.073599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:33:20.073599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:20.073970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:20.073970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:20.073970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:33:20.073970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:20.073970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:33:20.073970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:33:20.073970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:20.073970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:33:20.073970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:20.073970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:33:20.073970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:33:20.073970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:20.073970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:33:20.074299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:20.074299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:33:20.074625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:20.074625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:33:20.074625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:33:20.074625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:33:20.074625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:33:20.074625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:33:20.074625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:20.074625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:33:20.074625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:33:20.074625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:33:20.074625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:33:20.074625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:33:20.074625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:20.074625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:20.074625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:33:20.074625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:33:20.075052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:33:20.075052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:33:20.075052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:20.075052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:20.075052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:20.075052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:33:20.075052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:20.075052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:20.075052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:20.075052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:33:20.075052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:20.075052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:33:20.075052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:33:20.075052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:20.075052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:33:20.075052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:20.075052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:33:20.075052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:33:20.075052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:20.075563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:33:20.075563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:20.075563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:33:20.274916 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:33:20.275606 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:33:20.275812 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:33:20.277038 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:33:20.277423 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:33:20.277721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:20.278682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:33:20.278682 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:20.278682 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:33:20.278682 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:33:20.278682 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:20.278682 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:33:20.278682 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:20.278682 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:33:20.280301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:20.280301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:33:20.280301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:20.280301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:33:20.280301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:33:20.280301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:33:20.280301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:33:20.280301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:20.280301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:20.280301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:20.280301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:33:20.280301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:20.280954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:20.280954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:20.280954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:33:20.280954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:20.280954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:33:20.280954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:33:20.280954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:20.280954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:33:20.280954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:20.280954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:33:20.280954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:33:20.280954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:20.280954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:33:20.281522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:20.281522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:33:20.282102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:20.282102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:33:20.282102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:33:20.282102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:33:20.282102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:33:20.282102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:33:20.282102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:20.282102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:33:20.282102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:33:20.282102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:33:20.282102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:33:20.282102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:33:20.282102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:20.282102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:20.282102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:33:20.282102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:33:20.282839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:33:20.282839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:33:20.282839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:20.282839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:20.282839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:20.282839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:33:20.282839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:20.282839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:20.282839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:20.282839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:33:20.282839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:20.282839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:33:20.282839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:33:20.282839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:20.282839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:33:20.282839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:20.282839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:33:20.282839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:33:20.282839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:20.283688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:33:20.283688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:20.283688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:33:20.283688 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:33:20.283688 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:33:20.283688 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:33:20.283688 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:33:20.283688 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:33:20.284259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:20.284800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:33:20.284800 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:20.284800 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:33:20.284800 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:33:20.284800 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:20.284800 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:33:20.284800 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:20.284800 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:33:20.285862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:20.285862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:33:20.285862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:33:20.285862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:33:20.285862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:20.285862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:33:20.285862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:20.285862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:33:20.285862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:20.285862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:33:20.285862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:20.285862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:33:20.285862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:20.285862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:33:20.285862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:33:20.286394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:20.286394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:33:20.286394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:20.286394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:33:20.286394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:33:20.286394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:20.286394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:33:20.286394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:20.286394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:33:20.286686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:20.286686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:33:20.286686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:20.286686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:33:20.286686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:33:20.286686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:33:20.286686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:33:20.286686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:33:20.286686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:33:20.286686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:33:20.286686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:20.286686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:20.287044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:20.287044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:33:20.287044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:20.287044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:20.287044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:20.287044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:33:20.287044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:20.287044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:33:20.287044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:33:20.287044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:20.287044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:33:20.287044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:20.287044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:33:20.287044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:33:20.287471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:20.287471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:33:20.287471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:20.287471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:33:20.287703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:20.287703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:33:20.287703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:33:20.287703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:33:20.287703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:33:20.287703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:33:20.287703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:20.287703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:33:20.287703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:33:20.287703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:33:20.287703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:33:20.287703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:33:20.287703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:20.287703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:20.287703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:33:20.287703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:33:20.288170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:33:20.288170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:33:20.288170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:33:20.288170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:33:20.288170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:33:20.288170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:20.288170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:20.288170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:20.288170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:33:20.288170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:20.288170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:20.288170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:20.288170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:33:20.288170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:20.288170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:33:20.288170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:33:20.288170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:20.288170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:33:20.288664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:20.288664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:33:20.288664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:33:20.288664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:20.288664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:33:20.288664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:20.288664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:33:20.289065 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:33:20.289426 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:33:20.289555 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:33:20.289668 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:33:20.290293 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:33:20.290758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:20.291500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:33:20.291500 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:20.291500 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:33:20.291500 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:33:20.291500 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:20.291500 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:33:20.291500 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:20.291500 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:33:20.292243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:20.292243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:33:20.292243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:33:20.292243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:33:20.292243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:20.292243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:33:20.292243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:20.292243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:33:20.292243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:20.292243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:33:20.292243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:20.292243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:33:20.292243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:20.292243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:33:20.292243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:33:20.292687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:20.292687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:33:20.292687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:20.292687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:33:20.292687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:33:20.292687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:20.292687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:33:20.292687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:20.292687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:33:20.293110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:20.293110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:33:20.293110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:20.293110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:33:20.293110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:33:20.293110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:33:20.293110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:33:20.293110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:33:20.293110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:33:20.293110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:20.293110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:20.293450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:20.293450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:33:20.293450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:20.293450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:20.293450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:20.293450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:33:20.293450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:20.293450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:33:20.293450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:33:20.293450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:20.293450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:33:20.293450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:20.293450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:33:20.293450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:33:20.293842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:20.293842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:33:20.293842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:20.293842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:33:20.294154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:20.294154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:33:20.294154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:33:20.294154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:33:20.294154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:33:20.294154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:33:20.294154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:20.294154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:33:20.294154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:33:20.294154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:33:20.294154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:33:20.294154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:33:20.294154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:20.294154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:20.294154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:33:20.294154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:33:20.294154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:33:20.294648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:33:20.294648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:33:20.294648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:33:20.294648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:20.294648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:20.294648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:20.294648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:33:20.294648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:20.294648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:20.294648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:20.294648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:33:20.294648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:20.294648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:33:20.294648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:33:20.294648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:20.294648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:33:20.294648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:20.294648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:33:20.294648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:33:20.295155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:20.295155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:33:20.295155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:20.295155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:33:20.295155 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:33:20.295317 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:33:20.295424 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:33:20.295593 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:33:20.295902 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:33:20.296161 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:33:20.296300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:20.296817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:33:20.296817 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:20.296817 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:33:20.296817 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:33:20.296817 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:20.296817 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:33:20.296817 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:20.296817 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:33:20.297656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:20.297656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:33:20.297656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:20.297656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:33:20.297656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:33:20.297656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:33:20.297656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:33:20.297656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:20.297656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:20.297656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:20.297656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:33:20.297656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:20.298017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:20.298017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:20.298017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:33:20.298017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:20.298017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:33:20.298017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:33:20.298017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:20.298017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:33:20.298017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:20.298017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:33:20.298017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:33:20.298017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:20.298017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:33:20.298373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:20.298373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:33:20.298612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:20.298612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:33:20.298612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:33:20.298612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:33:20.298612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:33:20.298612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:33:20.298612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:20.298612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:33:20.298612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:33:20.298612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:33:20.298612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:33:20.298612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:33:20.298612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:20.298612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:20.298612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:33:20.298612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:33:20.299065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:33:20.299065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:33:20.299065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:20.299065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:20.299065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:20.299065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:33:20.299065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:20.299065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:20.299065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:20.299065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:33:20.299065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:20.299065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:33:20.299065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:33:20.299065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:20.299065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:33:20.299065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:20.299065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:33:20.299065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:33:20.299065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:20.299585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:33:20.299585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:20.299585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:33:25.034633 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:33:25.035564 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:33:25.035564 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:33:25.036669 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:33:25.038314 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG dbcounter [-] [105581] Writing DB stats neutron:SELECT=240 {{(pid=105581) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 14:33:25.072626 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:33:25.073055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:25.074011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:33:25.074011 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:25.074011 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:33:25.074011 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:33:25.074011 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:25.074011 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:33:25.074011 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:25.074011 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:33:25.075056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:25.075056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:33:25.075056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:25.075056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:33:25.075056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:33:25.075056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:33:25.075056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:33:25.075056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:25.075056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:25.075056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:25.075056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:33:25.075056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:25.075518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:25.075518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:25.075518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:33:25.075518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:25.075518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:33:25.075518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:33:25.075518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:25.075518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:33:25.075518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:25.075518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:33:25.075518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:33:25.075518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:25.075518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:33:25.075915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:25.075915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:33:25.075994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:25.075994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:33:25.075994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:33:25.075994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:33:25.075994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:33:25.075994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:33:25.075994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:25.075994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:33:25.075994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:33:25.075994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:33:25.075994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:33:25.075994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:33:25.075994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:25.075994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:25.075994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:33:25.075994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:33:25.076527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:33:25.076527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:33:25.076527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:25.076527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:25.076527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:25.076527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:33:25.076527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:25.076527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:25.076527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:25.076527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:33:25.076527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:25.076527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:33:25.076527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:33:25.076527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:25.076527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:33:25.076527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:25.076527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:33:25.076527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:33:25.076527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:25.077168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:33:25.077168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:25.077168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:33:25.275619 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:33:25.276322 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:33:25.276494 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:33:25.277591 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:33:25.278108 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:33:25.278256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:25.279147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:33:25.279147 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:25.279147 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:33:25.279147 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:33:25.279147 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:25.279147 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:33:25.279147 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:25.279147 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:33:25.280666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:25.280666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:33:25.280666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:25.280666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:33:25.280666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:33:25.280666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:33:25.280666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:33:25.280666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:25.280666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:25.280666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:25.280666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:33:25.280666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:25.281396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:25.281396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:25.281396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:33:25.281396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:25.281396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:33:25.281396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:33:25.281396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:25.281396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:33:25.281396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:25.281396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:33:25.281396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:33:25.281396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:25.281396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:33:25.282056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:25.282056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:33:25.282222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:25.282222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:33:25.282222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:33:25.282222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:33:25.282222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:33:25.282222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:33:25.282222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:25.282222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:33:25.282222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:33:25.282222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:33:25.282222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:33:25.282222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:33:25.282222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:25.282222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:25.282222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:33:25.282222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:33:25.282982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:33:25.282982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:33:25.282982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:25.282982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:25.282982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:25.282982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:33:25.282982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:25.282982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:25.282982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:25.282982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:33:25.282982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:25.282982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:33:25.282982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:33:25.282982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:25.282982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:33:25.282982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:25.282982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:33:25.282982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:33:25.282982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:25.283794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:33:25.283794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:25.283794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:33:25.283794 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:33:25.283794 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:33:25.283794 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:33:25.283794 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:33:25.284196 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:33:25.284626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:25.286507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:33:25.286507 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:25.286507 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:33:25.286507 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:33:25.286507 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:25.286507 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:33:25.286507 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:25.286507 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:33:25.287358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:25.287358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:33:25.287358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:33:25.287358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:33:25.287358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:25.287358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:33:25.287358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:25.287358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:33:25.287358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:25.287358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:33:25.287358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:25.287358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:33:25.287358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:25.287358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:33:25.287358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:33:25.287884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:25.287884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:33:25.287884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:25.287884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:33:25.287884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:33:25.287884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:25.287884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:33:25.287884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:25.287884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:33:25.288152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:25.288152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:33:25.288152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:25.288152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:33:25.288152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:33:25.288152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:33:25.288152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:33:25.288152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:33:25.288152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:33:25.288152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:33:25.288152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:25.288152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:25.288566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:25.288566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:33:25.288566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:25.288566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:25.288566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:25.288566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:33:25.288566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:25.288566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:33:25.288566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:33:25.288566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:25.288566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:33:25.288566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:25.288566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:33:25.288566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:33:25.288931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:25.288931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:33:25.288931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:25.288931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:33:25.289461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:25.289461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:33:25.289461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:33:25.289461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:33:25.289461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:33:25.289461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:33:25.289461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:25.289461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:33:25.289461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:33:25.289461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:33:25.289461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:33:25.289461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:33:25.289461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:25.289461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:25.289461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:33:25.289461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:33:25.290120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:33:25.290120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:33:25.290120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:33:25.290120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:33:25.290120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:33:25.290120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:25.290120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:25.290120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:25.290120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:33:25.290120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:25.290120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:25.290120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:25.290120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:33:25.290120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:25.290120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:33:25.290120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:33:25.290120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:25.290120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:33:25.290605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:25.290605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:33:25.290605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:33:25.290605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:25.290605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:33:25.290605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:25.290605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:33:25.290605 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:33:25.290605 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:33:25.290605 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:33:25.291112 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:33:25.291112 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:33:25.291112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:25.291112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:33:25.291112 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:25.291112 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:33:25.291112 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:33:25.291112 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:25.291112 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:33:25.291112 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:25.291112 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:33:25.291854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:25.291854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:33:25.291854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:33:25.291854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:33:25.291854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:25.291854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:33:25.291854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:25.291854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:33:25.291854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:25.291854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:33:25.291854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:25.291854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:33:25.291854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:25.291854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:33:25.291854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:33:25.292259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:25.292259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:33:25.292259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:25.292259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:33:25.292259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:33:25.292259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:25.292259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:33:25.292259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:25.292259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:33:25.292697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:25.292697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:33:25.292697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:25.292697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:33:25.292697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:33:25.292697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:33:25.292697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:33:25.292697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:33:25.292697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:33:25.292697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:25.292697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:25.293023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:25.293023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:33:25.293023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:25.293023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:25.293023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:25.293023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:33:25.293023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:25.293023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:33:25.293023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:33:25.293023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:25.293023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:33:25.293023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:25.293023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:33:25.293023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:33:25.293390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:25.293390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:33:25.293390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:25.293390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:33:25.293717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:25.293717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:33:25.293717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:33:25.293717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:33:25.293717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:33:25.293717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:33:25.293717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:25.293717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:33:25.293717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:33:25.293717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:33:25.293717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:33:25.293717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:33:25.293717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:25.293717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:25.293717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:33:25.293717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:33:25.293717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:33:25.294225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:33:25.294225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:33:25.294225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:33:25.294225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:25.294225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:25.294225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:25.294225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:33:25.294225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:25.294225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:25.294225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:25.294225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:33:25.294225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:25.294225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:33:25.294225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:33:25.294225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:25.294225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:33:25.294225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:25.294225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:33:25.294225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:33:25.294578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:25.294578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:33:25.294578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:25.294578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:33:25.295463 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:33:25.295603 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:33:25.295707 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:33:25.295845 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:33:25.296167 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:33:25.296502 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:33:25.296714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:25.297255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:33:25.297255 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:25.297255 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:33:25.297255 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:33:25.297255 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:25.297255 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:33:25.297255 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:25.297255 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:33:25.298051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:25.298051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:33:25.298051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:25.298051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:33:25.298051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:33:25.298051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:33:25.298051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:33:25.298051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:25.298051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:25.298051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:25.298051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:33:25.298051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:25.298316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:25.298316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:25.298316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:33:25.298316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:25.298316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:33:25.298316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:33:25.298316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:25.298316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:33:25.298316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:25.298316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:33:25.298316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:33:25.298316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:25.298316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:33:25.298566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:25.298566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:33:25.298992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:25.298992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:33:25.298992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:33:25.298992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:33:25.298992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:33:25.298992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:33:25.298992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:25.298992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:33:25.298992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:33:25.298992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:33:25.298992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:33:25.298992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:33:25.298992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:25.298992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:25.298992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:33:25.298992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:33:25.299297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:33:25.299297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:33:25.299297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:25.299297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:25.299297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:25.299297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:33:25.299297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:25.299297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:25.299297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:25.299297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:33:25.299297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:25.299297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:33:25.299297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:33:25.299297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:25.299297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:33:25.299297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:25.299297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:33:25.299297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:33:25.299297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:25.299665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:33:25.299665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:25.299665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:33:30.035341 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:33:30.036107 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:33:30.036107 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:33:30.036858 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:33:30.070348 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:33:30.070798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:30.071765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:33:30.071765 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:30.071765 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:33:30.071765 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:33:30.071765 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:30.071765 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:33:30.071765 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:30.071765 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:33:30.072996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:30.072996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:33:30.072996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:30.072996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:33:30.072996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:33:30.072996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:33:30.072996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:33:30.072996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:30.072996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:30.072996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:30.072996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:33:30.072996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:30.073852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:30.073852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:30.073852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:33:30.073852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:30.073852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:33:30.073852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:33:30.073852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:30.073852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:33:30.073852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:30.073852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:33:30.073852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:33:30.073852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:30.073852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:33:30.074652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:30.074652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:33:30.074652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:30.074652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:33:30.074652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:33:30.074652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:33:30.074652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:33:30.074652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:33:30.074652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:30.074652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:33:30.074652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:33:30.074652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:33:30.074652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:33:30.074652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:33:30.074652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:30.075549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:30.075549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:33:30.075549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:33:30.075549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:33:30.075549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:33:30.075549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:30.075549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:30.075549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:30.075549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:33:30.075549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:30.075549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:30.075549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:30.075549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:33:30.075549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:30.075549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:33:30.075549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:33:30.075549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:30.075549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:33:30.075549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:30.076768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:33:30.076768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:33:30.076768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:30.076768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:33:30.076768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:30.076768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:33:30.276314 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:33:30.277002 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:33:30.277394 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:33:30.278679 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:33:30.279135 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:33:30.279516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:30.280529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:33:30.280529 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:30.280529 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:33:30.280529 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:33:30.280529 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:30.280529 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:33:30.280529 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:30.280529 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:33:30.282263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:30.282263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:33:30.282263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:30.282263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:33:30.282263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:33:30.282263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:33:30.282263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:33:30.282263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:30.282263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:30.282263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:30.282263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:33:30.282263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:30.282940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:30.282940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:30.282940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:33:30.282940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:30.282940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:33:30.282940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:33:30.282940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:30.282940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:33:30.282940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:30.282940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:33:30.282940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:33:30.282940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:30.282940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:33:30.283663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:30.283663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:33:30.284062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:30.284062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:33:30.284062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:33:30.284062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:33:30.284062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:33:30.284062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:33:30.284062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:30.284062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:33:30.284062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:33:30.284062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:33:30.284062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:33:30.284062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:33:30.284062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:30.284062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:30.284062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:33:30.284062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:33:30.284872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:33:30.284872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:33:30.284872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:30.284872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:30.284872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:30.284872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:33:30.284872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:30.284872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:30.284872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:30.284872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:33:30.284872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:30.284872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:33:30.284872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:33:30.284872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:30.284872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:33:30.284872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:30.284872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:33:30.284872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:33:30.284872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:30.285779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:33:30.285779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:30.285779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:33:30.285779 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:33:30.285779 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:33:30.285779 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:33:30.285779 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:33:30.285779 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:33:30.286089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:30.286148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:33:30.286148 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:30.286148 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:33:30.286148 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:33:30.286148 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:30.286148 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:33:30.286148 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:30.286148 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:33:30.286934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:30.286934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:33:30.286934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:33:30.286934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:33:30.286934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:30.286934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:33:30.286934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:30.286934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:33:30.286934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:30.286934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:33:30.286934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:30.286934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:33:30.286934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:30.286934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:33:30.286934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:33:30.287463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:30.287463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:33:30.287463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:30.287463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:33:30.287463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:33:30.287463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:30.287463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:33:30.287463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:30.287463 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:33:30.287969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:30.287969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:33:30.287969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:30.287969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:33:30.287969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:33:30.287969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:33:30.287969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:33:30.287969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:33:30.287969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:33:30.287969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:33:30.287969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:30.287969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:30.288353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:30.288353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:33:30.288353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:30.288353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:30.288353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:30.288353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:33:30.288353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:30.288353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:33:30.288353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:33:30.288353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:30.288353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:33:30.288353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:30.288353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:33:30.288353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:33:30.288743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:30.288743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:33:30.288743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:30.288743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:33:30.288867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:30.288867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:33:30.288867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:33:30.288867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:33:30.288867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:33:30.288867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:33:30.288867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:30.288867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:33:30.288867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:33:30.288867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:33:30.288867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:33:30.288867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:33:30.288867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:30.288867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:30.288867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:33:30.288867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:33:30.289277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:33:30.289277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:33:30.289277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:33:30.289277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:33:30.289277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:33:30.289277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:30.289277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:30.289277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:30.289277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:33:30.289277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:30.289277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:30.289277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:30.289277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:33:30.289277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:30.289277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:33:30.289277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:33:30.289277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:30.289277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:33:30.289782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:30.289782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:33:30.289782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:33:30.289782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:30.289782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:33:30.289782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:30.289782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:33:30.289967 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:33:30.290273 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:33:30.290381 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:33:30.290538 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:33:30.290953 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:33:30.291424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:30.292210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:33:30.292210 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:30.292210 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:33:30.292210 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:33:30.292210 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:30.292210 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:33:30.292210 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:30.292210 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:33:30.292954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:30.292954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:33:30.292954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:33:30.292954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:33:30.292954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:30.292954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:33:30.292954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:30.292954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:33:30.292954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:30.292954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:33:30.292954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:30.292954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:33:30.292954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:30.292954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:33:30.292954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:33:30.293473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:30.293473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:33:30.293473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:30.293473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:33:30.293473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:33:30.293473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:30.293473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:33:30.293473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:30.293473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:33:30.293897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:30.293897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:33:30.293897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:30.293897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:33:30.293897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:33:30.293897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:33:30.293897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:33:30.293897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:33:30.293897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:33:30.293897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:30.293897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:30.294274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:30.294274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:33:30.294274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:30.294274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:30.294274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:30.294274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:33:30.294274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:30.294274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:33:30.294274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:33:30.294274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:30.294274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:33:30.294274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:30.294274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:33:30.294274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:33:30.294677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:30.294677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:33:30.294677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:30.294677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:33:30.294979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:30.294979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:33:30.294979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:33:30.294979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:33:30.294979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:33:30.294979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:33:30.294979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:30.294979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:33:30.294979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:33:30.294979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:33:30.294979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:33:30.294979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:33:30.294979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:30.294979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:30.294979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:33:30.294979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:33:30.294979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:33:30.295483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:33:30.295483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:33:30.295483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:33:30.295483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:30.295483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:30.295483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:30.295483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:33:30.295483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:30.295483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:30.295483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:30.295483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:33:30.295483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:30.295483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:33:30.295483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:33:30.295483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:30.295483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:33:30.295483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:30.295483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:33:30.295483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:33:30.295995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:30.295995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:33:30.295995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:30.295995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:33:30.295995 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:33:30.295995 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:33:30.296181 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:33:30.296315 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:33:30.296857 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:33:30.297353 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:33:30.297734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:30.298649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:33:30.298649 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:30.298649 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:33:30.298649 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:33:30.298649 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:30.298649 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:33:30.298649 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:30.298649 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:33:30.299476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:30.299476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:33:30.299476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:30.299476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:33:30.299476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:33:30.299476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:33:30.299476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:33:30.299476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:30.299476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:30.299476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:30.299476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:33:30.299476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:30.299837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:30.299837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:30.299837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:33:30.299837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:30.299837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:33:30.299837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:33:30.299837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:30.299837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:33:30.299837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:30.299837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:33:30.299837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:33:30.299837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:30.299837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:33:30.300207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:30.300207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:33:30.300414 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:30.300414 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:33:30.300414 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:33:30.300414 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:33:30.300414 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:33:30.300414 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:33:30.300414 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:30.300414 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:33:30.300414 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:33:30.300414 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:33:30.300414 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:33:30.300414 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:33:30.300414 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:30.300414 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:30.300414 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:33:30.300414 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:33:30.300867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:33:30.300867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:33:30.300867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:30.300867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:30.300867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:30.300867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:33:30.300867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:30.300867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:30.300867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:30.300867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:33:30.300867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:30.300867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:33:30.300867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:33:30.300867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:30.300867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:33:30.300867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:30.300867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:33:30.300867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:33:30.300867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:30.301378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:33:30.301378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:30.301378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:33:35.036182 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:33:35.040115 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:33:35.040115 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:33:35.040115 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:33:35.071708 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:33:35.072205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:35.074587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:33:35.074587 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:35.074587 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:33:35.074587 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:33:35.074587 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:35.074587 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:33:35.074587 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:35.074587 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:33:35.074587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:35.074587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:33:35.074587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:35.074587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:33:35.074587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:33:35.074587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:33:35.074587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:33:35.075179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:35.075179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:35.075179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:35.075179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:33:35.075179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:35.075179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:35.075179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:35.075179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:33:35.075179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:35.075179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:33:35.075179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:33:35.075179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:35.075179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:33:35.075179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:35.075573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:33:35.075573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:33:35.075573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:35.075573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:33:35.075573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:35.075573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:33:35.075573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:35.075573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:33:35.075573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:33:35.075573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:33:35.075573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:33:35.075573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:33:35.075573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:35.075573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:33:35.076026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:33:35.076026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:33:35.076026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:33:35.076026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:33:35.076026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:35.076026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:35.076026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:33:35.076026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:33:35.076026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:33:35.076026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:33:35.076026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:35.076026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:35.076026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:35.076026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:33:35.076026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:35.076026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:35.076026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:35.076026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:33:35.076026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:35.076026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:33:35.076026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:33:35.076577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:35.076577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:33:35.076577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:35.076577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:33:35.076577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:33:35.076577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:35.076577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:33:35.076577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:35.076577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:33:35.277032 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:33:35.277534 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:33:35.277723 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:33:35.278849 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:33:35.279245 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:33:35.279543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:35.280496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:33:35.280496 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:35.280496 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:33:35.280496 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:33:35.280496 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:35.280496 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:33:35.280496 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:35.280496 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:33:35.281913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:35.281913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:33:35.281913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:35.281913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:33:35.281913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:33:35.281913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:33:35.281913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:33:35.281913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:35.281913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:35.281913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:35.281913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:33:35.281913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:35.282533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:35.282533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:35.282533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:33:35.282533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:35.282533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:33:35.282533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:33:35.282533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:35.282533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:33:35.282533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:35.282533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:33:35.282533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:33:35.282533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:35.282533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:33:35.283067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:35.283067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:33:35.283545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:35.283545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:33:35.283545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:33:35.283545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:33:35.283545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:33:35.283545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:33:35.283545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:35.283545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:33:35.283545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:33:35.283545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:33:35.283545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:33:35.283545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:33:35.283545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:35.283545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:35.283545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:33:35.283545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:33:35.284466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:33:35.284466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:33:35.284466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:35.284466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:35.284466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:35.284466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:33:35.284466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:35.284466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:35.284466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:35.284466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:33:35.284466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:35.284466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:33:35.284466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:33:35.284466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:35.284466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:33:35.284466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:35.284466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:33:35.284466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:33:35.284466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:35.285535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:33:35.285535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:35.285535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:33:35.285535 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:33:35.285535 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:33:35.285535 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:33:35.285535 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:33:35.285878 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:33:35.286098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:35.286898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:33:35.286898 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:35.286898 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:33:35.286898 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:33:35.286898 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:35.286898 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:33:35.286898 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:35.286898 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:33:35.287547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:35.287547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:33:35.287547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:33:35.287547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:33:35.287547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:35.287547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:33:35.287547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:35.287547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:33:35.287547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:35.287547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:33:35.287547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:35.287547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:33:35.287547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:35.287547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:33:35.287547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:33:35.288241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:35.288241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:33:35.288241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:35.288241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:33:35.288241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:33:35.288241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:35.288241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:33:35.288241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:35.288241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:33:35.288629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:35.288629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:33:35.288629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:35.288629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:33:35.288629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:33:35.288629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:33:35.288629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:33:35.288629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:33:35.288629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:33:35.288629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:33:35.288629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:35.288629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:35.289085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:35.289085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:33:35.289085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:35.289085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:35.289085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:35.289085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:33:35.289085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:35.289085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:33:35.289085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:33:35.289085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:35.289085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:33:35.289085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:35.289085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:33:35.289085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:33:35.289571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:35.289571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:33:35.289571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:35.289571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:33:35.289571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:35.289571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:33:35.289571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:33:35.289571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:33:35.289571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:33:35.289571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:33:35.289571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:35.289571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:33:35.289571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:33:35.289571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:33:35.289981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:33:35.289981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:33:35.289981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:35.289981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:35.289981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:33:35.289981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:33:35.289981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:33:35.289981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:33:35.289981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:33:35.289981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:33:35.289981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:33:35.289981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:35.289981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:35.289981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:35.289981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:33:35.289981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:35.289981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:35.289981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:35.289981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:33:35.290490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:35.290490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:33:35.290490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:33:35.290490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:35.290490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:33:35.290490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:35.290490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:33:35.290490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:33:35.290490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:35.290490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:33:35.290490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:35.290490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:33:35.290490 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:33:35.290847 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:33:35.290847 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:33:35.290917 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:33:35.291321 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:33:35.291669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:35.292324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:33:35.292324 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:35.292324 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:33:35.292324 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:33:35.292324 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:35.292324 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:33:35.292324 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:35.292324 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:33:35.293086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:35.293086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:33:35.293086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:33:35.293086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:33:35.293086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:35.293086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:33:35.293086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:35.293086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:33:35.293086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:35.293086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:33:35.293086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:35.293086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:33:35.293086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:35.293086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:33:35.293086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:33:35.293559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:35.293559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:33:35.293559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:35.293559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:33:35.293559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:33:35.293559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:35.293559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:33:35.293559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:35.293559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:33:35.293898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:35.293898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:33:35.293898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:35.293898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:33:35.293898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:33:35.293898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:33:35.293898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:33:35.293898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:33:35.293898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:33:35.293898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:35.293898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:35.294211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:35.294211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:33:35.294211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:35.294211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:35.294211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:35.294211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:33:35.294211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:35.294211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:33:35.294211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:33:35.294211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:35.294211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:33:35.294211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:35.294211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:33:35.294211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:33:35.294610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:35.294610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:33:35.294610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:35.294610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:33:35.294902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:35.294902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:33:35.294902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:33:35.294902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:33:35.294902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:33:35.294902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:33:35.294902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:35.294902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:33:35.294902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:33:35.294902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:33:35.294902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:33:35.294902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:33:35.294902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:35.294902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:35.294902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:33:35.294902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:33:35.294902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:33:35.295495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:33:35.295495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:33:35.295495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:33:35.295495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:35.295495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:35.295495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:35.295495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:33:35.295495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:35.295495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:35.295495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:35.295495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:33:35.295495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:35.295495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:33:35.295495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:33:35.295495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:35.295495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:33:35.295495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:35.295495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:33:35.295495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:33:35.296054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:35.296054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:33:35.296054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:35.296054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:33:35.296272 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:33:35.296575 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:33:35.296687 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:33:35.296826 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:33:35.297271 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:33:35.297623 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:33:35.297835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:35.298528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:33:35.298528 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:35.298528 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:33:35.298528 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:33:35.298528 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:35.298528 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:33:35.298528 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:35.298528 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:33:35.299305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:35.299305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:33:35.299305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:35.299305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:33:35.299305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:33:35.299305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:33:35.299305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:33:35.299305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:35.299305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:35.299305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:35.299305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:33:35.299305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:35.299816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:35.299816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:35.299816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:33:35.299816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:35.299816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:33:35.299816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:33:35.299816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:35.299816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:33:35.299816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:35.299816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:33:35.299816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:33:35.299816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:35.299816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:33:35.300194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:35.300194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:33:35.300259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:35.300259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:33:35.300259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:33:35.300259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:33:35.300259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:33:35.300259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:33:35.300259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:35.300259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:33:35.300259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:33:35.300259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:33:35.300259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:33:35.300259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:33:35.300259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:35.300259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:35.300259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:33:35.300259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:33:35.300807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:33:35.300807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:33:35.300807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:35.300807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:35.300807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:35.300807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:33:35.300807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:35.300807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:35.300807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:35.300807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:33:35.300807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:35.300807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:33:35.300807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:33:35.300807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:35.300807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:33:35.300807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:35.300807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:33:35.300807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:33:35.300807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:35.301351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:33:35.301351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:35.301351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:33:40.037791 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:33:40.039213 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:33:40.039213 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:33:40.040111 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:33:40.071769 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:33:40.072169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:40.073122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:33:40.073122 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:40.073122 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:33:40.073122 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:33:40.073122 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:40.073122 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:33:40.073122 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:40.073122 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:33:40.074155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:40.074155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:33:40.074155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:40.074155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:33:40.074155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:33:40.074155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:33:40.074155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:33:40.074155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:40.074155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:40.074155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:40.074155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:33:40.074155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:40.074840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:40.074840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:40.074840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:33:40.074840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:40.074840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:33:40.074840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:33:40.074840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:40.074840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:33:40.074840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:40.074840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:33:40.074840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:33:40.074840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:40.074840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:33:40.075375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:40.075375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:33:40.075375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:40.075375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:33:40.075375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:33:40.075375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:33:40.075375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:33:40.075375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:33:40.075375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:40.075375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:33:40.075375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:33:40.075375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:33:40.075375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:33:40.075375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:33:40.075375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:40.075900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:40.075900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:33:40.075900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:33:40.075900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:33:40.075900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:33:40.075900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:40.075900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:40.075900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:40.075900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:33:40.075900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:40.075900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:40.075900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:40.075900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:33:40.075900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:40.075900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:33:40.075900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:33:40.075900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:40.075900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:33:40.075900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:40.076591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:33:40.076591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:33:40.076591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:40.076591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:33:40.076591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:40.076591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:33:40.277611 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:33:40.278149 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:33:40.278352 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:33:40.279472 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:33:40.279725 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:33:40.279886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:40.280678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:33:40.280678 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:40.280678 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:33:40.280678 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:33:40.280678 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:40.280678 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:33:40.280678 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:40.280678 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:33:40.282195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:40.282195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:33:40.282195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:40.282195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:33:40.282195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:33:40.282195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:33:40.282195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:33:40.282195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:40.282195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:40.282195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:40.282195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:33:40.282195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:40.282991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:40.282991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:40.282991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:33:40.282991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:40.282991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:33:40.282991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:33:40.282991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:40.282991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:33:40.282991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:40.282991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:33:40.282991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:33:40.282991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:40.282991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:33:40.283615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:40.283615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:33:40.283895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:40.283895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:33:40.283895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:33:40.283895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:33:40.283895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:33:40.283895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:33:40.283895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:40.283895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:33:40.283895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:33:40.283895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:33:40.283895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:33:40.283895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:33:40.283895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:40.283895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:40.283895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:33:40.283895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:33:40.284652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:33:40.284652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:33:40.284652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:40.284652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:40.284652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:40.284652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:33:40.284652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:40.284652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:40.284652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:40.284652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:33:40.284652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:40.284652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:33:40.284652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:33:40.284652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:40.284652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:33:40.284652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:40.284652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:33:40.284652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:33:40.284652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:40.285527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:33:40.285527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:40.285527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:33:40.285527 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:33:40.285767 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:33:40.285868 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:33:40.286060 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:33:40.286435 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:33:40.286792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:40.287380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:33:40.287380 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:40.287380 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:33:40.287380 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:33:40.287380 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:40.287380 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:33:40.287380 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:40.287380 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:33:40.288173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:40.288173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:33:40.288173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:33:40.288173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:33:40.288173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:40.288173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:33:40.288173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:40.288173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:33:40.288173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:40.288173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:33:40.288173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:40.288173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:33:40.288173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:40.288173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:33:40.288173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:33:40.288670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:40.288670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:33:40.288670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:40.288670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:33:40.288670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:33:40.288670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:40.288670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:33:40.288670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:40.288670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:33:40.288961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:40.288961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:33:40.288961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:40.288961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:33:40.288961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:33:40.288961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:33:40.288961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:33:40.288961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:33:40.288961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:33:40.288961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:33:40.288961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:40.288961 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:40.289313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:40.289313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:33:40.289313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:40.289313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:40.289313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:40.289313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:33:40.289313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:40.289313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:33:40.289313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:33:40.289313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:40.289313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:33:40.289313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:40.289313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:33:40.289313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:33:40.289718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:40.289718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:33:40.289718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:40.289718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:33:40.289890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:40.289890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:33:40.289890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:33:40.289890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:33:40.289890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:33:40.289890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:33:40.289890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:40.289890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:33:40.289890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:33:40.289890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:33:40.289890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:33:40.289890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:33:40.289890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:40.289890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:40.289890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:33:40.289890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:33:40.290701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:33:40.290701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:33:40.290701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:33:40.290701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:33:40.290701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:33:40.290701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:40.290701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:40.290701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:40.290701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:33:40.290701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:40.290701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:40.290701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:40.290701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:33:40.290701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:40.290701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:33:40.290701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:33:40.290701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:40.290701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:33:40.291250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:40.291250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:33:40.291250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:33:40.291250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:40.291250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:33:40.291250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:40.291250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:33:40.291250 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:33:40.291250 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:33:40.291250 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:33:40.291572 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:33:40.291678 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:33:40.291898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:40.292365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:33:40.292365 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:40.292365 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:33:40.292365 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:33:40.292365 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:40.292365 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:33:40.292365 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:40.292365 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:33:40.293111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:40.293111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:33:40.293111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:33:40.293111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:33:40.293111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:40.293111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:33:40.293111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:40.293111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:33:40.293111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:40.293111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:33:40.293111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:40.293111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:33:40.293111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:40.293111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:33:40.293111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:33:40.293547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:40.293547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:33:40.293547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:40.293547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:33:40.293547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:33:40.293547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:40.293547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:33:40.293547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:40.293547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:33:40.294004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:40.294004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:33:40.294004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:40.294004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:33:40.294004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:33:40.294004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:33:40.294004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:33:40.294004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:33:40.294004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:33:40.294004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:40.294004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:40.294337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:40.294337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:33:40.294337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:40.294337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:40.294337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:40.294337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:33:40.294337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:40.294337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:33:40.294337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:33:40.294337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:40.294337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:33:40.294337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:40.294337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:33:40.294337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:33:40.294851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:40.294851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:33:40.294851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:40.294851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:33:40.295051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:40.295051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:33:40.295051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:33:40.295051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:33:40.295051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:33:40.295051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:33:40.295051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:40.295051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:33:40.295051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:33:40.295051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:33:40.295051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:33:40.295051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:33:40.295051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:40.295051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:40.295051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:33:40.295051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:33:40.295051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:33:40.295552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:33:40.295552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:33:40.295552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:33:40.295552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:40.295552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:40.295552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:40.295552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:33:40.295552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:40.295552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:40.295552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:40.295552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:33:40.295552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:40.295552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:33:40.295552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:33:40.295552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:40.295552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:33:40.295552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:40.295552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:33:40.295552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:33:40.296069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:40.296069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:33:40.296069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:40.296069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:33:40.296645 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:33:40.296877 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:33:40.296983 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:33:40.297121 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:33:40.297390 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:33:40.297579 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:33:40.297683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:40.298064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:33:40.298064 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:40.298064 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:33:40.298064 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:33:40.298064 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:40.298064 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:33:40.298064 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:40.298064 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:33:40.298888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:40.298888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:33:40.298888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:40.298888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:33:40.298888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:33:40.298888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:33:40.298888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:33:40.298888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:40.298888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:40.298888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:40.298888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:33:40.298888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:40.299243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:40.299243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:40.299243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:33:40.299243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:40.299243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:33:40.299243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:33:40.299243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:40.299243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:33:40.299243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:40.299243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:33:40.299243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:33:40.299243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:40.299243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:33:40.299627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:40.299627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:33:40.299807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:40.299807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:33:40.299807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:33:40.299807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:33:40.299807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:33:40.299807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:33:40.299807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:40.299807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:33:40.299807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:33:40.299807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:33:40.299807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:33:40.299807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:33:40.299807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:40.299807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:40.299807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:33:40.299807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:33:40.300251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:33:40.300251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:33:40.300251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:40.300251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:40.300251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:40.300251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:33:40.300251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:40.300251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:40.300251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:40.300251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:33:40.300251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:40.300251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:33:40.300251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:33:40.300251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:40.300251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:33:40.300251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:40.300251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:33:40.300251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:33:40.300251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:40.300844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:33:40.300844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:40.300844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:33:45.038461 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:33:45.039151 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:33:45.039151 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:33:45.040039 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:33:45.074337 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:33:45.074531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:45.075003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:33:45.075003 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:45.075003 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:33:45.075003 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:33:45.075003 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:45.075003 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:33:45.075003 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:45.075003 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:33:45.075736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:45.075736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:33:45.075736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:45.075736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:33:45.075736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:33:45.075736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:33:45.075736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:33:45.075736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:45.075736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:45.075736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:45.075736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:33:45.075736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:45.076145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:45.076145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:45.076145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:33:45.076145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:45.076145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:33:45.076145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:33:45.076145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:45.076145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:33:45.076145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:45.076145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:33:45.076145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:33:45.076145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:45.076145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:33:45.076597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:45.076597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:33:45.076597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:45.076597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:33:45.076597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:33:45.076597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:33:45.076597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:33:45.076597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:33:45.076597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:45.076597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:33:45.076597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:33:45.076597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:33:45.076597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:33:45.076597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:33:45.076597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:45.077101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:45.077101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:33:45.077101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:33:45.077101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:33:45.077101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:33:45.077101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:45.077101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:45.077101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:45.077101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:33:45.077101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:45.077101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:45.077101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:45.077101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:33:45.077101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:45.077101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:33:45.077101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:33:45.077101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:45.077101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:33:45.077101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:45.077832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:33:45.077832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:33:45.077832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:45.077832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:33:45.077832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:45.077832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:33:45.278206 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:33:45.278693 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:33:45.278877 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:33:45.279892 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:33:45.280159 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:33:45.280294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:45.281004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:33:45.281004 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:45.281004 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:33:45.281004 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:33:45.281004 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:45.281004 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:33:45.281004 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:45.281004 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:33:45.282616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:45.282616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:33:45.282616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:45.282616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:33:45.282616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:33:45.282616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:33:45.282616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:33:45.282616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:45.282616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:45.282616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:45.282616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:33:45.282616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:45.283221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:45.283221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:45.283221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:33:45.283221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:45.283221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:33:45.283221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:33:45.283221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:45.283221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:33:45.283221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:45.283221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:33:45.283221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:33:45.283221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:45.283221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:33:45.283688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:45.283688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:33:45.284542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:45.284542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:33:45.284542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:33:45.284542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:33:45.284542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:33:45.284542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:33:45.284542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:45.284542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:33:45.284542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:33:45.284542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:33:45.284542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:33:45.284542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:33:45.284542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:45.284542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:45.284542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:33:45.284542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:33:45.284989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:33:45.284989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:33:45.284989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:45.284989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:45.284989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:45.284989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:33:45.284989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:45.284989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:45.284989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:45.284989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:33:45.284989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:45.284989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:33:45.284989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:33:45.284989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:45.284989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:33:45.284989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:45.284989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:33:45.284989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:33:45.284989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:45.285613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:33:45.285613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:45.285613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:33:45.285704 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:33:45.285915 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:33:45.286019 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:33:45.286136 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:33:45.286639 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:33:45.286929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:45.287700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:33:45.287700 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:45.287700 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:33:45.287700 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:33:45.287700 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:45.287700 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:33:45.287700 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:45.287700 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:33:45.288517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:45.288517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:33:45.288517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:33:45.288517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:33:45.288517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:45.288517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:33:45.288517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:45.288517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:33:45.288517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:45.288517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:33:45.288517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:45.288517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:33:45.288517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:45.288517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:33:45.288517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:33:45.289084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:45.289084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:33:45.289084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:45.289084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:33:45.289084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:33:45.289084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:45.289084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:33:45.289084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:45.289084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:33:45.289388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:45.289388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:33:45.289388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:45.289388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:33:45.289388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:33:45.289388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:33:45.289388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:33:45.289388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:33:45.289388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:33:45.289388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:33:45.289388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:45.289388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:45.289775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:45.289775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:33:45.289775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:45.289775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:45.289775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:45.289775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:33:45.289775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:45.289775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:33:45.289775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:33:45.289775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:45.289775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:33:45.289775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:45.289775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:33:45.289775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:33:45.290145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:45.290145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:33:45.290145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:45.290145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:33:45.290642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:45.290642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:33:45.290642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:33:45.290642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:33:45.290642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:33:45.290642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:33:45.290642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:45.290642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:33:45.290642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:33:45.290642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:33:45.290642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:33:45.290642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:33:45.290642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:45.290642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:45.290642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:33:45.290642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:33:45.291143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:33:45.291143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:33:45.291143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:33:45.291143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:33:45.291143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:33:45.291143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:45.291143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:45.291143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:45.291143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:33:45.291143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:45.291143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:45.291143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:45.291143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:33:45.291143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:45.291143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:33:45.291143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:33:45.291143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:45.291143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:33:45.291685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:45.291685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:33:45.291685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:33:45.291685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:45.291685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:33:45.291685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:45.291685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:33:45.291685 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:33:45.291685 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:33:45.291685 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:33:45.292004 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:33:45.292124 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:33:45.292481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:45.293098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:33:45.293098 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:45.293098 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:33:45.293098 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:33:45.293098 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:45.293098 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:33:45.293098 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:45.293098 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:33:45.293874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:45.293874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:33:45.293874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:33:45.293874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:33:45.293874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:45.293874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:33:45.293874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:45.293874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:33:45.293874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:45.293874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:33:45.293874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:45.293874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:33:45.293874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:45.293874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:33:45.293874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:33:45.294295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:45.294295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:33:45.294295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:45.294295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:33:45.294295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:33:45.294295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:45.294295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:33:45.294295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:45.294295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:33:45.294804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:45.294804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:33:45.294804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:45.294804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:33:45.294804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:33:45.294804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:33:45.294804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:33:45.294804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:33:45.294804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:33:45.294804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:45.294804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:45.295137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:45.295137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:33:45.295137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:45.295137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:45.295137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:45.295137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:33:45.295137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:45.295137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:33:45.295137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:33:45.295137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:45.295137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:33:45.295137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:45.295137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:33:45.295137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:33:45.295556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:45.295556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:33:45.295556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:45.295556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:33:45.295825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:45.295825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:33:45.295825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:33:45.295825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:33:45.295825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:33:45.295825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:33:45.295825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:45.295825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:33:45.295825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:33:45.295825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:33:45.295825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:33:45.295825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:33:45.295825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:45.295825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:45.295825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:33:45.295825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:33:45.295825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:33:45.296309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:33:45.296309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:33:45.296309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:33:45.296309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:45.296309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:45.296309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:45.296309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:33:45.296309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:45.296309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:45.296309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:45.296309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:33:45.296309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:45.296309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:33:45.296309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:33:45.296309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:45.296309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:33:45.296309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:45.296309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:33:45.296309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:33:45.296857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:45.296857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:33:45.296857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:45.296857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:33:45.296975 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:33:45.297239 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:33:45.297336 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:33:45.297515 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:33:45.298032 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:33:45.298427 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:33:45.298687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:45.299393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:33:45.299393 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:45.299393 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:33:45.299393 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:33:45.299393 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:45.299393 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:33:45.299393 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:45.299393 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:33:45.300377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:45.300377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:33:45.300377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:45.300377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:33:45.300377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:33:45.300377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:33:45.300377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:33:45.300377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:45.300377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:45.300377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:45.300377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:33:45.300377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:45.300788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:45.300788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:45.300788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:33:45.300788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:45.300788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:33:45.300788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:33:45.300788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:45.300788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:33:45.300788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:45.300788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:33:45.300788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:33:45.300788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:45.300788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:33:45.301143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:45.301143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:33:45.301425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:45.301425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:33:45.301425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:33:45.301425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:33:45.301425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:33:45.301425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:33:45.301425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:45.301425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:33:45.301425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:33:45.301425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:33:45.301425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:33:45.301425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:33:45.301425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:45.301425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:45.301425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:33:45.301425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:33:45.301881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:33:45.301881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:33:45.301881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:45.301881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:45.301881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:45.301881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:33:45.301881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:45.301881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:45.301881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:45.301881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:33:45.301881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:45.301881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:33:45.301881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:33:45.301881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:45.301881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:33:45.301881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:45.301881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:33:45.301881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:33:45.301881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:45.302424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:33:45.302424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:45.302424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:33:50.039892 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:33:50.040903 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:33:50.040903 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:33:50.042168 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:33:50.075083 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:33:50.075511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:50.076336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:33:50.076336 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:50.076336 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:33:50.076336 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:33:50.076336 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:50.076336 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:33:50.076336 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:50.076336 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:33:50.077114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:50.077114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:33:50.077114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:50.077114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:33:50.077114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:33:50.077114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:33:50.077114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:33:50.077114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:50.077114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:50.077114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:50.077114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:33:50.077114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:50.077789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:50.077789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:50.077789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:33:50.077789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:50.077789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:33:50.077789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:33:50.077789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:50.077789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:33:50.077789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:50.077789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:33:50.077789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:33:50.077789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:50.077789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:33:50.078473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:50.078473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:33:50.078473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:50.078473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:33:50.078473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:33:50.078473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:33:50.078473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:33:50.078473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:33:50.078473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:50.078473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:33:50.078473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:33:50.078473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:33:50.078473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:33:50.078473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:33:50.078473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:50.079247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:50.079247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:33:50.079247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:33:50.079247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:33:50.079247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:33:50.079247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:50.079247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:50.079247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:50.079247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:33:50.079247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:50.079247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:50.079247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:50.079247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:33:50.079247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:50.079247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:33:50.079247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:33:50.079247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:50.079247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:33:50.079247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:50.080317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:33:50.080317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:33:50.080317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:50.080317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:33:50.080317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:50.080317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:33:50.280952 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:33:50.281726 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:33:50.281838 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:33:50.282944 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:33:50.283368 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:33:50.283545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:50.284217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:33:50.284217 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:50.284217 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:33:50.284217 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:33:50.284217 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:50.284217 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:33:50.284217 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:50.284217 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:33:50.285237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:50.285237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:33:50.285237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:50.285237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:33:50.285237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:33:50.285237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:33:50.285237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:33:50.285237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:50.285237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:50.285237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:50.285237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:33:50.285237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:50.285619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:50.285619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:50.285619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:33:50.285619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:50.285619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:33:50.285619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:33:50.285619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:50.285619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:33:50.285619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:50.285619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:33:50.285619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:33:50.285619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:50.285619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:33:50.285956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:50.285956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:33:50.286380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:50.286380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:33:50.286380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:33:50.286380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:33:50.286380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:33:50.286380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:33:50.286380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:50.286380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:33:50.286380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:33:50.286380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:33:50.286380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:33:50.286380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:33:50.286380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:50.286380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:50.286380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:33:50.286380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:33:50.287823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:33:50.287823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:33:50.287823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:50.287823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:50.287823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:50.287823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:33:50.287823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:50.287823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:50.287823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:50.287823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:33:50.287823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:50.287823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:33:50.287823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:33:50.287823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:50.287823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:33:50.287823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:50.287823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:33:50.287823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:33:50.287823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:50.288743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:33:50.288743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:50.288743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:33:50.288743 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:33:50.288743 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:33:50.288743 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:33:50.288743 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:33:50.288743 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:33:50.289270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:50.289270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:33:50.289270 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:50.289270 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:33:50.289270 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:33:50.289270 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:50.289270 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:33:50.289270 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:50.289270 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:33:50.289270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:50.289270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:33:50.289270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:33:50.289270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:33:50.289270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:50.289270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:33:50.289270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:50.289270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:33:50.290159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:50.290159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:33:50.290159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:50.290159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:33:50.290159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:50.290159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:33:50.290159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:33:50.290159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:50.290159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:33:50.290159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:50.290159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:33:50.290159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:33:50.290159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:50.290159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:33:50.290159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:50.290159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:33:50.292543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:50.292543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:33:50.292543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:50.292543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:33:50.292543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:33:50.292543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:33:50.292543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:33:50.292543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:33:50.292543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:33:50.292543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:33:50.292543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:50.292543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:50.293780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:50.293780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:33:50.293780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:50.293780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:50.293780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:50.293780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:33:50.293780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:50.293780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:33:50.293780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:33:50.293780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:50.293780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:33:50.293780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:50.293780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:33:50.293780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:33:50.294842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:50.294842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:33:50.294842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:50.294842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:33:50.294842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:50.294842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:33:50.294842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:33:50.294842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:33:50.294842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:33:50.294842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:33:50.294842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:50.294842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:33:50.294842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:33:50.294842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:33:50.296128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:33:50.296128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:33:50.296128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:50.296128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:50.296128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:33:50.296128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:33:50.296128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:33:50.296128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:33:50.296128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:33:50.296128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:33:50.296128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:33:50.296128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:50.296128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:50.296128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:50.296128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:33:50.296128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:50.296128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:50.296128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:50.296128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:33:50.297419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:50.297419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:33:50.297419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:33:50.297419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:50.297419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:33:50.297419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:50.297419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:33:50.297419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:33:50.297419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:50.297419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:33:50.297419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:50.297419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:33:50.297419 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:33:50.297419 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:33:50.298075 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:33:50.298075 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:33:50.298075 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:33:50.298075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:50.298075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:33:50.298075 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:50.298075 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:33:50.298075 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:33:50.298075 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:50.298075 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:33:50.298075 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:50.298075 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:33:50.298426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:50.298426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:33:50.298426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:33:50.298426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:33:50.298426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:50.298426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:33:50.298426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:50.298426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:33:50.298426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:50.298426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:33:50.298426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:50.298426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:33:50.298426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:50.298426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:33:50.298426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:33:50.298805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:50.298805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:33:50.298805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:50.298805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:33:50.298805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:33:50.298805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:50.298805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:33:50.298805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:50.298805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:33:50.298805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:50.298805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:33:50.298805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:50.298805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:33:50.298805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:33:50.299177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:33:50.299177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:33:50.299177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:33:50.299177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:33:50.299177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:50.299177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:50.299177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:50.299177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:33:50.299177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:50.299177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:50.299177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:50.299177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:33:50.299177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:50.299522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:33:50.299522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:33:50.299522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:50.299522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:33:50.299522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:50.299522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:33:50.299522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:33:50.299522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:50.299522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:33:50.299522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:50.299522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:33:50.299522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:50.299910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:33:50.299910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:33:50.299910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:33:50.299910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:33:50.299910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:33:50.299910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:50.299910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:33:50.299910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:33:50.299910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:33:50.299910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:33:50.299910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:33:50.299910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:50.299910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:50.299910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:33:50.299910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:33:50.299910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:33:50.299910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:33:50.299910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:33:50.299910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:33:50.300515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:50.300515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:50.300515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:50.300515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:33:50.300515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:50.300515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:50.300515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:50.300515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:33:50.300515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:50.300515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:33:50.300515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:33:50.300515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:50.300515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:33:50.300515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:50.300515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:33:50.300515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:33:50.300515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:50.300515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:33:50.300515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:50.300515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:33:50.301097 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:33:50.301097 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:33:50.301097 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:33:50.301097 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:33:50.301097 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:33:50.301097 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:33:50.301287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:50.301287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:33:50.301287 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:50.301287 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:33:50.301287 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:33:50.301287 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:50.301287 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:33:50.301287 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:50.301287 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:33:50.301287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:50.301287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:33:50.301287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:50.301287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:33:50.301287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:33:50.301287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:33:50.301692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:33:50.301692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:50.301692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:50.301692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:50.301692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:33:50.301692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:50.301692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:50.301692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:50.301692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:33:50.301692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:50.301692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:33:50.301692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:33:50.301692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:50.301692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:33:50.302021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:50.302021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:33:50.302021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:33:50.302021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:50.302021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:33:50.302021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:50.302021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:33:50.302021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:50.302021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:33:50.302021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:33:50.302021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:33:50.302021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:33:50.302021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:33:50.302359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:50.302359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:33:50.302359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:33:50.302359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:33:50.302359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:33:50.302359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:33:50.302359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:50.302359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:50.302359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:33:50.302359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:33:50.302359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:33:50.302359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:33:50.302359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:50.302359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:50.302359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:50.302359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:33:50.302359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:50.302359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:50.302359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:50.302359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:33:50.302857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:50.302857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:33:50.302857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:33:50.302857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:50.302857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:33:50.302857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:50.302857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:33:50.302857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:33:50.302857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:50.302857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:33:50.302857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:50.302857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:33:55.040903 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:33:55.042636 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:33:55.042636 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:33:55.043255 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:33:55.077350 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:33:55.077777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:55.078634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:33:55.078634 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:55.078634 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:33:55.078634 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:33:55.078634 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:55.078634 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:33:55.078634 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:55.078634 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:33:55.079633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:55.079633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:33:55.079633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:55.079633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:33:55.079633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:33:55.079633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:33:55.079633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:33:55.079633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:55.079633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:55.079633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:55.079633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:33:55.079633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:55.080299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:55.080299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:55.080299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:33:55.080299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:55.080299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:33:55.080299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:33:55.080299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:55.080299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:33:55.080299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:55.080299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:33:55.080299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:33:55.080299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:55.080299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:33:55.080993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:55.080993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:33:55.080993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:55.080993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:33:55.080993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:33:55.080993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:33:55.080993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:33:55.080993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:33:55.080993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:55.080993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:33:55.080993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:33:55.080993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:33:55.080993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:33:55.080993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:33:55.080993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:55.081812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:55.081812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:33:55.081812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:33:55.081812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:33:55.081812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:33:55.081812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:55.081812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:55.081812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:55.081812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:33:55.081812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:55.081812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:55.081812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:55.081812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:33:55.081812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:55.081812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:33:55.081812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:33:55.081812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:55.081812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:33:55.081812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:55.082983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:33:55.082983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:33:55.082983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:55.082983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:33:55.082983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:55.082983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:33:55.281844 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:33:55.282660 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:33:55.282886 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:33:55.284132 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:33:55.284529 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:33:55.284782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:55.285769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:33:55.285769 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:55.285769 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:33:55.285769 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:33:55.285769 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:55.285769 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:33:55.285769 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:55.285769 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:33:55.287321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:55.287321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:33:55.287321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:55.287321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:33:55.287321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:33:55.287321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:33:55.287321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:33:55.287321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:55.287321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:55.287321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:55.287321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:33:55.287321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:55.288024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:55.288024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:55.288024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:33:55.288024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:55.288024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:33:55.288024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:33:55.288024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:55.288024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:33:55.288024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:55.288024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:33:55.288024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:33:55.288024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:55.288024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:33:55.288600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:55.288600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:33:55.289259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:55.289259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:33:55.289259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:33:55.289259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:33:55.289259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:33:55.289259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:33:55.289259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:55.289259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:33:55.289259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:33:55.289259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:33:55.289259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:33:55.289259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:33:55.289259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:55.289259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:55.289259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:33:55.289259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:33:55.290059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:33:55.290059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:33:55.290059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:55.290059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:55.290059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:55.290059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:33:55.290059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:55.290059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:55.290059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:55.290059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:33:55.290059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:55.290059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:33:55.290059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:33:55.290059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:55.290059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:33:55.290059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:55.290059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:33:55.290059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:33:55.290059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:55.291016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:33:55.291016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:55.291016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:33:55.291016 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:33:55.291016 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:33:55.291016 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:33:55.291016 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:33:55.291016 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:33:55.291292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:55.292008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:33:55.292008 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:55.292008 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:33:55.292008 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:33:55.292008 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:55.292008 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:33:55.292008 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:55.292008 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:33:55.292949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:55.292949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:33:55.292949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:33:55.292949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:33:55.292949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:55.292949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:33:55.292949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:55.292949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:33:55.292949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:55.292949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:33:55.292949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:55.292949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:33:55.292949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:55.292949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:33:55.292949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:33:55.293545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:55.293545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:33:55.293545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:55.293545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:33:55.293545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:33:55.293545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:55.293545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:33:55.293545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:55.293545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:33:55.293832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:55.293832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:33:55.293832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:55.293832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:33:55.293832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:33:55.293832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:33:55.293832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:33:55.293832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:33:55.293832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:33:55.293832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:33:55.293832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:55.293832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:55.294281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:55.294281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:33:55.294281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:55.294281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:55.294281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:55.294281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:33:55.294281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:55.294281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:33:55.294281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:33:55.294281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:55.294281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:33:55.294281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:55.294281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:33:55.294281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:33:55.294723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:55.294723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:33:55.294723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:55.294723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:33:55.294723 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:33:55.295256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:55.295256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:33:55.295256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:33:55.295256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:33:55.295256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:33:55.295256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:33:55.295256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:55.295256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:33:55.295256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:33:55.295256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:33:55.295256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:33:55.295256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:33:55.295256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:55.295256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:55.295256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:33:55.295256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:33:55.295740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:33:55.295740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:33:55.295740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:33:55.295740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:33:55.295740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:33:55.295740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:55.295740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:55.295740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:55.295740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:33:55.295740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:55.295740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:55.295740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:55.295740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:33:55.295740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:55.295740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:33:55.295740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:33:55.295740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:55.295740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:33:55.296236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:55.296236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:33:55.296236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:33:55.296236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:55.296236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:33:55.296236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:55.296236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:33:55.296236 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:33:55.296236 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:33:55.296236 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:33:55.296561 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:33:55.296642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:55.297317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:33:55.297317 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:55.297317 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:33:55.297317 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:33:55.297317 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:55.297317 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:33:55.297317 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:55.297317 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:33:55.298060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:55.298060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:33:55.298060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:33:55.298060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:33:55.298060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:55.298060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:33:55.298060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:55.298060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:33:55.298060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:55.298060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:33:55.298060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:55.298060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:33:55.298060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:55.298060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:33:55.298060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:33:55.298564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:55.298564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:33:55.298564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:55.298564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:33:55.298564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:33:55.298564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:55.298564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:33:55.298564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:55.298564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:33:55.299174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:55.299174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:33:55.299174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:55.299174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:33:55.299174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:33:55.299174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:33:55.299174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:33:55.299174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:33:55.299174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:33:55.299174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:55.299174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:55.299630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:55.299630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:33:55.299630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:55.299630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:55.299630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:55.299630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:33:55.299630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:55.299630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:33:55.299630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:33:55.299630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:55.299630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:33:55.299630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:55.299630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:33:55.299630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:33:55.300146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:55.300146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:33:55.300146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:55.300146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:33:55.300146 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:33:55.300632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:55.300632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:33:55.300632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:33:55.300632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:33:55.300632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:33:55.300632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:33:55.300632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:55.300632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:33:55.300632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:33:55.300632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:33:55.300632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:33:55.300632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:33:55.300632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:55.300632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:55.300632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:33:55.300632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:33:55.300632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:33:55.301125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:33:55.301125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:33:55.301125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:33:55.301125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:55.301125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:55.301125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:55.301125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:33:55.301125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:55.301125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:55.301125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:55.301125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:33:55.301125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:55.301125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:33:55.301125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:33:55.301125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:55.301125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:33:55.301125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:55.301125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:33:55.301125 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:33:55.301660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:55.301660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:33:55.301660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:55.301660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:33:55.301660 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:33:55.301660 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:33:55.301660 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:33:55.301660 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:33:55.301905 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:33:55.301905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:55.302312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:33:55.302312 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:55.302312 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:33:55.302312 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:33:55.302312 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:55.302312 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:33:55.302312 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:55.302312 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:33:55.303155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:55.303155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:33:55.303155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:55.303155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:33:55.303155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:33:55.303155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:33:55.303155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:33:55.303155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:55.303155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:55.303155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:55.303155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:33:55.303155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:55.303708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:33:55.303708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:55.303708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:33:55.303708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:55.303708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:33:55.303708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:33:55.303708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:55.303708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:33:55.303708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:55.303708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:33:55.303708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:33:55.303708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:55.303708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:33:55.304203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:55.304203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:33:55.304203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:55.304203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:33:55.304203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:33:55.304203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:33:55.304203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:33:55.304203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:33:55.304203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:55.304203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:33:55.304203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:33:55.304203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:33:55.304203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:33:55.304203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:33:55.304203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:33:55.304795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:33:55.304795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:33:55.304795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:33:55.304795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:33:55.304795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:33:55.304795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:55.304795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:55.304795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:33:55.304795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:33:55.304795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:33:55.304795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:33:55.304795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:33:55.304795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:33:55.304795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:33:55.304795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:33:55.304795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:33:55.304795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:33:55.304795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:33:55.304795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:33:55.305435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:33:55.305435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:33:55.305435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:33:55.305435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:33:55.305435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:33:55.305435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:34:00.041762 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:34:00.042330 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:34:00.042330 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:34:00.043572 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:34:00.072129 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:00.072535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:00.073241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:34:00.073241 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:00.073241 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:34:00.073241 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:34:00.073241 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:00.073241 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:34:00.073241 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:00.073241 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:34:00.073918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.031 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:00.073918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:34:00.073918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:00.073918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:34:00.073918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:34:00.073918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:34:00.073918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:34:00.073918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:00.073918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:00.073918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:00.073918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:34:00.073918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:00.074819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:00.074819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:00.074819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:34:00.074819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:00.074819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:34:00.074819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:34:00.074819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:00.074819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:34:00.074819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:00.074819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:34:00.074819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:34:00.074819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:00.074819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:34:00.075549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:00.075549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:34:00.075549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:00.075549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:34:00.075549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:34:00.075549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:34:00.075549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:34:00.075549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:34:00.075549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:00.075549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:34:00.075549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:34:00.075549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:34:00.075549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:34:00.075549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:34:00.075549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:00.076818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:00.076818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:34:00.076818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:34:00.076818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:34:00.076818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:34:00.076818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:00.076818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:00.076818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:00.076818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:34:00.076818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:00.076818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:00.076818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:00.076818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:34:00.076818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:00.076818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:34:00.076818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:34:00.076818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:00.076818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:34:00.076818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:00.078100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:34:00.078100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:34:00.078100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:00.078100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:34:00.078100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:00.078100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:34:00.282969 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:34:00.283958 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:34:00.284095 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:34:00.285013 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:00.285308 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:00.285577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:00.286696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:34:00.286696 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:00.286696 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:34:00.286696 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:34:00.286696 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:00.286696 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:34:00.286696 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:00.286696 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:34:00.287948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:00.287948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:34:00.287948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:00.287948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:34:00.287948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:34:00.287948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:34:00.287948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:34:00.287948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:00.287948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:00.287948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:00.287948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:34:00.287948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:00.288583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:00.288583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:00.288583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:34:00.288583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:00.288583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:34:00.288583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:34:00.288583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:00.288583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:34:00.288583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:00.288583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:34:00.288583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:34:00.288583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:00.288583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:34:00.289123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:00.289123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:34:00.289581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:00.289581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:34:00.289581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:34:00.289581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:34:00.289581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:34:00.289581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:34:00.289581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:00.289581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:34:00.289581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:34:00.289581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:34:00.289581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:34:00.289581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:34:00.289581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:00.289581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:00.289581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:34:00.289581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:34:00.290323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:34:00.290323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:34:00.290323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:00.290323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:00.290323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:00.290323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:34:00.290323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:00.290323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:00.290323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:00.290323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:34:00.290323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:00.290323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:34:00.290323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:34:00.290323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:00.290323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:34:00.290323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:00.290323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:34:00.290323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:34:00.290323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:00.291365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:34:00.291365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:00.291365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:34:00.291365 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:34:00.291365 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:34:00.291365 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:34:00.291365 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:34:00.291365 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:00.293221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:00.293221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:34:00.293221 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:00.293221 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:34:00.293221 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:34:00.293221 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:00.293221 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:34:00.293221 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:00.293221 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:34:00.293221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:00.293221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:34:00.293221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:34:00.293221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:34:00.293221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:00.293221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:34:00.293221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:00.293221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:34:00.294115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:00.294115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:34:00.294115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:00.294115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:34:00.294115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:00.294115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:34:00.294115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:34:00.294115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:00.294115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:34:00.294115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:00.294115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:34:00.294115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:34:00.294115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:00.294115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:34:00.294115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:00.294115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:34:00.295017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:00.295017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:34:00.295017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:00.295017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:34:00.295017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:34:00.295017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:34:00.295017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:34:00.295017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:34:00.295017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:34:00.295017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:34:00.295017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:00.295017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:00.295295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:00.295295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:34:00.295295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:00.295295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:00.295295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:00.295295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:34:00.295295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:00.295295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:34:00.295295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:34:00.295295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:00.295295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:34:00.295295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:00.295295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:34:00.295295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:34:00.295590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:00.295590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:34:00.295590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:00.295590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:34:00.295590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:00.295590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:34:00.295590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:34:00.295590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:34:00.295590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:34:00.295590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:34:00.295590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:00.295590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:34:00.295590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:34:00.295590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:34:00.295879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:34:00.295879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:34:00.295879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:00.295879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:00.295879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:34:00.295879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:34:00.295879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:34:00.295879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:34:00.295879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:34:00.295879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:34:00.295879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:34:00.295879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:00.295879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:00.295879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:00.295879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:34:00.295879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:00.295879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:00.295879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:00.295879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:34:00.296269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:00.296269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:34:00.296269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:34:00.296269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:00.296269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:34:00.296269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:00.296269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:34:00.296269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:34:00.296269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:00.296269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:34:00.296269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:00.296269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:34:00.296269 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:34:00.296269 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:34:00.296728 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:34:00.296728 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:34:00.296938 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:00.297370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:00.297957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:34:00.297957 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:00.297957 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:34:00.297957 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:34:00.297957 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:00.297957 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:34:00.297957 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:00.297957 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:34:00.298792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:00.298792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:34:00.298792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:34:00.298792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:34:00.298792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:00.298792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:34:00.298792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:00.298792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:34:00.298792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:00.298792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:34:00.298792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:00.298792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:34:00.298792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:00.298792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:34:00.298792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:34:00.299112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:00.299112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:34:00.299112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:00.299112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:34:00.299112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:34:00.299112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:00.299112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:34:00.299112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:00.299112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:34:00.299776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:00.299776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:34:00.299776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:00.299776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:34:00.299776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:34:00.299776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:34:00.299776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:34:00.299776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:34:00.299776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:34:00.299776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:00.299776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:00.300044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:00.300044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:34:00.300044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:00.300044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:00.300044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:00.300044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:34:00.300044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:00.300044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:34:00.300044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:34:00.300044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:00.300044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:34:00.300044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:00.300044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:34:00.300044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:34:00.300320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:00.300320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:34:00.300320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:00.300320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:34:00.300834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:00.300834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:34:00.300834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:34:00.300834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:34:00.300834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:34:00.300834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:34:00.300834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:00.300834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:34:00.300834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:34:00.300834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:34:00.300834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:34:00.300834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:34:00.300834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:00.300834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:00.300834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:34:00.300834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:34:00.300834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:34:00.301190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:34:00.301190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:34:00.301190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:34:00.301190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:00.301190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:00.301190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:00.301190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:34:00.301190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:00.301190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:00.301190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:00.301190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:34:00.301190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:00.301190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:34:00.301190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:34:00.301190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:00.301190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:34:00.301190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:00.301190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:34:00.301190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:34:00.301564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:00.301564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:34:00.301564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:00.301564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:34:00.301564 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:34:00.301564 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:34:00.301564 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:34:00.301736 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:34:00.302007 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:00.302293 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:00.302451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:00.302920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:34:00.302920 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:00.302920 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:34:00.302920 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:34:00.302920 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:00.302920 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:34:00.302920 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:00.302920 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:34:00.303711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:00.303711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:34:00.303711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:00.303711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:34:00.303711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:34:00.303711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:34:00.303711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:34:00.303711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:00.303711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:00.303711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:00.303711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:34:00.303711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:00.304111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:00.304111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:00.304111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:34:00.304111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:00.304111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:34:00.304111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:34:00.304111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:00.304111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:34:00.304111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:00.304111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:34:00.304111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:34:00.304111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:00.304111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:34:00.304519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:00.304519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:34:00.304660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:00.304660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:34:00.304660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:34:00.304660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:34:00.304660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:34:00.304660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:34:00.304660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:00.304660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:34:00.304660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:34:00.304660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:34:00.304660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:34:00.304660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:34:00.304660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:00.304660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:00.304660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:34:00.304660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:34:00.305137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:34:00.305137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:34:00.305137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:00.305137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:00.305137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:00.305137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:34:00.305137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:00.305137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:00.305137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:00.305137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:34:00.305137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:00.305137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:34:00.305137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:34:00.305137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:00.305137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:34:00.305137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:00.305137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:34:00.305137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:34:00.305137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:00.305700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:34:00.305700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:00.305700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:34:05.042434 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:34:05.043352 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:34:05.043684 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:34:05.044986 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:34:05.078251 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:05.078552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:05.079153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:34:05.079153 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:05.079153 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:34:05.079153 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:34:05.079153 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:05.079153 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:34:05.079153 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:05.079153 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:34:05.080073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:05.080073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:34:05.080073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:05.080073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:34:05.080073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:34:05.080073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:34:05.080073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:34:05.080073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:05.080073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:05.080073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:05.080073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:34:05.080073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:05.080738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:05.080738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:05.080738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:34:05.080738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:05.080738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:34:05.080738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:34:05.080738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:05.080738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:34:05.080738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:05.080738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:34:05.080738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:34:05.080738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:05.080738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:34:05.081908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:05.081908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:34:05.081908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:05.081908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:34:05.081908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:34:05.081908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:34:05.081908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:34:05.081908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:34:05.081908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:05.081908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:34:05.081908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:34:05.081908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:34:05.081908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:34:05.081908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:34:05.081908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:05.082662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:05.082662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:34:05.082662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:34:05.082662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:34:05.082662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:34:05.082662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:05.082662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:05.082662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:05.082662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:34:05.082662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:05.082662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:05.082662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:05.082662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:34:05.082662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:05.082662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:34:05.082662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:34:05.082662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:05.082662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:34:05.082662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:05.083906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:34:05.083906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:34:05.083906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:05.083906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:34:05.083906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:05.083906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:34:05.283975 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:34:05.284758 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:34:05.285042 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:34:05.286117 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:05.286554 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:05.286816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:05.287963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:34:05.287963 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:05.287963 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:34:05.287963 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:34:05.287963 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:05.287963 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:34:05.287963 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:05.287963 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:34:05.289607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:05.289607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:34:05.289607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:05.289607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:34:05.289607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:34:05.289607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:34:05.289607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:34:05.289607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:05.289607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:05.289607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:05.289607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:34:05.289607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:05.290205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:05.290205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:05.290205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:34:05.290205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:05.290205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:34:05.290205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:34:05.290205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:05.290205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:34:05.290205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:05.290205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:34:05.290205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:34:05.290205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:05.290205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:34:05.291773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:05.291773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:34:05.291773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:05.291773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:34:05.291773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:34:05.291773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:34:05.291773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:34:05.291773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:34:05.291773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:05.291773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:34:05.291773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:34:05.291773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:34:05.291773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:34:05.291773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:34:05.291773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:05.293126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:05.293126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:34:05.293126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:34:05.293126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:34:05.293126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:34:05.293126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:05.293126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:05.293126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:05.293126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:34:05.293126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:05.293126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:05.293126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:05.293126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:34:05.293126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:05.293126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:34:05.293126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:34:05.293126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:05.293126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:34:05.293126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:05.294240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:34:05.294240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:34:05.294240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:05.294240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:34:05.294240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:05.294240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:34:05.294240 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:34:05.294240 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:34:05.294240 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:34:05.294240 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:34:05.294952 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:05.294952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:05.294952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:34:05.294952 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:05.294952 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:34:05.294952 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:34:05.294952 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:05.294952 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:34:05.294952 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:05.294952 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:34:05.294952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:05.294952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:34:05.294952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:34:05.294952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:34:05.295599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:05.295599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:34:05.295599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:05.295599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:34:05.295599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:05.295599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:34:05.295599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:05.295599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:34:05.295599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:05.295599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:34:05.295599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:34:05.295599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:05.295599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:34:05.295599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:05.295599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:34:05.295599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:34:05.295599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:05.295599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:34:05.296268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:05.296268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:34:05.296268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:05.296268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:34:05.296268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:05.296268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:34:05.296268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:34:05.296268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:34:05.296268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:34:05.296268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:34:05.296268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:34:05.296268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:34:05.296689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:05.296689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:05.296689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:05.296689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:34:05.296689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:05.296689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:05.296689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:05.296689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:34:05.296689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:05.296689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:34:05.296689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:34:05.296689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:05.296689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:34:05.296689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:05.297158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:34:05.297158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:34:05.297158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:05.297158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:34:05.297158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:05.297158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:34:05.297388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:05.297388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:34:05.297388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:34:05.297388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:34:05.297388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:34:05.297388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:34:05.297388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:05.297388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:34:05.297388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:34:05.297388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:34:05.297388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:34:05.297388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:34:05.297388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:05.297388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:05.297388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:34:05.297388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:34:05.299198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:34:05.299198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:34:05.299198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:34:05.299198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:34:05.299198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:34:05.299198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:05.299198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:05.299198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:05.299198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:34:05.299198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:05.299198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:05.299198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:05.299198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:34:05.299198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:05.299198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:34:05.299198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:34:05.299198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:05.299198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:34:05.300566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:05.300566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:34:05.300566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:34:05.300566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:05.300566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:34:05.300566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:05.300566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:34:05.300566 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:34:05.300566 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:34:05.300566 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:34:05.301199 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:34:05.301199 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:05.301199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:05.301199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:34:05.301199 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:05.301199 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:34:05.301199 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:34:05.301199 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:05.301199 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:34:05.301199 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:05.301199 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:34:05.301199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:05.301689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:34:05.301689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:34:05.301689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:34:05.301689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:05.301689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:34:05.301689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:05.301689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:34:05.301689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:05.301689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:34:05.301689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:05.301689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:34:05.301689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:05.301689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:34:05.301689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:34:05.301689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:05.301689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:34:05.301689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:05.302058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:34:05.302058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:34:05.302058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:05.302058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:34:05.302058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:05.302058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:34:05.302058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:05.302058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:34:05.302058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:05.302058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:34:05.302058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:34:05.302058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:34:05.302058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:34:05.302328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:34:05.302328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:34:05.302328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:05.302328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:05.302328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:05.302328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:34:05.302328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:05.302328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:05.302328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:05.302328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:34:05.302328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:05.302328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:34:05.302328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:34:05.302806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:05.302806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:34:05.302806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:05.302806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:34:05.302806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:34:05.302806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:05.302806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:34:05.302806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:05.302806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:34:05.302806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:05.302806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:34:05.302806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:34:05.303035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:34:05.303035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:34:05.303035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:34:05.303035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:05.303035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:34:05.303035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:34:05.303035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:34:05.303035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:34:05.303035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:34:05.303035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:05.303035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:05.303035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:34:05.303035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:34:05.303035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:34:05.303035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:34:05.303035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:34:05.303035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:34:05.303035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:05.303035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:05.303366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:05.303366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:34:05.303366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:05.303366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:05.303366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:05.303366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:34:05.303366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:05.303366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:34:05.303366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:34:05.303366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:05.303366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:34:05.303366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:05.303366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:34:05.303366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:34:05.303366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:05.303366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:34:05.303366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:05.303366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:34:05.303810 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:34:05.303810 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:34:05.303810 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:34:05.303810 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:34:05.303810 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:05.303810 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:05.304105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:05.304223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:34:05.304223 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:05.304223 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:34:05.304223 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:34:05.304223 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:05.304223 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:34:05.304223 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:05.304223 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:34:05.305037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:05.305037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:34:05.305037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:05.305037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:34:05.305037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:34:05.305037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:34:05.305037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:34:05.305037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:05.305037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:05.305037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:05.305037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:34:05.305037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:05.305546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:05.305546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:05.305546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:34:05.305546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:05.305546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:34:05.305546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:34:05.305546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:05.305546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:34:05.305546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:05.305546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:34:05.305546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:34:05.305546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:05.305546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:34:05.306031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:05.306031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:34:05.306031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:05.306031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:34:05.306031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:34:05.306031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:34:05.306031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:34:05.306031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:34:05.306031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:05.306031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:34:05.306031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:34:05.306031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:34:05.306031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:34:05.306031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:34:05.306031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:05.306358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:05.306358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:34:05.306358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:34:05.306358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:34:05.306358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:34:05.306358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:05.306358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:05.306358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:05.306358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:34:05.306358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:05.306358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:05.306358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:05.306358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:34:05.306358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:05.306358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:34:05.306358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:34:05.306358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:05.306358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:34:05.306358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:05.306727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:34:05.306727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:34:05.306727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:05.306727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:34:05.306727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:05.306727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:34:10.043371 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:34:10.044037 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:34:10.044307 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:34:10.045713 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:34:10.079719 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:10.080115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:10.080965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:34:10.080965 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:10.080965 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:34:10.080965 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:34:10.080965 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:10.080965 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:34:10.080965 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:10.080965 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:34:10.081748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:10.081748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:34:10.081748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:10.081748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:34:10.081748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:34:10.081748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:34:10.081748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:34:10.081748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:10.081748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:10.081748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:10.081748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:34:10.081748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:10.082393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:10.082393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:10.082393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:34:10.082393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:10.082393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:34:10.082393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:34:10.082393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:10.082393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:34:10.082393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:10.082393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:34:10.082393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:34:10.082393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:10.082393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:34:10.083431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:10.083431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:34:10.083431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:10.083431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:34:10.083431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:34:10.083431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:34:10.083431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:34:10.083431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:34:10.083431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:10.083431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:34:10.083431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:34:10.083431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:34:10.083431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:34:10.083431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:34:10.083431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:10.084276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:10.084276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:34:10.084276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:34:10.084276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:34:10.084276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:34:10.084276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:10.084276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:10.084276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:10.084276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:34:10.084276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:10.084276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:10.084276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:10.084276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:34:10.084276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:10.084276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:34:10.084276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:34:10.084276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:10.084276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:34:10.084276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:10.085318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:34:10.085318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:34:10.085318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:10.085318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:34:10.085318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:10.085318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:34:10.284681 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:34:10.285086 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:34:10.285196 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:34:10.286109 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:10.286518 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:10.286759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:10.287521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:34:10.287521 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:10.287521 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:34:10.287521 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:34:10.287521 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:10.287521 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:34:10.287521 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:10.287521 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:34:10.288548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:10.288548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:34:10.288548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:10.288548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:34:10.288548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:34:10.288548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:34:10.288548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:34:10.288548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:10.288548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:10.288548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:10.288548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:34:10.288548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:10.289152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:10.289152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:10.289152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:34:10.289152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:10.289152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:34:10.289152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:34:10.289152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:10.289152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:34:10.289152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:10.289152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:34:10.289152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:34:10.289152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:10.289152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:34:10.289810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:10.289810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:34:10.289810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:10.289810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:34:10.289810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:34:10.289810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:34:10.289810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:34:10.289810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:34:10.289810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:10.289810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:34:10.289810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:34:10.289810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:34:10.289810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:34:10.289810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:34:10.289810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:10.290520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:10.290520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:34:10.290520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:34:10.290520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:34:10.290520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:34:10.290520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:10.290520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:10.290520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:10.290520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:34:10.290520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:10.290520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:10.290520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:10.290520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:34:10.290520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:10.290520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:34:10.290520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:34:10.290520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:10.290520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:34:10.290520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:10.291368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:34:10.291368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:34:10.291368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:10.291368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:34:10.291368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:10.291368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:34:10.292383 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:34:10.292697 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:34:10.292792 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:34:10.292920 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:34:10.293362 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:10.293774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:10.294367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:34:10.294367 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:10.294367 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:34:10.294367 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:34:10.294367 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:10.294367 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:34:10.294367 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:10.294367 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:34:10.295102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:10.295102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:34:10.295102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:34:10.295102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:34:10.295102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:10.295102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:34:10.295102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:10.295102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:34:10.295102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:10.295102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:34:10.295102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:10.295102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:34:10.295102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:10.295102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:34:10.295102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:34:10.295569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:10.295569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:34:10.295569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:10.295569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:34:10.295569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:34:10.295569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:10.295569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:34:10.295569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:10.295569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:34:10.295899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:10.295899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:34:10.295899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:10.295899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:34:10.295899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:34:10.295899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:34:10.295899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:34:10.295899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:34:10.295899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:34:10.295899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:34:10.295899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:10.295899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:10.296243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:10.296243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:34:10.296243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:10.296243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:10.296243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:10.296243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:34:10.296243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:10.296243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:34:10.296243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:34:10.296243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:10.296243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:34:10.296243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:10.296243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:34:10.296243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:34:10.296650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:10.296650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:34:10.296650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:10.296650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:34:10.296891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:10.296891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:34:10.296891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:34:10.296891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:34:10.296891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:34:10.296891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:34:10.296891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:10.296891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:34:10.296891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:34:10.296891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:34:10.296891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:34:10.296891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:34:10.296891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:10.296891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:10.296891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:34:10.296891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:34:10.297316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:34:10.297316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:34:10.297316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:34:10.297316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:34:10.297316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:34:10.297316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:10.297316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:10.297316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:10.297316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:34:10.297316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:10.297316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:10.297316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:10.297316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:34:10.297316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:10.297316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:34:10.297316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:34:10.297316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:10.297316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:34:10.297833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:10.297833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:34:10.297833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:34:10.297833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:10.297833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:34:10.297833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:10.297833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:34:10.298222 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:34:10.298494 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:34:10.298599 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:34:10.298721 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:34:10.299167 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:10.299539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:10.300212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:34:10.300212 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:10.300212 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:34:10.300212 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:34:10.300212 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:10.300212 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:34:10.300212 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:10.300212 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:34:10.300973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:10.300973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:34:10.300973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:34:10.300973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:34:10.300973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:10.300973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:34:10.300973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:10.300973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:34:10.300973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:10.300973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:34:10.300973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:10.300973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:34:10.300973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:10.300973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:34:10.300973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:34:10.301359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:10.301359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:34:10.301359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:10.301359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:34:10.301359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:34:10.301359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:10.301359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:34:10.301359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:10.301359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:34:10.301836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:10.301836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:34:10.301836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:10.301836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:34:10.301836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:34:10.301836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:34:10.301836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:34:10.301836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:34:10.301836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:34:10.301836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:10.301836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:10.302173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:10.302173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:34:10.302173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:10.302173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:10.302173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:10.302173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:34:10.302173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:10.302173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:34:10.302173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:34:10.302173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:10.302173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:34:10.302173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:10.302173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:34:10.302173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:34:10.302576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:10.302576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:34:10.302576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:10.302576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:34:10.303000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:10.303000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:34:10.303000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:34:10.303000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:34:10.303000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:34:10.303000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:34:10.303000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:10.303000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:34:10.303000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:34:10.303000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:34:10.303000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:34:10.303000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:34:10.303000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:10.303000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:10.303000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:34:10.303000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:34:10.303000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:34:10.303506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:34:10.303506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:34:10.303506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:34:10.303506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:10.303506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:10.303506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:10.303506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:34:10.303506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:10.303506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:10.303506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:10.303506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:34:10.303506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:10.303506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:34:10.303506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:34:10.303506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:10.303506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:34:10.303506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:10.303506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:34:10.303506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:34:10.304018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:10.304018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:34:10.304018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:10.304018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:34:10.304018 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:34:10.304018 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:34:10.304018 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:34:10.304018 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:34:10.304270 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:10.304527 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:10.304738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:10.305390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:34:10.305390 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:10.305390 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:34:10.305390 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:34:10.305390 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:10.305390 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:34:10.305390 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:10.305390 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:34:10.306218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:10.306218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:34:10.306218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:10.306218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:34:10.306218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:34:10.306218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:34:10.306218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:34:10.306218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:10.306218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:10.306218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:10.306218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:34:10.306218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:10.306820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:10.306820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:10.306820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:34:10.306820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:10.306820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:34:10.306820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:34:10.306820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:10.306820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:34:10.306820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:10.306820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:34:10.306820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:34:10.306820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:10.306820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:34:10.307197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:10.307197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:34:10.307197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:10.307197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:34:10.307197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:34:10.307197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:34:10.307197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:34:10.307197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:34:10.307197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:10.307197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:34:10.307197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:34:10.307197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:34:10.307197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:34:10.307197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:34:10.307197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:10.308228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:10.308228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:34:10.308228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:34:10.308228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:34:10.308228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:34:10.308228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:10.308228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:10.308228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:10.308228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:34:10.308228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:10.308228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:10.308228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:10.308228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:34:10.308228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:10.308228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:34:10.308228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:34:10.308228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:10.308228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:34:10.308228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:10.308782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:34:10.308782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:34:10.308782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:10.308782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:34:10.308782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:10.308782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:34:15.044268 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:34:15.045286 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:34:15.045286 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:34:15.046530 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:34:15.087235 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:15.087681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:15.088670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:34:15.088670 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:15.088670 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:34:15.088670 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:34:15.088670 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:15.088670 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:34:15.088670 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:15.088670 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:34:15.089695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:15.089695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:34:15.089695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:15.089695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:34:15.089695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:34:15.089695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:34:15.089695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:34:15.089695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:15.089695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:15.089695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:15.089695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:34:15.089695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:15.090335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:15.090335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:15.090335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:34:15.090335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:15.090335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:34:15.090335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:34:15.090335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:15.090335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:34:15.090335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:15.090335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:34:15.090335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:34:15.090335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:15.090335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:34:15.091055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:15.091055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:34:15.091055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:15.091055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:34:15.091055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:34:15.091055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:34:15.091055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:34:15.091055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:34:15.091055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:15.091055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:34:15.091055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:34:15.091055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:34:15.091055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:34:15.091055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:34:15.091055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:15.091934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:15.091934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:34:15.091934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:34:15.091934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:34:15.091934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:34:15.091934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:15.091934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:15.091934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:15.091934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:34:15.091934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:15.091934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:15.091934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:15.091934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:34:15.091934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:15.091934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:34:15.091934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:34:15.091934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:15.091934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:34:15.091934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:15.092761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:34:15.092761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:34:15.092761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:15.092761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:34:15.092761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:15.092761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:34:15.285325 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:34:15.285958 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:34:15.286147 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:34:15.287052 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:15.287251 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:15.287358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:15.287970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:34:15.287970 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:15.287970 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:34:15.287970 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:34:15.287970 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:15.287970 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:34:15.287970 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:15.287970 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:34:15.289643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:15.289643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:34:15.289643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:15.289643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:34:15.289643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:34:15.289643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:34:15.289643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:34:15.289643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:15.289643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:15.289643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:15.289643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:34:15.289643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:15.290209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:15.290209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:15.290209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:34:15.290209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:15.290209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:34:15.290209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:34:15.290209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:15.290209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:34:15.290209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:15.290209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:34:15.290209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:34:15.290209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:15.290209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:34:15.290796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:15.290796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:34:15.291376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:15.291376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:34:15.291376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:34:15.291376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:34:15.291376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:34:15.291376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:34:15.291376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:15.291376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:34:15.291376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:34:15.291376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:34:15.291376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:34:15.291376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:34:15.291376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:15.291376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:15.291376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:34:15.291376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:34:15.292116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:34:15.292116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:34:15.292116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:15.292116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:15.292116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:15.292116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:34:15.292116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:15.292116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:15.292116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:15.292116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:34:15.292116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:15.292116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:34:15.292116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:34:15.292116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:15.292116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:34:15.292116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:15.292116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:34:15.292116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:34:15.292116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:15.292947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:34:15.292947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:15.292947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:34:15.292947 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:34:15.293156 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:34:15.293353 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:34:15.293552 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:34:15.294028 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:15.294439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:15.295065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:34:15.295065 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:15.295065 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:34:15.295065 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:34:15.295065 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:15.295065 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:34:15.295065 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:15.295065 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:34:15.295806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:15.295806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:34:15.295806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:34:15.295806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:34:15.295806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:15.295806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:34:15.295806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:15.295806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:34:15.295806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:15.295806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:34:15.295806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:15.295806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:34:15.295806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:15.295806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:34:15.295806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:34:15.296647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:15.296647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:34:15.296647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:15.296647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:34:15.296647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:34:15.296647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:15.296647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:34:15.296647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:15.296647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:34:15.296647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:15.296647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:34:15.296647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:15.296647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:34:15.296647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:34:15.297337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:34:15.297337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:34:15.297337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:34:15.297337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:34:15.297337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:34:15.297337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:15.297337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:15.297337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:15.297337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:34:15.297337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:15.297337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:15.297337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:15.297337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:34:15.297838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:15.297838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:34:15.297838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:34:15.297838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:15.297838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:34:15.297838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:15.297838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:34:15.297838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:34:15.297838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:15.297838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:34:15.297838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:15.297838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:34:15.298176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:15.298176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:34:15.298176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:34:15.298176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:34:15.298176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:34:15.298176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:34:15.298176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:15.298176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:34:15.298176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:34:15.298176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:34:15.298176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:34:15.298176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:34:15.298176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:15.298176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:15.298176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:34:15.298176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:34:15.298659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:34:15.298659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:34:15.298659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:34:15.298659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:34:15.298659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:34:15.298659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:15.298659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:15.298659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:15.298659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:34:15.298659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:15.298659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:15.298659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:15.298659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:34:15.298659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:15.298659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:34:15.298659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:34:15.298659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:15.298659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:34:15.299159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:15.299159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:34:15.299159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:34:15.299159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:15.299159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:34:15.299159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:15.299159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:34:15.299159 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:34:15.299159 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:34:15.299159 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:34:15.299497 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:34:15.299671 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:15.299957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:15.300595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:34:15.300595 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:15.300595 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:34:15.300595 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:34:15.300595 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:15.300595 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:34:15.300595 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:15.300595 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:34:15.301318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:15.301318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:34:15.301318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:34:15.301318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:34:15.301318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:15.301318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:34:15.301318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:15.301318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:34:15.301318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:15.301318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:34:15.301318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:15.301318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:34:15.301318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:15.301318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:34:15.301318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:34:15.301753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:15.301753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:34:15.301753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:15.301753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:34:15.301753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:34:15.301753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:15.301753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:34:15.301753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:15.301753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:34:15.302191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:15.302191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:34:15.302191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:15.302191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:34:15.302191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:34:15.302191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:34:15.302191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:34:15.302191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:34:15.302191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:34:15.302191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:15.302191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:15.302540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:15.302540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:34:15.302540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:15.302540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:15.302540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:15.302540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:34:15.302540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:15.302540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:34:15.302540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:34:15.302540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:15.302540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:34:15.302540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:15.302540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:34:15.302540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:34:15.302918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:15.302918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:34:15.302918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:15.302918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:34:15.303303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:15.303303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:34:15.303303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:34:15.303303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:34:15.303303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:34:15.303303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:34:15.303303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:15.303303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:34:15.303303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:34:15.303303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:34:15.303303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:34:15.303303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:34:15.303303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:15.303303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:15.303303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:34:15.303303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:34:15.303303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:34:15.303803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:34:15.303803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:34:15.303803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:34:15.303803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:15.303803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:15.303803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:15.303803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:34:15.303803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:15.303803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:15.303803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:15.303803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:34:15.303803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:15.303803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:34:15.303803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:34:15.303803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:15.303803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:34:15.303803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:15.303803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:34:15.303803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:34:15.304288 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:15.304288 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:34:15.304288 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:15.304288 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:34:15.304288 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:34:15.304288 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:34:15.304288 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:34:15.304288 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:34:15.304593 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:15.304812 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:15.304982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:15.305519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:34:15.305519 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:15.305519 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:34:15.305519 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:34:15.305519 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:15.305519 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:34:15.305519 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:15.305519 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:34:15.306351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:15.306351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:34:15.306351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:15.306351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:34:15.306351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:34:15.306351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:34:15.306351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:34:15.306351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:15.306351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:15.306351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:15.306351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:34:15.306351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:15.306734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:15.306734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:15.306734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:34:15.306734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:15.306734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:34:15.306734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:34:15.306734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:15.306734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:34:15.306734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:15.306734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:34:15.306734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:34:15.306734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:15.306734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:34:15.307085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:15.307085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:34:15.307354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:15.307354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:34:15.307354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:34:15.307354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:34:15.307354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:34:15.307354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:34:15.307354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:15.307354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:34:15.307354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:34:15.307354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:34:15.307354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:34:15.307354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:34:15.307354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:15.307354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:15.307354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:34:15.307354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:34:15.307842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:34:15.307842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:34:15.307842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:15.307842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:15.307842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:15.307842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:34:15.307842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:15.307842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:15.307842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:15.307842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:34:15.307842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:15.307842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:34:15.307842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:34:15.307842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:15.307842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:34:15.307842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:15.307842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:34:15.307842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:34:15.307842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:15.308346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:34:15.308346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:15.308346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:34:20.045177 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:34:20.046065 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:34:20.046065 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:34:20.047233 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:34:20.079531 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:20.079970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:20.080832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:34:20.080832 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:20.080832 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:34:20.080832 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:34:20.080832 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:20.080832 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:34:20.080832 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:20.080832 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:34:20.081625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:20.081625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:34:20.081625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:20.081625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:34:20.081625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:34:20.081625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:34:20.081625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:34:20.081625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:20.081625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:20.081625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:20.081625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:34:20.081625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:20.082250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:20.082250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:20.082250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:34:20.082250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:20.082250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:34:20.082250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:34:20.082250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:20.082250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:34:20.082250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:20.082250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:34:20.082250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:34:20.082250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:20.082250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:34:20.082826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:20.082826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:34:20.082826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:20.082826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:34:20.082826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:34:20.082826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:34:20.082826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:34:20.082826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:34:20.082826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:20.082826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:34:20.082826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:34:20.082826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:34:20.082826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:34:20.082826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:34:20.082826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:20.083519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:20.083519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:34:20.083519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:34:20.083519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:34:20.083519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:34:20.083519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:20.083519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:20.083519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:20.083519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:34:20.083519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:20.083519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:20.083519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:20.083519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:34:20.083519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:20.083519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:34:20.083519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:34:20.083519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:20.083519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:34:20.083519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:20.084370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:34:20.084370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:34:20.084370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:20.084370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:34:20.084370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:20.084370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:34:20.286132 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:34:20.286801 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:34:20.286990 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:34:20.288135 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:20.288527 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:20.288787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:20.289685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:34:20.289685 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:20.289685 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:34:20.289685 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:34:20.289685 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:20.289685 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:34:20.289685 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:20.289685 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:34:20.291254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:20.291254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:34:20.291254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:20.291254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:34:20.291254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:34:20.291254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:34:20.291254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:34:20.291254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:20.291254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:20.291254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:20.291254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:34:20.291254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:20.291900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:20.291900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:20.291900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:34:20.291900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:20.291900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:34:20.291900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:34:20.291900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:20.291900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:34:20.291900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:20.291900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:34:20.291900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:34:20.291900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:20.291900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:34:20.292481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:20.292481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:34:20.293085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:20.293085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:34:20.293085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:34:20.293085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:34:20.293085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:34:20.293085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:34:20.293085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:20.293085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:34:20.293085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:34:20.293085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:34:20.293085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:34:20.293085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:34:20.293085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:20.293085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:20.293085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:34:20.293085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:34:20.293948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:34:20.293948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:34:20.293948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:20.293948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:20.293948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:20.293948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:34:20.293948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:20.293948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:20.293948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:20.293948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:34:20.293948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:20.293948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:34:20.293948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:34:20.293948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:20.293948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:34:20.293948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:20.293948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:34:20.293948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:34:20.293948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:20.294952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:34:20.294952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:20.294952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:34:20.294952 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:34:20.294952 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:34:20.294952 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:34:20.294952 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:34:20.294952 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:20.295620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:20.295875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:34:20.295875 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:20.295875 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:34:20.295875 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:34:20.295875 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:20.295875 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:34:20.295875 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:20.295875 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:34:20.297177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:20.297177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:34:20.297177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:34:20.297177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:34:20.297177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:20.297177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:34:20.297177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:20.297177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:34:20.297177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:20.297177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:34:20.297177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:20.297177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:34:20.297177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:20.297177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:34:20.297177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:34:20.297894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:20.297894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:34:20.297894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:20.297894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:34:20.297894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:34:20.297894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:20.297894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:34:20.297894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:20.297894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:34:20.298715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:20.298715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:34:20.298715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:20.298715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:34:20.298715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:34:20.298715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:34:20.298715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:34:20.298715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:34:20.298715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:34:20.298715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:34:20.298715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:20.298715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:20.299462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:20.299462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:34:20.299462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:20.299462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:20.299462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:20.299462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:34:20.299462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:20.299462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:34:20.299462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:34:20.299462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:20.299462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:34:20.299462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:20.299462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:34:20.299462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:34:20.300512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:20.300512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:34:20.300512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:20.300512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:34:20.300782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:20.300782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:34:20.300782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:34:20.300782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:34:20.300782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:34:20.300782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:34:20.300782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:20.300782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:34:20.300782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:34:20.300782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:34:20.300782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:34:20.300782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:34:20.300782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:20.300782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:20.300782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:34:20.300782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:34:20.301659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:34:20.301659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:34:20.301659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:34:20.301659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:34:20.301659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:34:20.301659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:20.301659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:20.301659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:20.301659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:34:20.301659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:20.301659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:20.301659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:20.301659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:34:20.301659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:20.301659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:34:20.301659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:34:20.301659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:20.301659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:34:20.302625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:20.302625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:34:20.302625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:34:20.302625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:20.302625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:34:20.302625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:20.302625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:34:20.302625 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:34:20.302625 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:34:20.302625 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:34:20.303082 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:34:20.303082 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:20.303082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:20.303475 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:34:20.303475 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:20.303475 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:34:20.303475 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:34:20.303475 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:20.303475 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:34:20.303475 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:20.303475 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:34:20.304798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:20.304798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:34:20.304798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:34:20.304798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:34:20.304798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:20.304798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:34:20.304798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:20.304798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:34:20.304798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:20.304798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:34:20.304798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:20.304798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:34:20.304798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:20.304798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:34:20.304798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:34:20.305320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:20.305320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:34:20.305320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:20.305320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:34:20.305320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:34:20.305320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:20.305320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:34:20.305320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:20.305320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:34:20.305320 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:34:20.306567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:20.306567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:34:20.306567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:20.306567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:34:20.306567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:34:20.306567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:34:20.306567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:34:20.306567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:34:20.306567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:34:20.306567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:20.306567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:20.306927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:20.306927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:34:20.306927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:20.306927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:20.306927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:20.306927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:34:20.306927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:20.306927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:34:20.306927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:34:20.306927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:20.306927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:34:20.306927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:20.306927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:34:20.306927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:34:20.307353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:20.307353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:34:20.307353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:20.307353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:34:20.308294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:20.308294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:34:20.308294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:34:20.308294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:34:20.308294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:34:20.308294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:34:20.308294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:20.308294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:34:20.308294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:34:20.308294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:34:20.308294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:34:20.308294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:34:20.308294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:20.308294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:20.308294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:34:20.308294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:34:20.308294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:34:20.308893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:34:20.308893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:34:20.308893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:34:20.308893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:20.308893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:20.308893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:20.308893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:34:20.308893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:20.308893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:20.308893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:20.308893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:34:20.308893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:20.308893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:34:20.308893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:34:20.308893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:20.308893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:34:20.308893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:20.308893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:34:20.308893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:34:20.310500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:20.310500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:34:20.310500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:20.310500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:34:20.310500 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:34:20.310500 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:34:20.310500 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:34:20.310500 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:20.310889 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:20.310889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:20.310889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:34:20.310889 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:20.310889 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:34:20.310889 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:34:20.310889 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:20.310889 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:34:20.310889 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:20.310889 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:34:20.311255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:20.311255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:34:20.311255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:20.311255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:34:20.311255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:34:20.311255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:34:20.311255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:34:20.311255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:20.311255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:20.311255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:20.311255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:34:20.311255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:20.311620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:20.311620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:20.311620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:34:20.311620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:20.311620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:34:20.311620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:34:20.311620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:20.311620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:34:20.311620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:20.311620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:34:20.311620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:34:20.311620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:20.311620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:34:20.311954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:20.311954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:34:20.311954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:20.311954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:34:20.311954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:34:20.311954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:34:20.311954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:34:20.311954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:34:20.311954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:20.311954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:34:20.311954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:34:20.311954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:34:20.311954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:34:20.311954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:34:20.311954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:20.312382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:20.312382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:34:20.312382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:34:20.312382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:34:20.312382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:34:20.312382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:20.312382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:20.312382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:20.312382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:34:20.312382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:20.312382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:20.312382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:20.312382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:34:20.312382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:20.312382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:34:20.312382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:34:20.312382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:20.312382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:34:20.312382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:20.312936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:34:20.312936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:34:20.312936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:20.312936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:34:20.312936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:20.312936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:34:25.045861 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:34:25.048018 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:34:25.048018 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:34:25.048018 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:34:25.049915 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG dbcounter [-] [105581] Writing DB stats neutron:SELECT=240 {{(pid=105581) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 14:34:25.076057 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:25.076472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:25.077727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:34:25.077727 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:25.077727 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:34:25.077727 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:34:25.077727 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:25.077727 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:34:25.077727 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:25.077727 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:34:25.079135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.031 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:25.079135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:34:25.079135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:25.079135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:34:25.079135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:34:25.079135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:34:25.079135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:34:25.079135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:25.079135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:25.079135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:25.079135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:34:25.079135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:25.079891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:25.079891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:25.079891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:34:25.079891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:25.079891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:34:25.079891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:34:25.079891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:25.079891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:34:25.079891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:25.079891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:34:25.079891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:34:25.079891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:25.079891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:34:25.080459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:25.080459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:34:25.080883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:25.080883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:34:25.080883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:34:25.080883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:34:25.080883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:34:25.080883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:34:25.080883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:25.080883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:34:25.080883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:34:25.080883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:34:25.080883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:34:25.080883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:34:25.080883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:25.080883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:25.080883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:34:25.080883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:34:25.081709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:34:25.081709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:34:25.081709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:25.081709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:25.081709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:25.081709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:34:25.081709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:25.081709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:25.081709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:25.081709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:34:25.081709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:25.081709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:34:25.081709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:34:25.081709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:25.081709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:34:25.081709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:25.081709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:34:25.081709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:34:25.081709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:25.082502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:34:25.082502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:25.082502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:34:25.286781 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:34:25.287188 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:34:25.287374 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:34:25.288323 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:25.288656 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:25.288803 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:25.289502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:34:25.289502 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:25.289502 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:34:25.289502 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:34:25.289502 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:25.289502 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:34:25.289502 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:25.289502 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:34:25.290877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:25.290877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:34:25.290877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:25.290877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:34:25.290877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:34:25.290877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:34:25.290877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:34:25.290877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:25.290877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:25.290877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:25.290877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:34:25.290877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:25.291486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:25.291486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:25.291486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:34:25.291486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:25.291486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:34:25.291486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:34:25.291486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:25.291486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:34:25.291486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:25.291486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:34:25.291486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:34:25.291486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:25.291486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:34:25.292019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:25.292019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:34:25.292616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:25.292616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:34:25.292616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:34:25.292616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:34:25.292616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:34:25.292616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:34:25.292616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:25.292616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:34:25.292616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:34:25.292616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:34:25.292616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:34:25.292616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:34:25.292616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:25.292616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:25.292616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:34:25.292616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:34:25.293342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:34:25.293342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:34:25.293342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:25.293342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:25.293342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:25.293342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:34:25.293342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:25.293342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:25.293342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:25.293342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:34:25.293342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:25.293342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:34:25.293342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:34:25.293342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:25.293342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:34:25.293342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:25.293342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:34:25.293342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:34:25.293342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:25.294156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:34:25.294156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:25.294156 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:34:25.294156 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:34:25.294491 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:34:25.294876 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:34:25.294876 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:34:25.295513 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:25.295938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:25.296641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:34:25.296641 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:25.296641 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:34:25.296641 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:34:25.296641 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:25.296641 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:34:25.296641 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:25.296641 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:34:25.297872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:25.297872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:34:25.297872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:34:25.297872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:34:25.297872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:25.297872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:34:25.297872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:25.297872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:34:25.297872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:25.297872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:34:25.297872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:25.297872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:34:25.297872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:25.297872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:34:25.297872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:34:25.298589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:25.298589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:34:25.298589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:25.298589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:34:25.298589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:34:25.298589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:25.298589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:34:25.298589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:25.298589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:34:25.298589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:25.298589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:34:25.298589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:25.298589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:34:25.298589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:34:25.299187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:34:25.299187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:34:25.299187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:34:25.299187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:34:25.299187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:34:25.299187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:25.299187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:25.299187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:25.299187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:34:25.299187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:25.299187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:25.299187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:25.299187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:34:25.299781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:25.299781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:34:25.299781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:34:25.299781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:25.299781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:34:25.299781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:25.299781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:34:25.299781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:34:25.299781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:25.299781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:34:25.299781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:25.299781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:34:25.300461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:25.300461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:34:25.300461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:34:25.300461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:34:25.300461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:34:25.300461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:34:25.300461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:25.300461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:34:25.300461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:34:25.300461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:34:25.300461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:34:25.300461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:34:25.300461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:25.300461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:25.300461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:34:25.300461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:34:25.301159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:34:25.301159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:34:25.301159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:34:25.301159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:34:25.301159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:34:25.301159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:25.301159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:25.301159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:25.301159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:34:25.301159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:25.301159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:25.301159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:25.301159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:34:25.301159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:25.301159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:34:25.301159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:34:25.301159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:25.301159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:34:25.302157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:25.302157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:34:25.302157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:34:25.302157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:25.302157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:34:25.302157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:25.302157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:34:25.302157 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:34:25.302157 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:34:25.302157 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:34:25.302516 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:34:25.302516 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:25.302516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:25.302985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:34:25.302985 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:25.302985 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:34:25.302985 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:34:25.302985 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:25.302985 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:34:25.302985 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:25.302985 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:34:25.303689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:25.303689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:34:25.303689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:34:25.303689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:34:25.303689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:25.303689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:34:25.303689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:25.303689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:34:25.303689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:25.303689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:34:25.303689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:25.303689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:34:25.303689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:25.303689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:34:25.303689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:34:25.304180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:25.304180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:34:25.304180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:25.304180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:34:25.304180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:34:25.304180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:25.304180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:34:25.304180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:25.304180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:34:25.304510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:25.304510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:34:25.304510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:25.304510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:34:25.304510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:34:25.304510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:34:25.304510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:34:25.304510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:34:25.304510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:34:25.304510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:25.304510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:25.304866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:25.304866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:34:25.304866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:25.304866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:25.304866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:25.304866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:34:25.304866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:25.304866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:34:25.304866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:34:25.304866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:25.304866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:34:25.304866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:25.304866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:34:25.304866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:34:25.305145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:25.305145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:34:25.305145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:25.305145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:34:25.305547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:25.305547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:34:25.305547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:34:25.305547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:34:25.305547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:34:25.305547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:34:25.305547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:25.305547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:34:25.305547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:34:25.305547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:34:25.305547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:34:25.305547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:34:25.305547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:25.305547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:25.305547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:34:25.305547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:34:25.305547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:34:25.305913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:34:25.305913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:34:25.305913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:34:25.305913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:25.305913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:25.305913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:25.305913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:34:25.305913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:25.305913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:25.305913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:25.305913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:34:25.305913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:25.305913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:34:25.305913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:34:25.305913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:25.305913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:34:25.305913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:25.305913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:34:25.305913 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:34:25.306283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:25.306283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:34:25.306283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:25.306283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:34:25.308678 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:34:25.308936 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:34:25.309039 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:34:25.309177 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:34:25.309690 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:25.310050 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:25.310271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:25.310999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:34:25.310999 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:25.310999 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:34:25.310999 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:34:25.310999 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:25.310999 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:34:25.310999 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:25.310999 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:34:25.311886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:25.311886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:34:25.311886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:25.311886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:34:25.311886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:34:25.311886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:34:25.311886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:34:25.311886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:25.311886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:25.311886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:25.311886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:34:25.311886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:25.312138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:25.312138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:25.312138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:34:25.312138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:25.312138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:34:25.312138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:34:25.312138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:25.312138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:34:25.312138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:25.312138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:34:25.312138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:34:25.312138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:25.312138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:34:25.312382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:25.312382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:34:25.312750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:25.312750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:34:25.312750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:34:25.312750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:34:25.312750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:34:25.312750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:34:25.312750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:25.312750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:34:25.312750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:34:25.312750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:34:25.312750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:34:25.312750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:34:25.312750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:25.312750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:25.312750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:34:25.312750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:34:25.313100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:34:25.313100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:34:25.313100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:25.313100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:25.313100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:25.313100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:34:25.313100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:25.313100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:25.313100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:25.313100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:34:25.313100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:25.313100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:34:25.313100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:34:25.313100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:25.313100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:34:25.313100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:25.313100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:34:25.313100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:34:25.313100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:25.313537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:34:25.313537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:25.313537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:34:30.046663 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:34:30.047538 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:34:30.047538 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:34:30.048262 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:34:30.081915 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:30.082280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:30.083093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:34:30.083093 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:30.083093 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:34:30.083093 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:34:30.083093 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:30.083093 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:34:30.083093 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:30.083093 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:34:30.084021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:30.084021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:34:30.084021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:30.084021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:34:30.084021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:34:30.084021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:34:30.084021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:34:30.084021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:30.084021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:30.084021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:30.084021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:34:30.084021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:30.084850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:30.084850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:30.084850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:34:30.084850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:30.084850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:34:30.084850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:34:30.084850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:30.084850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:34:30.084850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:30.084850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:34:30.084850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:34:30.084850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:30.084850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:34:30.085535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:30.085535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:34:30.085535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:30.085535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:34:30.085535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:34:30.085535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:34:30.085535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:34:30.085535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:34:30.085535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:30.085535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:34:30.085535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:34:30.085535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:34:30.085535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:34:30.085535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:34:30.085535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:30.086314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:30.086314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:34:30.086314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:34:30.086314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:34:30.086314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:34:30.086314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:30.086314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:30.086314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:30.086314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:34:30.086314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:30.086314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:30.086314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:30.086314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:34:30.086314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:30.086314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:34:30.086314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:34:30.086314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:30.086314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:34:30.086314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:30.087263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:34:30.087263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:34:30.087263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:30.087263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:34:30.087263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:30.087263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:34:30.287485 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:34:30.287798 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:34:30.287900 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:34:30.288767 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:30.289161 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:30.289662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:30.290396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:34:30.290396 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:30.290396 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:34:30.290396 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:34:30.290396 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:30.290396 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:34:30.290396 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:30.290396 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:34:30.291046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:30.291046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:34:30.291046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:30.291046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:34:30.291046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:34:30.291046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:34:30.291046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:34:30.291046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:30.291046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:30.291046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:30.291046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:34:30.291046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:30.291662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:30.291662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:30.291662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:34:30.291662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:30.291662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:34:30.291662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:34:30.291662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:30.291662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:34:30.291662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:30.291662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:34:30.291662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:34:30.291662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:30.291662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:34:30.292230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:30.292230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:34:30.292230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:30.292230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:34:30.292230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:34:30.292230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:34:30.292230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:34:30.292230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:34:30.292230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:30.292230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:34:30.292230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:34:30.292230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:34:30.292230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:34:30.292230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:34:30.292230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:30.292940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:30.292940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:34:30.292940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:34:30.292940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:34:30.292940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:34:30.292940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:30.292940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:30.292940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:30.292940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:34:30.292940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:30.292940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:30.292940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:30.292940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:34:30.292940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:30.292940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:34:30.292940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:34:30.292940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:30.292940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:34:30.292940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:30.293773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:34:30.293773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:34:30.293773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:30.293773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:34:30.293773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:30.293773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:34:30.294600 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:34:30.294758 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:34:30.294835 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:34:30.294904 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:34:30.295243 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:30.295364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:30.296069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:34:30.296069 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:30.296069 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:34:30.296069 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:34:30.296069 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:30.296069 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:34:30.296069 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:30.296069 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:34:30.297383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:30.297383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:34:30.297383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:34:30.297383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:34:30.297383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:30.297383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:34:30.297383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:30.297383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:34:30.297383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:30.297383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:34:30.297383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:30.297383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:34:30.297383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:30.297383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:34:30.297383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:34:30.298067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:30.298067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:34:30.298067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:30.298067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:34:30.298067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:34:30.298067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:30.298067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:34:30.298067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:30.298067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:34:30.298798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:30.298798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:34:30.298798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:30.298798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:34:30.298798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:34:30.298798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:34:30.298798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:34:30.298798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:34:30.298798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:34:30.298798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:34:30.298798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:30.298798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:30.299192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:30.299192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:34:30.299192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:30.299192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:30.299192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:30.299192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:34:30.299192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:30.299192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:34:30.299192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:34:30.299192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:30.299192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:34:30.299192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:30.299192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:34:30.299192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:34:30.299569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:30.299569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:34:30.299569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:30.299569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:34:30.300471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:30.300471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:34:30.300471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:34:30.300471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:34:30.300471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:34:30.300471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:34:30.300471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:30.300471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:34:30.300471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:34:30.300471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:34:30.300471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:34:30.300471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:34:30.300471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:30.300471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:30.300471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:34:30.300471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:34:30.300891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:34:30.300891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:34:30.300891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:34:30.300891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:34:30.300891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:34:30.300891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:30.300891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:30.300891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:30.300891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:34:30.300891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:30.300891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:30.300891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:30.300891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:34:30.300891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:30.300891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:34:30.300891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:34:30.300891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:30.300891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:34:30.301362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:30.301362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:34:30.301362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:34:30.301362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:30.301362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:34:30.301362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:30.301362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:34:30.302179 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:34:30.302695 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:34:30.302864 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:34:30.303047 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:34:30.303370 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:30.303626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:30.304122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:34:30.304122 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:30.304122 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:34:30.304122 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:34:30.304122 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:30.304122 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:34:30.304122 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:30.304122 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:34:30.305089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:30.305089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:34:30.305089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:34:30.305089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:34:30.305089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:30.305089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:34:30.305089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:30.305089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:34:30.305089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:30.305089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:34:30.305089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:30.305089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:34:30.305089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:30.305089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:34:30.305089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:34:30.305527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:30.305527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:34:30.305527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:30.305527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:34:30.305527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:34:30.305527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:30.305527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:34:30.305527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:30.305527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:34:30.306072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:30.306072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:34:30.306072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:30.306072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:34:30.306072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:34:30.306072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:34:30.306072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:34:30.306072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:34:30.306072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:34:30.306072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:30.306072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:30.306436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:30.306436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:34:30.306436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:30.306436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:30.306436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:30.306436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:34:30.306436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:30.306436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:34:30.306436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:34:30.306436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:30.306436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:34:30.306436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:30.306436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:34:30.306436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:34:30.306806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:30.306806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:34:30.306806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:30.306806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:34:30.307196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:30.307196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:34:30.307196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:34:30.307196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:34:30.307196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:34:30.307196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:34:30.307196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:30.307196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:34:30.307196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:34:30.307196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:34:30.307196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:34:30.307196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:34:30.307196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:30.307196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:30.307196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:34:30.307196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:34:30.307196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:34:30.307748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:34:30.307748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:34:30.307748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:34:30.307748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:30.307748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:30.307748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:30.307748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:34:30.307748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:30.307748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:30.307748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:30.307748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:34:30.307748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:30.307748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:34:30.307748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:34:30.307748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:30.307748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:34:30.307748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:30.307748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:34:30.307748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:34:30.308244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:30.308244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:34:30.308244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:30.308244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:34:30.309164 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:34:30.309518 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:34:30.309626 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:34:30.309770 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:34:30.310066 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:30.310305 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:30.310492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:30.310960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:34:30.310960 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:30.310960 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:34:30.310960 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:34:30.310960 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:30.310960 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:34:30.310960 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:30.310960 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:34:30.311860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:30.311860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:34:30.311860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:30.311860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:34:30.311860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:34:30.311860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:34:30.311860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:34:30.311860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:30.311860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:30.311860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:30.311860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:34:30.311860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:30.312205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:30.312205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:30.312205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:34:30.312205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:30.312205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:34:30.312205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:34:30.312205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:30.312205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:34:30.312205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:30.312205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:34:30.312205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:34:30.312205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:30.312205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:34:30.312560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:30.312560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:34:30.313056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:30.313056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:34:30.313056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:34:30.313056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:34:30.313056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:34:30.313056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:34:30.313056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:30.313056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:34:30.313056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:34:30.313056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:34:30.313056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:34:30.313056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:34:30.313056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:30.313056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:30.313056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:34:30.313056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:34:30.313534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:34:30.313534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:34:30.313534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:30.313534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:30.313534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:30.313534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:34:30.313534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:30.313534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:30.313534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:30.313534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:34:30.313534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:30.313534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:34:30.313534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:34:30.313534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:30.313534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:34:30.313534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:30.313534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:34:30.313534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:34:30.313534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:30.314053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:34:30.314053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:30.314053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:34:34.272826 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:34:34.273669 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_for_inconsistencies {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:34:34.273669 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Checking Neutron and OVN revision consistency. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:34:34.283015 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Synchronizing Neutron and OVN databases started {{(pid=105581) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:440}}
May 09 14:34:34.283156 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Number of inconsistencies found at create/update: networks=4, routers=1, router_ports=1 {{(pid=105581) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:419}}
May 09 14:34:34.283250 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Fixing resource 0b4a5724-2518-4e33-a6d3-0d92c749f303 (type: networks) at create/update {{(pid=105581) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 14:34:34.320553 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-0b4a5724-2518-4e33-a6d3-0d92c749f303, resource={'id': '0b4a5724-2518-4e33-a6d3-0d92c749f303', 'name': 'private', 'tenant_id': '5796380176014a32af8ff613b146faed', 'project_id': '5796380176014a32af8ff613b146faed', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['23eb2e2e-ec2f-4f98-bc47-7125a2f92952', 'db2636e9-0880-488d-9aa3-e100ce5eeb9f'], 'standard_attr_id': 12, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-09T14:04:20Z', 'updated_at': '2026-05-09T14:04:23Z', 'revision_number': 3, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 52278}, resource_type=networks, if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:34.320553 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-0b4a5724-2518-4e33-a6d3-0d92c749f303, col_values=(('external_ids', {'neutron:network_name': 'private', 'neutron:mtu': '1372', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:34.320949 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-f71b8cb1-fe70-438c-84f9-bbb7008a596d, external_ids_update=None, columns={'tag': 52278}, if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:34.320949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:34.321749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:34:34.321749 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:34.321749 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:34:34.321749 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:34:34.321749 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:34.321749 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:34:34.321749 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:34.321749 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:34:34.322688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Failed to fix resource 0b4a5724-2518-4e33-a6d3-0d92c749f303 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:34.322688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:34:34.322688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 14:34:34.322688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 14:34:34.322688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 14:34:34.322688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 14:34:34.322688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 09 14:34:34.322688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:34:34.322688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:34.322688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:34.322688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:34.323262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:34:34.323262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:34.323262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:34.323262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:34.323262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:34:34.323262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:34.323262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:34:34.323262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:34:34.323262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:34.323262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:34:34.323262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:34.323262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:34:34.323262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:34:34.323262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:34.323854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:34:34.323854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:34.323854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:34:34.323854 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Fixing resource a7d094cd-23f3-4e0c-9e88-24be6ee17b2c (type: networks) at create/update {{(pid=105581) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 14:34:34.346419 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-a7d094cd-23f3-4e0c-9e88-24be6ee17b2c, resource={'id': 'a7d094cd-23f3-4e0c-9e88-24be6ee17b2c', 'name': 'public', 'tenant_id': '5796380176014a32af8ff613b146faed', 'project_id': '5796380176014a32af8ff613b146faed', 'admin_state_up': True, 'mtu': 1430, 'status': 'ACTIVE', 'subnets': ['2f0821e3-c6a2-49d6-80ee-3480a17ba8d2', '644269b7-6b84-4d8a-ad28-d59a7c6b7c5e'], 'standard_attr_id': 18, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': True, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'is_default': True, 'tags': [], 'created_at': '2026-05-09T14:04:26Z', 'updated_at': '2026-05-09T14:04:34Z', 'revision_number': 3, 'provider:network_type': 'flat', 'provider:physical_network': 'public', 'provider:segmentation_id': None}, resource_type=networks, if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:34.346811 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-a7d094cd-23f3-4e0c-9e88-24be6ee17b2c, col_values=(('external_ids', {'neutron:network_name': 'public', 'neutron:mtu': '1430', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'flat', 'neutron:provnet-physical-network': 'public'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false', 'fdb_age_threshold': '300', 'broadcast-arps-to-all-routers': 'true'})), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:34.347563 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-08fea115-9b05-4667-a0d0-487c303084a5, external_ids_update=None, columns={'tag': []}, if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:34.347780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:34.348618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:34:34.348618 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:34.348618 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:34:34.348618 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:34:34.348618 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:34.348618 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:34:34.348618 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:34.348618 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:34:34.350341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Failed to fix resource a7d094cd-23f3-4e0c-9e88-24be6ee17b2c (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:34.350341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:34:34.350341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 14:34:34.350341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 14:34:34.350341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 14:34:34.350341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 14:34:34.350341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 09 14:34:34.350341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:34:34.350341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:34.350341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:34.350341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:34.350708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:34:34.350708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:34.350708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:34.350708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:34.350708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:34:34.350708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:34.350708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:34:34.350708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:34:34.350708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:34.350708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:34:34.350708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:34.350708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:34:34.350708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:34:34.350708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:34.351115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:34:34.351115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:34.351115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:34:34.351115 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Fixing resource 3e80d551-6de5-4370-806b-36d1db5f6fda (type: networks) at create/update {{(pid=105581) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 14:34:34.380349 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-3e80d551-6de5-4370-806b-36d1db5f6fda, resource={'id': '3e80d551-6de5-4370-806b-36d1db5f6fda', 'name': 'admin_net', 'tenant_id': '590c644b03ae4bed862ecf492c6b2eb2', 'project_id': '590c644b03ae4bed862ecf492c6b2eb2', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['453849e5-22a7-43ca-911e-9401ed2b0d74'], 'standard_attr_id': 42, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-09T14:07:41Z', 'updated_at': '2026-05-09T14:07:42Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 35952}, resource_type=networks, if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:34.380742 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-3e80d551-6de5-4370-806b-36d1db5f6fda, col_values=(('external_ids', {'neutron:network_name': 'admin_net', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:34.381257 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-f84050fb-ff65-46ce-b744-016e1ff15440, external_ids_update=None, columns={'tag': 35952}, if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:34.381421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:34.382223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:34:34.382223 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:34.382223 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:34:34.382223 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:34:34.382223 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:34.382223 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:34:34.382223 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:34.382223 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:34:34.383295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Failed to fix resource 3e80d551-6de5-4370-806b-36d1db5f6fda (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:34.383295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:34:34.383295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 14:34:34.383295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 14:34:34.383295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 14:34:34.383295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 14:34:34.383295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 09 14:34:34.383295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:34:34.383295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:34.383295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:34.383295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:34.383604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:34:34.383604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:34.383604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:34.383604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:34.383604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:34:34.383604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:34.383604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:34:34.383604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:34:34.383604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:34.383604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:34:34.383604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:34.383604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:34:34.383604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:34:34.383604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:34.383958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:34:34.383958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:34.383958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:34:34.383958 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Fixing resource 632a2ddb-5634-4bb3-aad0-f05d97abd45e (type: networks) at create/update {{(pid=105581) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 14:34:34.412507 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-632a2ddb-5634-4bb3-aad0-f05d97abd45e, resource={'id': '632a2ddb-5634-4bb3-aad0-f05d97abd45e', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-5urlz', 'tenant_id': '1f17f4a2a7ac4543a1253522e61993d2', 'project_id': '1f17f4a2a7ac4543a1253522e61993d2', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['32b523fe-135f-4f9e-a98b-fc0537fcdb35'], 'standard_attr_id': 52, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-09T14:08:15Z', 'updated_at': '2026-05-09T14:08:16Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 39245}, resource_type=networks, if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:34.412923 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-632a2ddb-5634-4bb3-aad0-f05d97abd45e, col_values=(('external_ids', {'neutron:network_name': 'k8s-clusterapi-cluster-magnum-system-kube-5urlz', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:34.413487 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-bca44f95-fe96-432e-84b8-1edc05e10ce0, external_ids_update=None, columns={'tag': 39245}, if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:34.413658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:34.414538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:34:34.414538 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:34.414538 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:34:34.414538 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:34:34.414538 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:34.414538 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:34:34.414538 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:34.414538 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:34:34.415747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Failed to fix resource 632a2ddb-5634-4bb3-aad0-f05d97abd45e (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:34.415747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:34:34.415747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 14:34:34.415747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 14:34:34.415747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 14:34:34.415747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 14:34:34.415747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 09 14:34:34.415747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:34:34.415747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:34.415747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:34.415747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:34.416081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:34:34.416081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:34.416081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:34.416081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:34.416081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:34:34.416081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:34.416081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:34:34.416081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:34:34.416081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:34.416081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:34:34.416081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:34.416081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:34:34.416081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:34:34.416081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:34.416447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:34:34.416447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:34.416447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:34:34.416447 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Fixing resource 09e042cf-39a9-42c8-b683-b29ffc42b049 (type: routers) at create/update {{(pid=105581) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 14:34:34.491992 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, resource={'id': '09e042cf-39a9-42c8-b683-b29ffc42b049', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-5urlz', 'project_id': '1f17f4a2a7ac4543a1253522e61993d2', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': 'a7d094cd-23f3-4e0c-9e88-24be6ee17b2c', 'external_fixed_ips': [{'subnet_id': '2f0821e3-c6a2-49d6-80ee-3480a17ba8d2', 'ip_address': '172.24.5.175'}, {'subnet_id': '644269b7-6b84-4d8a-ad28-d59a7c6b7c5e', 'ip_address': '2001:db8::d8'}], 'enable_snat': True}, 'gw_port_id': '71ded037-e812-40c0-8d8f-70c141c74323', 'description': 'Created by cluster-api-provider-openstack cluster magnum-system-kube-5urlz', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': 'a7d094cd-23f3-4e0c-9e88-24be6ee17b2c', 'external_fixed_ips': [{'ip_address': '172.24.5.175', 'subnet_id': '2f0821e3-c6a2-49d6-80ee-3480a17ba8d2'}, {'ip_address': '2001:db8::d8', 'subnet_id': '644269b7-6b84-4d8a-ad28-d59a7c6b7c5e'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2026-05-09T14:08:17Z', 'updated_at': '2026-05-09T14:08:19Z', 'revision_number': 3, 'tenant_id': '1f17f4a2a7ac4543a1253522e61993d2'}, resource_type=routers, if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:34.492366 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, columns={'external_ids': {'neutron:router_name': 'k8s-clusterapi-cluster-magnum-system-kube-5urlz', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:34.492751 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-a7d094cd-23f3-4e0c-9e88-24be6ee17b2c, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "71ded037-e812-40c0-8d8f-70c141c74323"')], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:34.493004 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-a7d094cd-23f3-4e0c-9e88-24be6ee17b2c, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "71ded037-e812-40c0-8d8f-70c141c74323"')], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:34.493222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:34.494100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:34:34.494100 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:34.494100 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:34:34.494100 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:34:34.494100 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:34.494100 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:34:34.494100 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:34.494100 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:34:34.494339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Unable to update router 09e042cf-39a9-42c8-b683-b29ffc42b049. Error: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:34.495921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Failed to fix resource 09e042cf-39a9-42c8-b683-b29ffc42b049 (type: routers): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:34.495921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:34:34.495921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 14:34:34.495921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 14:34:34.495921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 14:34:34.495921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 14:34:34.495921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1635, in update_router
May 09 14:34:34.495921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with excutils.save_and_reraise_exception():
May 09 14:34:34.495921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 271, in __exit__
May 09 14:34:34.495921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.force_reraise()
May 09 14:34:34.495921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 233, in force_reraise
May 09 14:34:34.496266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise self.value
May 09 14:34:34.496266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1569, in update_router
May 09 14:34:34.496266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:34:34.496266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:34.496266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:34.496266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:34.496266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:34:34.496266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:34.496266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:34.496266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:34.496266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:34:34.496266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:34.496266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:34:34.496266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:34:34.496694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:34.496694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:34:34.496694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:34.496694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:34:34.496694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:34:34.496694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:34.496694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:34:34.496694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:34.496694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:34:34.496694 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Fixing resource 71ded037-e812-40c0-8d8f-70c141c74323 (type: router_ports) at create/update {{(pid=105581) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 14:34:34.603986 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-71ded037-e812-40c0-8d8f-70c141c74323, resource={'id': '71ded037-e812-40c0-8d8f-70c141c74323', 'name': '', 'network_id': 'a7d094cd-23f3-4e0c-9e88-24be6ee17b2c', 'project_id': '', 'mac_address': 'fa:16:3e:c7:86:37', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '09e042cf-39a9-42c8-b683-b29ffc42b049', 'device_owner': 'network:router_gateway', 'standard_attr_id': 57, 'fixed_ips': [{'subnet_id': '2f0821e3-c6a2-49d6-80ee-3480a17ba8d2', 'ip_address': '172.24.5.175'}, {'subnet_id': '644269b7-6b84-4d8a-ad28-d59a7c6b7c5e', 'ip_address': '2001:db8::d8'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0000173183', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'ovs_create_tap': False, 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2026-05-09T14:08:17Z', 'updated_at': '2026-05-09T14:08:19Z', 'revision_number': 5, 'tenant_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:34.604389 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-71ded037-e812-40c0-8d8f-70c141c74323, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': '2f0821e3-c6a2-49d6-80ee-3480a17ba8d2 644269b7-6b84-4d8a-ad28-d59a7c6b7c5e', 'neutron:network_name': 'neutron-a7d094cd-23f3-4e0c-9e88-24be6ee17b2c', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-09e042cf-39a9-42c8-b683-b29ffc42b049'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['2001:db8::d8/64', '172.24.5.175/24'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:34.605285 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=71ded037-e812-40c0-8d8f-70c141c74323, lrouter_port=lrp-71ded037-e812-40c0-8d8f-70c141c74323, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:34.605545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:34.606427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:34:34.606427 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:34.606427 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:34:34.606427 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:34:34.606427 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:34.606427 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:34:34.606427 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:34.606427 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:34:34.607692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Failed to fix resource 71ded037-e812-40c0-8d8f-70c141c74323 (type: router_ports): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:34.607692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:34:34.607692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 14:34:34.607692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 14:34:34.607692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 14:34:34.607692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 14:34:34.607692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1976, in update_router_port
May 09 14:34:34.607692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:34:34.607692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:34.607692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:34.607692 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:34.608347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:34:34.608347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:34.608347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:34.608347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:34.608347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:34:34.608347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:34.608347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:34:34.608347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:34:34.608347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:34.608347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:34:34.608347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:34.608347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:34:34.608347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:34:34.608347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:34.609002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:34:34.609002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:34.609002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:34:34.609002 np0000173183 neutron-ovn-maintenance-worker[105581]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_for_inconsistencies finished in 0.334 seconds
May 09 14:34:35.047737 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:34:35.048509 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:34:35.048702 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:34:35.050044 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:34:35.088291 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:35.088291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:35.088879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:34:35.088879 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:35.088879 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:34:35.088879 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:34:35.088879 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:35.088879 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:34:35.088879 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:35.088879 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:34:35.089934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:35.089934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:34:35.089934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:35.089934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:34:35.089934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:34:35.089934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:34:35.089934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:34:35.089934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:35.089934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:35.089934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:35.089934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:34:35.089934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:35.091201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:35.091201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:35.091201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:34:35.091201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:35.091201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:34:35.091201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:34:35.091201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:35.091201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:34:35.091201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:35.091201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:34:35.091201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:34:35.091201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:35.091201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:34:35.091813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:35.091813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:34:35.091813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:35.091813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:34:35.091813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:34:35.091813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:34:35.091813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:34:35.091813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:34:35.091813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:35.091813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:34:35.091813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:34:35.091813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:34:35.091813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:34:35.091813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:34:35.091813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:35.092498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:35.092498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:34:35.092498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:34:35.092498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:34:35.092498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:34:35.092498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:35.092498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:35.092498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:35.092498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:34:35.092498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:35.092498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:35.092498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:35.092498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:34:35.092498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:35.092498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:34:35.092498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:34:35.092498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:35.092498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:34:35.092498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:35.093332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:34:35.093332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:34:35.093332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:35.093332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:34:35.093332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:35.093332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:34:35.288603 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:34:35.289320 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:34:35.289320 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:34:35.289979 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:35.290181 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:35.290357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:35.290919 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:34:35.290919 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:35.290919 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:34:35.290919 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:34:35.290919 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:35.290919 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:34:35.290919 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:35.290919 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:34:35.291588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:35.291588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:34:35.291588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:35.291588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:34:35.291588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:34:35.291588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:34:35.291588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:34:35.291588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:35.291588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:35.291588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:35.291588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:34:35.291588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:35.291932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:35.291932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:35.291932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:34:35.291932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:35.291932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:34:35.291932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:34:35.291932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:35.291932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:34:35.291932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:35.291932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:34:35.291932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:34:35.291932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:35.291932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:34:35.292238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:35.292238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:34:35.292363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:35.292363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:34:35.292363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:34:35.292363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:34:35.292363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:34:35.292363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:34:35.292363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:35.292363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:34:35.292363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:34:35.292363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:34:35.292363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:34:35.292363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:34:35.292363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:35.292363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:35.292363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:34:35.292363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:34:35.292746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:34:35.292746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:34:35.292746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:35.292746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:35.292746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:35.292746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:34:35.292746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:35.292746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:35.292746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:35.292746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:34:35.292746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:35.292746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:34:35.292746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:34:35.292746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:35.292746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:34:35.292746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:35.292746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:34:35.292746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:34:35.292746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:35.293210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:34:35.293210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:35.293210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:34:35.294942 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:34:35.295274 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:34:35.295399 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:34:35.295562 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:34:35.295777 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:35.296062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:35.296295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:34:35.296295 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:35.296295 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:34:35.296295 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:34:35.296295 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:35.296295 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:34:35.296295 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:35.296295 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:34:35.297095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:35.297095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:34:35.297095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:34:35.297095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:34:35.297095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:35.297095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:34:35.297095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:35.297095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:34:35.297095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:35.297095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:34:35.297095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:35.297095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:34:35.297095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:35.297095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:34:35.297095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:34:35.297556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:35.297556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:34:35.297556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:35.297556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:34:35.297556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:34:35.297556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:35.297556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:34:35.297556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:35.297556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:34:35.298060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:35.298060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:34:35.298060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:35.298060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:34:35.298060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:34:35.298060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:34:35.298060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:34:35.298060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:34:35.298060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:34:35.298060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:34:35.298060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:35.298060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:35.298479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:35.298479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:34:35.298479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:35.298479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:35.298479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:35.298479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:34:35.298479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:35.298479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:34:35.298479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:34:35.298479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:35.298479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:34:35.298479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:35.298479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:34:35.298479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:34:35.298861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:35.298861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:34:35.298861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:35.298861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:34:35.299521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:35.299521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:34:35.299521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:34:35.299521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:34:35.299521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:34:35.299521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:34:35.299521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:35.299521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:34:35.299521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:34:35.299521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:34:35.299521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:34:35.299521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:34:35.299521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:35.299521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:35.299521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:34:35.299521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:34:35.300014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:34:35.300014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:34:35.300014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:34:35.300014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:34:35.300014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:34:35.300014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:35.300014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:35.300014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:35.300014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:34:35.300014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:35.300014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:35.300014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:35.300014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:34:35.300014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:35.300014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:34:35.300014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:34:35.300014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:35.300014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:34:35.300567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:35.300567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:34:35.300567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:34:35.300567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:35.300567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:34:35.300567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:35.300567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:34:35.302904 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:34:35.303443 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:34:35.303570 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:34:35.303652 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:34:35.303918 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:35.304065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:35.304432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:34:35.304432 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:35.304432 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:34:35.304432 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:34:35.304432 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:35.304432 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:34:35.304432 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:35.304432 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:34:35.305158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:35.305158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:34:35.305158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:34:35.305158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:34:35.305158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:35.305158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:34:35.305158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:35.305158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:34:35.305158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:35.305158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:34:35.305158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:35.305158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:34:35.305158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:35.305158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:34:35.305158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:34:35.305937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:35.305937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:34:35.305937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:35.305937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:34:35.305937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:34:35.305937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:35.305937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:34:35.305937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:35.305937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:34:35.306679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:35.306679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:34:35.306679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:35.306679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:34:35.306679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:34:35.306679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:34:35.306679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:34:35.306679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:34:35.306679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:34:35.306679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:35.306679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:35.307710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:35.307710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:34:35.307710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:35.307710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:35.307710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:35.307710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:34:35.307710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:35.307710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:34:35.307710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:34:35.307710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:35.307710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:34:35.307710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:35.307710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:34:35.307710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:34:35.308393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:35.308393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:34:35.308393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:35.308393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:34:35.308393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:35.308393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:34:35.308393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:34:35.308393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:34:35.308393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:34:35.308393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:34:35.308393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:35.308393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:34:35.308393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:34:35.308393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:34:35.309070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:34:35.309070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:34:35.309070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:35.309070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:35.309070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:34:35.309070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:34:35.309070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:34:35.309070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:34:35.309070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:34:35.309070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:34:35.309070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:35.309070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:35.309070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:35.309070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:34:35.309070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:35.309070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:35.309070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:35.309070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:34:35.309070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:35.309909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:34:35.309909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:34:35.309909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:35.309909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:34:35.309909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:35.309909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:34:35.309909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:34:35.309909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:35.309909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:34:35.309909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:35.309909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:34:35.309909 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:34:35.309909 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:34:35.310541 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:34:35.310541 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:34:35.310541 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:35.310541 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:35.310757 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:35.310930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:34:35.310930 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:35.310930 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:34:35.310930 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:34:35.310930 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:35.310930 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:34:35.310930 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:35.310930 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:34:35.311796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:35.311796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:34:35.311796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:35.311796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:34:35.311796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:34:35.311796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:34:35.311796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:34:35.311796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:35.311796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:35.311796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:35.311796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:34:35.311796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:35.312047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:35.312047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:35.312047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:34:35.312047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:35.312047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:34:35.312047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:34:35.312047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:35.312047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:34:35.312047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:35.312047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:34:35.312047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:34:35.312047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:35.312047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:34:35.312284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:35.312284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:34:35.312805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:35.312805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:34:35.312805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:34:35.312805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:34:35.312805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:34:35.312805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:34:35.312805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:35.312805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:34:35.312805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:34:35.312805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:34:35.312805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:34:35.312805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:34:35.312805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:35.312805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:35.312805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:34:35.312805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:34:35.313097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:34:35.313097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:34:35.313097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:35.313097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:35.313097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:35.313097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:34:35.313097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:35.313097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:35.313097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:35.313097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:34:35.313097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:35.313097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:34:35.313097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:34:35.313097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:35.313097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:34:35.313097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:35.313097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:34:35.313097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:34:35.313097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:35.313469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:34:35.313469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:35.313469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:34:40.048473 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:34:40.049459 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:34:40.049459 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:34:40.050495 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:34:40.088544 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:40.088544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:40.089229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:34:40.089229 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:40.089229 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:34:40.089229 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:34:40.089229 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:40.089229 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:34:40.089229 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:40.089229 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:34:40.090153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:40.090153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:34:40.090153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:40.090153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:34:40.090153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:34:40.090153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:34:40.090153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:34:40.090153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:40.090153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:40.090153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:40.090153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:34:40.090153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:40.090858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:40.090858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:40.090858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:34:40.090858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:40.090858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:34:40.090858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:34:40.090858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:40.090858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:34:40.090858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:40.090858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:34:40.090858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:34:40.090858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:40.090858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:34:40.091584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:40.091584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:34:40.091584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:40.091584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:34:40.091584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:34:40.091584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:34:40.091584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:34:40.091584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:34:40.091584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:40.091584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:34:40.091584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:34:40.091584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:34:40.091584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:34:40.091584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:34:40.091584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:40.092364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:40.092364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:34:40.092364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:34:40.092364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:34:40.092364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:34:40.092364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:40.092364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:40.092364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:40.092364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:34:40.092364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:40.092364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:40.092364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:40.092364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:34:40.092364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:40.092364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:34:40.092364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:34:40.092364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:40.092364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:34:40.092364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:40.095634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:34:40.095634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:34:40.095634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:40.095634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:34:40.095634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:40.095634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:34:40.289334 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:34:40.289923 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:34:40.290021 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:34:40.291552 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:40.291552 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:40.291888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:40.292551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:34:40.292551 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:40.292551 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:34:40.292551 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:34:40.292551 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:40.292551 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:34:40.292551 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:40.292551 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:34:40.293594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:40.293594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:34:40.293594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:40.293594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:34:40.293594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:34:40.293594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:34:40.293594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:34:40.293594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:40.293594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:40.293594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:40.293594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:34:40.293594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:40.294222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:40.294222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:40.294222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:34:40.294222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:40.294222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:34:40.294222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:34:40.294222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:40.294222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:34:40.294222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:40.294222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:34:40.294222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:34:40.294222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:40.294222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:34:40.294823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:40.294823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:34:40.294823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:40.294823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:34:40.294823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:34:40.294823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:34:40.294823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:34:40.294823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:34:40.294823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:40.294823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:34:40.294823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:34:40.294823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:34:40.294823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:34:40.294823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:34:40.294823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:40.295535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:40.295535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:34:40.295535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:34:40.295535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:34:40.295535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:34:40.295535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:40.295535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:40.295535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:40.295535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:34:40.295535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:40.295535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:40.295535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:40.295535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:34:40.295535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:40.295535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:34:40.295535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:34:40.295535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:40.295535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:34:40.295535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:40.296345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:34:40.296345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:34:40.296345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:40.296345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:34:40.296345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:40.296345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:34:40.296345 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:34:40.296345 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:34:40.296345 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:34:40.296345 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:34:40.297532 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:40.297532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:40.298014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:34:40.298014 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:40.298014 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:34:40.298014 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:34:40.298014 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:40.298014 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:34:40.298014 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:40.298014 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:34:40.298362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:40.298362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:34:40.298362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:34:40.298362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:34:40.298362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:40.298362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:34:40.298362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:40.298362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:34:40.298362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:40.298362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:34:40.298362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:40.298362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:34:40.298362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:40.298362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:34:40.298362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:34:40.299220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:40.299220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:34:40.299220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:40.299220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:34:40.299220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:34:40.299220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:40.299220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:34:40.299220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:40.299220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:34:40.299220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:40.299220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:34:40.299220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:40.299220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:34:40.299220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:34:40.300301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:34:40.300301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:34:40.300301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:34:40.300301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:34:40.300301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:34:40.300301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:40.300301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:40.300301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:40.300301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:34:40.300301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:40.300301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:40.300301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:40.300301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:34:40.300956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:40.300956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:34:40.300956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:34:40.300956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:40.300956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:34:40.300956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:40.300956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:34:40.300956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:34:40.300956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:40.300956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:34:40.300956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:40.300956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:34:40.301858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:40.301858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:34:40.301858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:34:40.301858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:34:40.301858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:34:40.301858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:34:40.301858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:40.301858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:34:40.301858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:34:40.301858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:34:40.301858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:34:40.301858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:34:40.301858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:40.301858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:40.301858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:34:40.301858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:34:40.302597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:34:40.302597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:34:40.302597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:34:40.302597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:34:40.302597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:34:40.302597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:40.302597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:40.302597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:40.302597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:34:40.302597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:40.302597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:40.302597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:40.302597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:34:40.302597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:40.302597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:34:40.302597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:34:40.302597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:40.302597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:34:40.303401 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:40.303401 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:34:40.303401 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:34:40.303401 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:40.303401 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:34:40.303401 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:40.303401 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:34:40.303401 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:34:40.303812 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:34:40.303812 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:34:40.304072 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:34:40.304701 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:40.305014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:40.305691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:34:40.305691 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:40.305691 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:34:40.305691 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:34:40.305691 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:40.305691 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:34:40.305691 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:40.305691 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:34:40.306418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:40.306418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:34:40.306418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:34:40.306418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:34:40.306418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:40.306418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:34:40.306418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:40.306418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:34:40.306418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:40.306418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:34:40.306418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:40.306418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:34:40.306418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:40.306418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:34:40.306418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:34:40.306747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:40.306747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:34:40.306747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:40.306747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:34:40.306747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:34:40.306747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:40.306747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:34:40.306747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:40.306747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:34:40.307443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:40.307443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:34:40.307443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:40.307443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:34:40.307443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:34:40.307443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:34:40.307443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:34:40.307443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:34:40.307443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:34:40.307443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:40.307443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:40.307845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:40.307845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:34:40.307845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:40.307845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:40.307845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:40.307845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:34:40.307845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:40.307845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:34:40.307845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:34:40.307845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:40.307845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:34:40.307845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:40.307845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:34:40.307845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:34:40.308136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:40.308136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:34:40.308136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:40.308136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:34:40.308863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:40.308863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:34:40.308863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:34:40.308863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:34:40.308863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:34:40.308863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:34:40.308863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:40.308863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:34:40.308863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:34:40.308863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:34:40.308863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:34:40.308863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:34:40.308863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:40.308863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:40.308863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:34:40.308863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:34:40.308863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:34:40.309218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:34:40.309218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:34:40.309218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:34:40.309218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:40.309218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:40.309218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:40.309218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:34:40.309218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:40.309218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:40.309218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:40.309218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:34:40.309218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:40.309218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:34:40.309218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:34:40.309218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:40.309218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:34:40.309218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:40.309218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:34:40.309218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:34:40.309645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:40.309645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:34:40.309645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:40.309645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:34:40.309898 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:34:40.310139 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:34:40.310252 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:34:40.310530 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:34:40.310859 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:40.311091 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:40.311300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:40.311722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:34:40.311722 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:40.311722 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:34:40.311722 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:34:40.311722 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:40.311722 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:34:40.311722 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:40.311722 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:34:40.312702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:40.312702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:34:40.312702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:40.312702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:34:40.312702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:34:40.312702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:34:40.312702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:34:40.312702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:40.312702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:40.312702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:40.312702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:34:40.312702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:40.312967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:40.312967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:40.312967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:34:40.312967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:40.312967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:34:40.312967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:34:40.312967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:40.312967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:34:40.312967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:40.312967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:34:40.312967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:34:40.312967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:40.312967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:34:40.313238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:40.313238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:34:40.313877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:40.313877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:34:40.313877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:34:40.313877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:34:40.313877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:34:40.313877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:34:40.313877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:40.313877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:34:40.313877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:34:40.313877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:34:40.313877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:34:40.313877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:34:40.313877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:40.313877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:40.313877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:34:40.313877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:34:40.314446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:34:40.314446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:34:40.314446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:40.314446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:40.314446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:40.314446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:34:40.314446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:40.314446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:40.314446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:40.314446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:34:40.314446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:40.314446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:34:40.314446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:34:40.314446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:40.314446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:34:40.314446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:40.314446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:34:40.314446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:34:40.314446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:40.314959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:34:40.314959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:40.314959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:34:45.048996 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:34:45.050588 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:34:45.050588 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:34:45.050958 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:34:45.081615 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:45.081992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:45.082709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:34:45.082709 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:45.082709 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:34:45.082709 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:34:45.082709 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:45.082709 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:34:45.082709 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:45.082709 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:34:45.083388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:45.083388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:34:45.083388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:45.083388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:34:45.083388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:34:45.083388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:34:45.083388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:34:45.083388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:45.083388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:45.083388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:45.083388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:34:45.083388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:45.084044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:45.084044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:45.084044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:34:45.084044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:45.084044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:34:45.084044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:34:45.084044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:45.084044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:34:45.084044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:45.084044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:34:45.084044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:34:45.084044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:45.084044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:34:45.084742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:45.084742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:34:45.084742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:45.084742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:34:45.084742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:34:45.084742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:34:45.084742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:34:45.084742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:34:45.084742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:45.084742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:34:45.084742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:34:45.084742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:34:45.084742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:34:45.084742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:34:45.084742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:45.085560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:45.085560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:34:45.085560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:34:45.085560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:34:45.085560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:34:45.085560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:45.085560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:45.085560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:45.085560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:34:45.085560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:45.085560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:45.085560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:45.085560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:34:45.085560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:45.085560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:34:45.085560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:34:45.085560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:45.085560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:34:45.085560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:45.086735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:34:45.086735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:34:45.086735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:45.086735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:34:45.086735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:45.086735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:34:45.289973 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:34:45.290598 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:34:45.290789 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:34:45.291915 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:45.292239 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:45.292514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:45.293384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:34:45.293384 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:45.293384 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:34:45.293384 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:34:45.293384 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:45.293384 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:34:45.293384 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:45.293384 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:34:45.294914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:45.294914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:34:45.294914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:45.294914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:34:45.294914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:34:45.294914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:34:45.294914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:34:45.294914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:45.294914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:45.294914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:45.294914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:34:45.294914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:45.295561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:45.295561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:45.295561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:34:45.295561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:45.295561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:34:45.295561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:34:45.295561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:45.295561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:34:45.295561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:45.295561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:34:45.295561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:34:45.295561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:45.295561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:34:45.296168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:45.296168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:34:45.296840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:45.296840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:34:45.296840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:34:45.296840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:34:45.296840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:34:45.296840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:34:45.296840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:45.296840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:34:45.296840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:34:45.296840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:34:45.296840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:34:45.296840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:34:45.296840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:45.296840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:45.296840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:34:45.296840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:34:45.297944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:34:45.297944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:34:45.297944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:45.297944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:45.297944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:45.297944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:34:45.297944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:45.297944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:45.297944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:45.297944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:34:45.297944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:45.297944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:34:45.297944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:34:45.297944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:45.297944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:34:45.297944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:45.297944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:34:45.297944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:34:45.297944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:45.298832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:34:45.298832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:45.298832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:34:45.298832 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:34:45.298832 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:34:45.298832 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:34:45.298832 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:34:45.298832 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:45.299314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:45.299314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:34:45.299314 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:45.299314 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:34:45.299314 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:34:45.299314 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:45.299314 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:34:45.299314 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:45.299314 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:34:45.299796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:45.299796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:34:45.299796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:34:45.299796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:34:45.299796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:45.299796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:34:45.299796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:45.299796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:34:45.299796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:45.299796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:34:45.299796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:45.299796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:34:45.299796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:45.299796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:34:45.299796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:34:45.300578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:45.300578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:34:45.300578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:45.300578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:34:45.300578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:34:45.300578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:45.300578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:34:45.300578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:45.300578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:34:45.300578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:45.300578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:34:45.300578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:45.300578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:34:45.300578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:34:45.301008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:34:45.301008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:34:45.301008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:34:45.301008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:34:45.301008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:34:45.301008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:45.301008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:45.301008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:45.301008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:34:45.301008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:45.301008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:45.301008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:45.301008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:34:45.301389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:45.301389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:34:45.301389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:34:45.301389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:45.301389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:34:45.301389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:45.301389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:34:45.301389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:34:45.301389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:45.301389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:34:45.301389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:45.301389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:34:45.301853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:45.301853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:34:45.301853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:34:45.301853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:34:45.301853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:34:45.301853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:34:45.301853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:45.301853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:34:45.301853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:34:45.301853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:34:45.301853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:34:45.301853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:34:45.301853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:45.301853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:45.301853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:34:45.301853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:34:45.302313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:34:45.302313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:34:45.302313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:34:45.302313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:34:45.302313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:34:45.302313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:45.302313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:45.302313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:45.302313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:34:45.302313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:45.302313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:45.302313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:45.302313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:34:45.302313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:45.302313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:34:45.302313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:34:45.302313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:45.302313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:34:45.302839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:45.302839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:34:45.302839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:34:45.302839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:45.302839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:34:45.302839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:45.302839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:34:45.303840 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:34:45.304089 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:34:45.304194 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:34:45.304316 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:34:45.304642 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:45.304857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:45.305301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:34:45.305301 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:45.305301 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:34:45.305301 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:34:45.305301 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:45.305301 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:34:45.305301 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:45.305301 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:34:45.306014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:45.306014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:34:45.306014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:34:45.306014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:34:45.306014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:45.306014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:34:45.306014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:45.306014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:34:45.306014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:45.306014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:34:45.306014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:45.306014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:34:45.306014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:45.306014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:34:45.306014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:34:45.306468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:45.306468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:34:45.306468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:45.306468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:34:45.306468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:34:45.306468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:45.306468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:34:45.306468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:45.306468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:34:45.306881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:45.306881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:34:45.306881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:45.306881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:34:45.306881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:34:45.306881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:34:45.306881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:34:45.306881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:34:45.306881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:34:45.306881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:45.306881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:45.307244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:45.307244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:34:45.307244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:45.307244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:45.307244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:45.307244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:34:45.307244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:45.307244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:34:45.307244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:34:45.307244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:45.307244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:34:45.307244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:45.307244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:34:45.307244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:34:45.307669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:45.307669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:34:45.307669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:45.307669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:34:45.307933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:45.307933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:34:45.307933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:34:45.307933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:34:45.307933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:34:45.307933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:34:45.307933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:45.307933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:34:45.307933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:34:45.307933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:34:45.307933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:34:45.307933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:34:45.307933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:45.307933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:45.307933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:34:45.307933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:34:45.307933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:34:45.308380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:34:45.308380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:34:45.308380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:34:45.308380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:45.308380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:45.308380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:45.308380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:34:45.308380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:45.308380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:45.308380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:45.308380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:34:45.308380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:45.308380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:34:45.308380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:34:45.308380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:45.308380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:34:45.308380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:45.308380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:34:45.308380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:34:45.308890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:45.308890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:34:45.308890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:45.308890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:34:45.310302 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:34:45.310640 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:34:45.310750 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:34:45.310891 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:34:45.311437 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:45.311788 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:45.312000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:45.312693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:34:45.312693 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:45.312693 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:34:45.312693 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:34:45.312693 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:45.312693 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:34:45.312693 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:45.312693 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:34:45.313529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:45.313529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:34:45.313529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:45.313529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:34:45.313529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:34:45.313529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:34:45.313529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:34:45.313529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:45.313529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:45.313529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:45.313529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:34:45.313529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:45.313873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:45.313873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:45.313873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:34:45.313873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:45.313873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:34:45.313873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:34:45.313873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:45.313873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:34:45.313873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:45.313873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:34:45.313873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:34:45.313873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:45.313873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:34:45.314220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:45.314220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:34:45.314470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:45.314470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:34:45.314470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:34:45.314470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:34:45.314470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:34:45.314470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:34:45.314470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:45.314470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:34:45.314470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:34:45.314470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:34:45.314470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:34:45.314470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:34:45.314470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:45.314470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:45.314470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:34:45.314470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:34:45.314928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:34:45.314928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:34:45.314928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:45.314928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:45.314928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:45.314928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:34:45.314928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:45.314928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:45.314928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:45.314928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:34:45.314928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:45.314928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:34:45.314928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:34:45.314928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:45.314928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:34:45.314928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:45.314928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:34:45.314928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:34:45.314928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:45.315469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:34:45.315469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:45.315469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:34:50.050182 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:34:50.051039 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:34:50.051108 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:34:50.052345 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:34:50.092277 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:50.092879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:50.093742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:34:50.093742 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:50.093742 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:34:50.093742 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:34:50.093742 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:50.093742 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:34:50.093742 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:50.093742 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:34:50.094556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:50.094556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:34:50.094556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:50.094556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:34:50.094556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:34:50.094556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:34:50.094556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:34:50.094556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:50.094556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:50.094556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:50.094556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:34:50.094556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:50.095214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:50.095214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:50.095214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:34:50.095214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:50.095214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:34:50.095214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:34:50.095214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:50.095214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:34:50.095214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:50.095214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:34:50.095214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:34:50.095214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:50.095214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:34:50.095866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:50.095866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:34:50.095866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:50.095866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:34:50.095866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:34:50.095866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:34:50.095866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:34:50.095866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:34:50.095866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:50.095866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:34:50.095866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:34:50.095866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:34:50.095866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:34:50.095866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:34:50.095866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:50.096633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:50.096633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:34:50.096633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:34:50.096633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:34:50.096633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:34:50.096633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:50.096633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:50.096633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:50.096633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:34:50.096633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:50.096633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:50.096633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:50.096633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:34:50.096633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:50.096633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:34:50.096633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:34:50.096633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:50.096633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:34:50.096633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:50.097499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:34:50.097499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:34:50.097499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:50.097499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:34:50.097499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:50.097499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:34:50.290709 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:34:50.291243 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:34:50.291472 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:34:50.292735 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:50.293103 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:50.293390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:50.294303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:34:50.294303 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:50.294303 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:34:50.294303 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:34:50.294303 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:50.294303 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:34:50.294303 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:50.294303 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:34:50.295817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:50.295817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:34:50.295817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:50.295817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:34:50.295817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:34:50.295817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:34:50.295817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:34:50.295817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:50.295817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:50.295817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:50.295817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:34:50.295817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:50.296612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:50.296612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:50.296612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:34:50.296612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:50.296612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:34:50.296612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:34:50.296612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:50.296612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:34:50.296612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:50.296612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:34:50.296612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:34:50.296612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:50.296612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:34:50.297114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:50.297114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:34:50.297450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:50.297450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:34:50.297450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:34:50.297450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:34:50.297450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:34:50.297450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:34:50.297450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:50.297450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:34:50.297450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:34:50.297450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:34:50.297450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:34:50.297450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:34:50.297450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:50.297450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:50.297450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:34:50.297450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:34:50.298089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:34:50.298089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:34:50.298089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:50.298089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:50.298089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:50.298089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:34:50.298089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:50.298089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:50.298089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:50.298089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:34:50.298089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:50.298089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:34:50.298089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:34:50.298089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:50.298089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:34:50.298089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:50.298089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:34:50.298089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:34:50.298089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:50.298871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:34:50.298871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:50.298871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:34:50.298871 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:34:50.298871 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:34:50.298871 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:34:50.298871 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:34:50.299186 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:50.299186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:50.299685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:34:50.299685 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:50.299685 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:34:50.299685 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:34:50.299685 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:50.299685 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:34:50.299685 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:50.299685 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:34:50.300795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:50.300795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:34:50.300795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:34:50.300795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:34:50.300795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:50.300795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:34:50.300795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:50.300795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:34:50.300795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:50.300795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:34:50.300795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:50.300795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:34:50.300795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:50.300795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:34:50.300795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:34:50.301718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:50.301718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:34:50.301718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:50.301718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:34:50.301718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:34:50.301718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:50.301718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:34:50.301718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:50.301718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:34:50.301718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:50.301718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:34:50.301718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:50.301718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:34:50.301718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:34:50.302449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:34:50.302449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:34:50.302449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:34:50.302449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:34:50.302449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:34:50.302449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:50.302449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:50.302449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:50.302449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:34:50.302449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:50.302449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:50.302449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:50.302449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:34:50.302785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:50.302785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:34:50.302785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:34:50.302785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:50.302785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:34:50.302785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:50.302785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:34:50.302785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:34:50.302785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:50.302785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:34:50.302785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:50.302785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:34:50.303078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:50.303078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:34:50.303078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:34:50.303078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:34:50.303078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:34:50.303078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:34:50.303078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:50.303078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:34:50.303078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:34:50.303078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:34:50.303078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:34:50.303078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:34:50.303078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:50.303078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:50.303078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:34:50.303078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:34:50.303504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:34:50.303504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:34:50.303504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:34:50.303504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:34:50.303504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:34:50.303504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:50.303504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:50.303504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:50.303504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:34:50.303504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:50.303504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:50.303504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:50.303504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:34:50.303504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:50.303504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:34:50.303504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:34:50.303504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:50.303504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:34:50.303967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:50.303967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:34:50.303967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:34:50.303967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:50.303967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:34:50.303967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:50.303967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:34:50.304357 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:34:50.304665 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:34:50.304798 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:34:50.304920 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:34:50.305271 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:50.305550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:50.306228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:34:50.306228 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:50.306228 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:34:50.306228 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:34:50.306228 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:50.306228 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:34:50.306228 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:50.306228 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:34:50.306958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:50.306958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:34:50.306958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:34:50.306958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:34:50.306958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:50.306958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:34:50.306958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:50.306958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:34:50.306958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:50.306958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:34:50.306958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:50.306958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:34:50.306958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:50.306958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:34:50.306958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:34:50.307356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:50.307356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:34:50.307356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:50.307356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:34:50.307356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:34:50.307356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:50.307356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:34:50.307356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:50.307356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:34:50.308903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:50.308903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:34:50.308903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:50.308903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:34:50.308903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:34:50.308903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:34:50.308903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:34:50.308903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:34:50.308903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:34:50.308903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:50.308903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:50.309766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:50.309766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:34:50.309766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:50.309766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:50.309766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:50.309766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:34:50.309766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:50.309766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:34:50.309766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:34:50.309766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:50.309766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:34:50.309766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:50.309766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:34:50.309766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:34:50.310154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:50.310154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:34:50.310154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:50.310154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:34:50.310154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:50.310154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:34:50.310154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:34:50.310154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:34:50.310154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:34:50.310154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:34:50.310154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:50.310154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:34:50.310154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:34:50.310154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:34:50.310518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:34:50.310518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:34:50.310518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:50.310518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:50.310518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:34:50.310518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:34:50.310518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:34:50.310518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:34:50.310518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:34:50.310518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:34:50.310518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:50.310518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:50.310518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:50.310518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:34:50.310518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:50.310518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:50.310518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:50.310518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:34:50.310518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:50.310976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:34:50.310976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:34:50.310976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:50.310976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:34:50.310976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:50.310976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:34:50.310976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:34:50.310976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:50.310976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:34:50.310976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:50.310976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:34:50.310976 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:34:50.310976 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:34:50.311350 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:34:50.311350 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:34:50.311649 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:50.312010 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:50.312277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:50.313026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:34:50.313026 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:50.313026 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:34:50.313026 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:34:50.313026 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:50.313026 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:34:50.313026 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:50.313026 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:34:50.313825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:50.313825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:34:50.313825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:50.313825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:34:50.313825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:34:50.313825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:34:50.313825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:34:50.313825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:50.313825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:50.313825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:50.313825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:34:50.313825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:50.314146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:50.314146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:50.314146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:34:50.314146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:50.314146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:34:50.314146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:34:50.314146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:50.314146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:34:50.314146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:50.314146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:34:50.314146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:34:50.314146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:50.314146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:34:50.314477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:50.314477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:34:50.314756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:50.314756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:34:50.314756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:34:50.314756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:34:50.314756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:34:50.314756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:34:50.314756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:50.314756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:34:50.314756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:34:50.314756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:34:50.314756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:34:50.314756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:34:50.314756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:50.314756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:50.314756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:34:50.314756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:34:50.315191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:34:50.315191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:34:50.315191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:50.315191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:50.315191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:50.315191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:34:50.315191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:50.315191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:50.315191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:50.315191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:34:50.315191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:50.315191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:34:50.315191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:34:50.315191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:50.315191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:34:50.315191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:50.315191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:34:50.315191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:34:50.315191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:50.315660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:34:50.315660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:50.315660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:34:55.050881 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:34:55.051756 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:34:55.051756 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:34:55.052996 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:34:55.084573 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:55.085007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:55.086234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:34:55.086234 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:55.086234 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:34:55.086234 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:34:55.086234 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:55.086234 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:34:55.086234 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:55.086234 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:34:55.087461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:55.087461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:34:55.087461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:55.087461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:34:55.087461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:34:55.087461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:34:55.087461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:34:55.087461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:55.087461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:55.087461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:55.087461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:34:55.087461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:55.087916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:55.087916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:55.087916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:34:55.087916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:55.087916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:34:55.087916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:34:55.087916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:55.087916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:34:55.087916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:55.087916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:34:55.087916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:34:55.087916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:55.087916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:34:55.088356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:55.088356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:34:55.088527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:55.088527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:34:55.088527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:34:55.088527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:34:55.088527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:34:55.088527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:34:55.088527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:55.088527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:34:55.088527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:34:55.088527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:34:55.088527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:34:55.088527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:34:55.088527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:55.088527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:55.088527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:34:55.088527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:34:55.089024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:34:55.089024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:34:55.089024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:55.089024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:55.089024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:55.089024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:34:55.089024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:55.089024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:55.089024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:55.089024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:34:55.089024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:55.089024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:34:55.089024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:34:55.089024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:55.089024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:34:55.089024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:55.089024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:34:55.089024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:34:55.089024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:55.089637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:34:55.089637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:55.089637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:34:55.291396 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:34:55.292091 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:34:55.292223 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:34:55.293523 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:55.293861 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:55.294052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:55.295011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:34:55.295011 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:55.295011 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:34:55.295011 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:34:55.295011 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:55.295011 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:34:55.295011 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:55.295011 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:34:55.296586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:55.296586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:34:55.296586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:55.296586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:34:55.296586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:34:55.296586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:34:55.296586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:34:55.296586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:55.296586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:55.296586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:55.296586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:34:55.296586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:55.297392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:55.297392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:55.297392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:34:55.297392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:55.297392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:34:55.297392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:34:55.297392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:55.297392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:34:55.297392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:55.297392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:34:55.297392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:34:55.297392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:55.297392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:34:55.297997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:55.297997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:34:55.298098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:55.298098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:34:55.298098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:34:55.298098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:34:55.298098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:34:55.298098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:34:55.298098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:55.298098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:34:55.298098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:34:55.298098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:34:55.298098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:34:55.298098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:34:55.298098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:55.298098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:55.298098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:34:55.298098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:34:55.298801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:34:55.298801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:34:55.298801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:55.298801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:55.298801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:55.298801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:34:55.298801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:55.298801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:55.298801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:55.298801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:34:55.298801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:55.298801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:34:55.298801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:34:55.298801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:55.298801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:34:55.298801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:55.298801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:34:55.298801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:34:55.298801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:55.299895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:34:55.299895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:55.299895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:34:55.299895 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:34:55.299895 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:34:55.299895 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:34:55.299895 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:34:55.299895 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:55.300314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:55.300959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:34:55.300959 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:55.300959 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:34:55.300959 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:34:55.300959 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:55.300959 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:34:55.300959 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:55.300959 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:34:55.302040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:55.302040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:34:55.302040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:34:55.302040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:34:55.302040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:55.302040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:34:55.302040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:55.302040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:34:55.302040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:55.302040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:34:55.302040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:55.302040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:34:55.302040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:55.302040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:34:55.302040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:34:55.302762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:55.302762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:34:55.302762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:55.302762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:34:55.302762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:34:55.302762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:55.302762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:34:55.302762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:55.302762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:34:55.303269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:55.303269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:34:55.303269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:55.303269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:34:55.303269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:34:55.303269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:34:55.303269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:34:55.303269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:34:55.303269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:34:55.303269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:34:55.303269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:55.303269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:55.303666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:55.303666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:34:55.303666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:55.303666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:55.303666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:55.303666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:34:55.303666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:55.303666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:34:55.303666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:34:55.303666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:55.303666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:34:55.303666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:55.303666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:34:55.303666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:34:55.304071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:55.304071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:34:55.304071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:55.304071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:34:55.304835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:55.304835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:34:55.304835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:34:55.304835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:34:55.304835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:34:55.304835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:34:55.304835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:55.304835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:34:55.304835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:34:55.304835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:34:55.304835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:34:55.304835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:34:55.304835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:55.304835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:55.304835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:34:55.304835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:34:55.305256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:34:55.305256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:34:55.305256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:34:55.305256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:34:55.305256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:34:55.305256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:55.305256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:55.305256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:55.305256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:34:55.305256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:55.305256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:55.305256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:55.305256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:34:55.305256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:55.305256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:34:55.305256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:34:55.305256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:55.305256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:34:55.305754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:55.305754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:34:55.305754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:34:55.305754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:55.305754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:34:55.305754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:55.305754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:34:55.305754 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:34:55.305754 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:34:55.305754 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:34:55.306201 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:34:55.306201 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:55.306275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:55.306958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:34:55.306958 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:55.306958 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:34:55.306958 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:34:55.306958 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:55.306958 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:34:55.306958 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:55.306958 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:34:55.307683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:55.307683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:34:55.307683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:34:55.307683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:34:55.307683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:55.307683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:34:55.307683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:55.307683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:34:55.307683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:55.307683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:34:55.307683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:55.307683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:34:55.307683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:55.307683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:34:55.307683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:34:55.308124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:55.308124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:34:55.308124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:55.308124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:34:55.308124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:34:55.308124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:55.308124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:34:55.308124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:55.308124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:34:55.308544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:55.308544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:34:55.308544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:55.308544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:34:55.308544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:34:55.308544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:34:55.308544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:34:55.308544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:34:55.308544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:34:55.308544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:55.308544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:55.308920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:55.308920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:34:55.308920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:55.308920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:55.308920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:55.308920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:34:55.308920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:55.308920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:34:55.308920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:34:55.308920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:55.308920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:34:55.308920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:55.308920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:34:55.308920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:34:55.309323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:55.309323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:34:55.309323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:55.309323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:34:55.309621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:55.309621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:34:55.309621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:34:55.309621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:34:55.309621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:34:55.309621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:34:55.309621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:55.309621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:34:55.309621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:34:55.309621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:34:55.309621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:34:55.309621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:34:55.309621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:55.309621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:55.309621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:34:55.309621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:34:55.309621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:34:55.310124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:34:55.310124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:34:55.310124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:34:55.310124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:55.310124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:55.310124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:55.310124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:34:55.310124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:55.310124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:55.310124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:55.310124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:34:55.310124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:55.310124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:34:55.310124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:34:55.310124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:55.310124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:34:55.310124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:55.310124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:34:55.310124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:34:55.310657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:55.310657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:34:55.310657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:55.310657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:34:55.311098 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:34:55.311201 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:34:55.311319 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:34:55.311482 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:34:55.311933 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:55.312258 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:34:55.312514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:55.313517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:34:55.313517 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:55.313517 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:34:55.313517 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:34:55.313517 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:55.313517 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:34:55.313517 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:55.313517 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:34:55.313955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:55.313955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:34:55.313955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:55.313955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:34:55.313955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:34:55.313955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:34:55.313955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:34:55.313955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:55.313955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:55.313955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:55.313955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:34:55.313955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:55.314326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:34:55.314326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:55.314326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:34:55.314326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:55.314326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:34:55.314326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:34:55.314326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:55.314326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:34:55.314326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:55.314326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:34:55.314326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:34:55.314326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:55.314326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:34:55.314697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:55.314697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:34:55.314915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:55.314915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:34:55.314915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:34:55.314915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:34:55.314915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:34:55.314915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:34:55.314915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:55.314915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:34:55.314915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:34:55.314915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:34:55.314915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:34:55.314915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:34:55.314915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:34:55.314915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:34:55.314915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:34:55.314915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:34:55.315367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:34:55.315367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:34:55.315367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:55.315367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:55.315367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:34:55.315367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:34:55.315367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:34:55.315367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:34:55.315367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:34:55.315367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:34:55.315367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:34:55.315367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:34:55.315367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:34:55.315367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:34:55.315367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:34:55.315367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:34:55.315367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:34:55.315367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:34:55.315367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:34:55.315930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:34:55.315930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:34:55.315930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:35:00.051513 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:35:00.052177 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:35:00.052177 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:35:00.052857 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:35:00.085711 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:35:00.085918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:00.086430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:35:00.086430 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:00.086430 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:35:00.086430 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:35:00.086430 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:00.086430 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:35:00.086430 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:00.086430 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:35:00.087472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:00.087472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:35:00.087472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:00.087472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:35:00.087472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:35:00.087472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:35:00.087472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:35:00.087472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:00.087472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:00.087472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:00.087472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:35:00.087472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:00.088108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:00.088108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:00.088108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:35:00.088108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:00.088108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:35:00.088108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:35:00.088108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:00.088108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:35:00.088108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:00.088108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:35:00.088108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:35:00.088108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:00.088108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:35:00.088776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:00.088776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:35:00.088776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:00.088776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:35:00.088776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:35:00.088776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:35:00.088776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:35:00.088776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:35:00.088776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:00.088776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:35:00.088776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:35:00.088776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:35:00.088776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:35:00.088776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:35:00.088776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:00.089582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:00.089582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:35:00.089582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:35:00.089582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:35:00.089582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:35:00.089582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:00.089582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:00.089582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:00.089582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:35:00.089582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:00.089582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:00.089582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:00.089582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:35:00.089582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:00.089582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:35:00.089582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:35:00.089582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:00.089582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:35:00.089582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:00.090588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:35:00.090588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:35:00.090588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:00.090588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:35:00.090588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:00.090588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:35:00.292096 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:35:00.292628 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:35:00.293106 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:35:00.294017 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:35:00.294397 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:35:00.294690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:00.295870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:35:00.295870 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:00.295870 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:35:00.295870 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:35:00.295870 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:00.295870 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:35:00.295870 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:00.295870 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:35:00.297345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:00.297345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:35:00.297345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:00.297345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:35:00.297345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:35:00.297345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:35:00.297345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:35:00.297345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:00.297345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:00.297345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:00.297345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:35:00.297345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:00.297888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:00.297888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:00.297888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:35:00.297888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:00.297888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:35:00.297888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:35:00.297888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:00.297888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:35:00.297888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:00.297888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:35:00.297888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:35:00.297888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:00.297888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:35:00.298462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:00.298462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:35:00.299142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:00.299142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:35:00.299142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:35:00.299142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:35:00.299142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:35:00.299142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:35:00.299142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:00.299142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:35:00.299142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:35:00.299142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:35:00.299142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:35:00.299142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:35:00.299142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:00.299142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:00.299142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:35:00.299142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:35:00.299886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:35:00.299886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:35:00.299886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:00.299886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:00.299886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:00.299886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:35:00.299886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:00.299886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:00.299886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:00.299886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:35:00.299886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:00.299886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:35:00.299886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:35:00.299886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:00.299886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:35:00.299886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:00.299886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:35:00.299886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:35:00.299886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:00.300704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:35:00.300704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:00.300704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:35:00.300704 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:35:00.300704 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:35:00.300704 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:35:00.300704 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:35:00.300704 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:35:00.301133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:00.301706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:35:00.301706 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:00.301706 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:35:00.301706 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:35:00.301706 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:00.301706 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:35:00.301706 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:00.301706 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:35:00.302904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:00.302904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:35:00.302904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:35:00.302904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:35:00.302904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:00.302904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:35:00.302904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:00.302904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:35:00.302904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:00.302904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:35:00.302904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:00.302904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:35:00.302904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:00.302904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:35:00.302904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:35:00.303430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:00.303430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:35:00.303430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:00.303430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:35:00.303430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:35:00.303430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:00.303430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:35:00.303430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:00.303430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:35:00.303698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:00.303698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:35:00.303698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:00.303698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:35:00.303698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:35:00.303698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:35:00.303698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:35:00.303698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:35:00.303698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:35:00.303698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:35:00.303698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:00.303698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:00.304122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:00.304122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:35:00.304122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:00.304122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:00.304122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:00.304122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:35:00.304122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:00.304122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:35:00.304122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:35:00.304122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:00.304122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:35:00.304122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:00.304122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:35:00.304122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:35:00.304527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:00.304527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:35:00.304527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:00.304527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:35:00.307808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:00.307808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:35:00.307808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:35:00.307808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:35:00.307808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:35:00.307808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:35:00.307808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:00.307808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:35:00.307808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:35:00.307808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:35:00.307808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:35:00.307808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:35:00.307808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:00.307808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:00.307808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:35:00.307808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:35:00.308242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:35:00.308242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:35:00.308242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:35:00.308242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:35:00.308242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:35:00.308242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:00.308242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:00.308242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:00.308242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:35:00.308242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:00.308242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:00.308242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:00.308242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:35:00.308242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:00.308242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:35:00.308242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:35:00.308242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:00.308242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:35:00.308743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:00.308743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:35:00.308743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:35:00.308743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:00.308743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:35:00.308743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:00.308743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:35:00.308743 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:35:00.308743 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:35:00.308743 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:35:00.309043 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:35:00.309043 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:35:00.309043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:00.309445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:35:00.309445 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:00.309445 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:35:00.309445 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:35:00.309445 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:00.309445 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:35:00.309445 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:00.309445 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:35:00.310109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:00.310109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:35:00.310109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:35:00.310109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:35:00.310109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:00.310109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:35:00.310109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:00.310109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:35:00.310109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:00.310109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:35:00.310109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:00.310109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:35:00.310109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:00.310109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:35:00.310109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:35:00.310541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:00.310541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:35:00.310541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:00.310541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:35:00.310541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:35:00.310541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:00.310541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:35:00.310541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:00.310541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:35:00.310960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:00.310960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:35:00.310960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:00.310960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:35:00.310960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:35:00.310960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:35:00.310960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:35:00.310960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:35:00.310960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:35:00.310960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:00.310960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:00.311278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:00.311278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:35:00.311278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:00.311278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:00.311278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:00.311278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:35:00.311278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:00.311278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:35:00.311278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:35:00.311278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:00.311278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:35:00.311278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:00.311278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:35:00.311278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:35:00.311660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:00.311660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:35:00.311660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:00.311660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:35:00.315019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:00.315019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:35:00.315019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:35:00.315019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:35:00.315019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:35:00.315019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:35:00.315019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:00.315019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:35:00.315019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:35:00.315019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:35:00.315019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:35:00.315019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:35:00.315019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:00.315019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:00.315019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:35:00.315019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:35:00.315019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:35:00.315537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:35:00.315537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:35:00.315537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:35:00.315537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:00.315537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:00.315537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:00.315537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:35:00.315537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:00.315537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:00.315537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:00.315537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:35:00.315537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:00.315537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:35:00.315537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:35:00.315537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:00.315537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:35:00.315537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:00.315537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:35:00.315537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:35:00.316050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:00.316050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:35:00.316050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:00.316050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:35:00.316050 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:35:00.316050 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:35:00.316050 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:35:00.316050 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:35:00.316284 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:35:00.316284 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:35:00.316284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:00.319654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:35:00.319654 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:00.319654 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:35:00.319654 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:35:00.319654 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:00.319654 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:35:00.319654 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:00.319654 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:35:00.320287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:00.320287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:35:00.320287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:00.320287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:35:00.320287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:35:00.320287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:35:00.320287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:35:00.320287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:00.320287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:00.320287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:00.320287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:35:00.320287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:00.320760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:00.320760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:00.320760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:35:00.320760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:00.320760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:35:00.320760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:35:00.320760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:00.320760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:35:00.320760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:00.320760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:35:00.320760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:35:00.320760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:00.320760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:35:00.321113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:00.321113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:35:00.321113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:00.321113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:35:00.321113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:35:00.321113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:35:00.321113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:35:00.321113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:35:00.321113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:00.321113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:35:00.321113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:35:00.321113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:35:00.321113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:35:00.321113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:35:00.321113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:00.321532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:00.321532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:35:00.321532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:35:00.321532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:35:00.321532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:35:00.321532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:00.321532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:00.321532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:00.321532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:35:00.321532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:00.321532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:00.321532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:00.321532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:35:00.321532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:00.321532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:35:00.321532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:35:00.321532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:00.321532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:35:00.321532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:00.322108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:35:00.322108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:35:00.322108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:00.322108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:35:00.322108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:00.322108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:35:05.052199 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:35:05.052950 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:35:05.052950 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:35:05.054167 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:35:05.086885 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:35:05.087080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:05.087879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:35:05.087879 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:05.087879 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:35:05.087879 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:35:05.087879 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:05.087879 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:35:05.087879 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:05.087879 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:35:05.088756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:05.088756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:35:05.088756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:05.088756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:35:05.088756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:35:05.088756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:35:05.088756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:35:05.088756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:05.088756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:05.088756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:05.088756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:35:05.088756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:05.089372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:05.089372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:05.089372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:35:05.089372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:05.089372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:35:05.089372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:35:05.089372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:05.089372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:35:05.089372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:05.089372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:35:05.089372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:35:05.089372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:05.089372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:35:05.090006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:05.090006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:35:05.090006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:05.090006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:35:05.090006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:35:05.090006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:35:05.090006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:35:05.090006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:35:05.090006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:05.090006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:35:05.090006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:35:05.090006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:35:05.090006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:35:05.090006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:35:05.090006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:05.090904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:05.090904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:35:05.090904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:35:05.090904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:35:05.090904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:35:05.090904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:05.090904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:05.090904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:05.090904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:35:05.090904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:05.090904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:05.090904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:05.090904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:35:05.090904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:05.090904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:35:05.090904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:35:05.090904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:05.090904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:35:05.090904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:05.091700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:35:05.091700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:35:05.091700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:05.091700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:35:05.091700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:05.091700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:35:05.292954 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:35:05.293395 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:35:05.293622 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:35:05.294590 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:35:05.294898 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:35:05.295083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:05.296228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:35:05.296228 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:05.296228 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:35:05.296228 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:35:05.296228 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:05.296228 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:35:05.296228 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:05.296228 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:35:05.298625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:05.298625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:35:05.298625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:05.298625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:35:05.298625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:35:05.298625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:35:05.298625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:35:05.298625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:05.298625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:05.298625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:05.298625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:35:05.298625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:05.299464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:05.299464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:05.299464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:35:05.299464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:05.299464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:35:05.299464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:35:05.299464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:05.299464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:35:05.299464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:05.299464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:35:05.299464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:35:05.299464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:05.299464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:35:05.300279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:05.300279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:35:05.300486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:05.300486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:35:05.300486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:35:05.300486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:35:05.300486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:35:05.300486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:35:05.300486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:05.300486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:35:05.300486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:35:05.300486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:35:05.300486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:35:05.300486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:35:05.300486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:05.300486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:05.300486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:35:05.300486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:35:05.301576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:35:05.301576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:35:05.301576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:05.301576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:05.301576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:05.301576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:35:05.301576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:05.301576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:05.301576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:05.301576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:35:05.301576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:05.301576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:35:05.301576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:35:05.301576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:05.301576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:35:05.301576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:05.301576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:35:05.301576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:35:05.301576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:05.302798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:35:05.302798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:05.302798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:35:05.302798 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:35:05.302798 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:35:05.302798 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:35:05.302798 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:35:05.302798 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:35:05.303386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:05.303386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:35:05.303386 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:05.303386 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:35:05.303386 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:35:05.303386 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:05.303386 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:35:05.303386 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:05.303386 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:35:05.303958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:05.303958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:35:05.303958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:35:05.303958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:35:05.303958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:05.303958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:35:05.303958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:05.303958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:35:05.303958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:05.303958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:35:05.303958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:05.303958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:35:05.303958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:05.303958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:35:05.303958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:35:05.305154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:05.305154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:35:05.305154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:05.305154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:35:05.305154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:35:05.305154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:05.305154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:35:05.305154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:05.305154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:35:05.305154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:05.305154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:35:05.305154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:05.305154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:35:05.305154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:35:05.305762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:35:05.305762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:35:05.305762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:35:05.305762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:35:05.305762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:35:05.305762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:05.305762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:05.305762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:05.305762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:35:05.305762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:05.305762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:05.305762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:05.305762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:35:05.306250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:05.306250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:35:05.306250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:35:05.306250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:05.306250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:35:05.306250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:05.306250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:35:05.306250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:35:05.306250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:05.306250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:35:05.306250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:05.306250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:35:05.306858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:05.306858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:35:05.306858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:35:05.306858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:35:05.306858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:35:05.306858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:35:05.306858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:05.306858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:35:05.306858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:35:05.306858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:35:05.306858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:35:05.306858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:35:05.306858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:05.306858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:05.306858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:35:05.306858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:35:05.307940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:35:05.307940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:35:05.307940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:35:05.307940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:35:05.307940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:35:05.307940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:05.307940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:05.307940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:05.307940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:35:05.307940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:05.307940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:05.307940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:05.307940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:35:05.307940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:05.307940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:35:05.307940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:35:05.307940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:05.307940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:35:05.308869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:05.308869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:35:05.308869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:35:05.308869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:05.308869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:35:05.308869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:05.308869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:35:05.308869 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:35:05.308869 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:35:05.309241 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:35:05.309241 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:35:05.309523 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:35:05.309854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:05.310520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:35:05.310520 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:05.310520 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:35:05.310520 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:35:05.310520 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:05.310520 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:35:05.310520 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:05.310520 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:35:05.310985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:05.310985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:35:05.310985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:35:05.310985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:35:05.310985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:05.310985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:35:05.310985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:05.310985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:35:05.310985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:05.310985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:35:05.310985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:05.310985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:35:05.310985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:05.310985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:35:05.310985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:35:05.311551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:05.311551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:35:05.311551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:05.311551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:35:05.311551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:35:05.311551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:05.311551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:35:05.311551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:05.311551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:35:05.311893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:05.311893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:35:05.311893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:05.311893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:35:05.311893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:35:05.311893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:35:05.311893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:35:05.311893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:35:05.311893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:35:05.311893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:05.311893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:05.312304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:05.312304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:35:05.312304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:05.312304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:05.312304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:05.312304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:35:05.312304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:05.312304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:35:05.312304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:35:05.312304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:05.312304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:35:05.312304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:05.312304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:35:05.312304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:35:05.312902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:05.312902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:35:05.312902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:05.312902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:35:05.312902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:05.312902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:35:05.312902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:35:05.312902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:35:05.312902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:35:05.312902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:35:05.312902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:05.312902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:35:05.312902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:35:05.312902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:35:05.313297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:35:05.313297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:35:05.313297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:05.313297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:05.313297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:35:05.313297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:35:05.313297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:35:05.313297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:35:05.313297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:35:05.313297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:35:05.313297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:05.313297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:05.313297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:05.313297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:35:05.313297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:05.313297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:05.313297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:05.313297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:35:05.313297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:05.313810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:35:05.313810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:35:05.313810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:05.313810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:35:05.313810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:05.313810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:35:05.313810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:35:05.313810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:05.313810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:35:05.313810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:05.313810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:35:05.315741 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:35:05.315964 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:35:05.316116 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:35:05.316356 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:35:05.316814 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:35:05.317305 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:35:05.317569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:05.318282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:35:05.318282 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:05.318282 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:35:05.318282 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:35:05.318282 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:05.318282 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:35:05.318282 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:05.318282 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:35:05.319566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:05.319566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:35:05.319566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:05.319566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:35:05.319566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:35:05.319566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:35:05.319566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:35:05.319566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:05.319566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:05.319566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:05.319566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:35:05.319566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:05.319910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:05.319910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:05.319910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:35:05.319910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:05.319910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:35:05.319910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:35:05.319910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:05.319910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:35:05.319910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:05.319910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:35:05.319910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:35:05.319910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:05.319910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:35:05.320263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:05.320263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:35:05.321114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:05.321114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:35:05.321114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:35:05.321114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:35:05.321114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:35:05.321114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:35:05.321114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:05.321114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:35:05.321114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:35:05.321114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:35:05.321114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:35:05.321114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:35:05.321114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:05.321114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:05.321114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:35:05.321114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:35:05.321589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:35:05.321589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:35:05.321589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:05.321589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:05.321589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:05.321589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:35:05.321589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:05.321589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:05.321589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:05.321589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:35:05.321589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:05.321589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:35:05.321589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:35:05.321589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:05.321589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:35:05.321589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:05.321589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:35:05.321589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:35:05.321589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:05.322104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:35:05.322104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:05.322104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:35:10.053607 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:35:10.054245 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:35:10.054301 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:35:10.055533 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:35:10.088033 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:35:10.088559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:10.089261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:35:10.089261 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:10.089261 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:35:10.089261 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:35:10.089261 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:10.089261 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:35:10.089261 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:10.089261 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:35:10.090268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:10.090268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:35:10.090268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:10.090268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:35:10.090268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:35:10.090268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:35:10.090268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:35:10.090268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:10.090268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:10.090268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:10.090268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:35:10.090268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:10.090963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:10.090963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:10.090963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:35:10.090963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:10.090963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:35:10.090963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:35:10.090963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:10.090963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:35:10.090963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:10.090963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:35:10.090963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:35:10.090963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:10.090963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:35:10.091631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:10.091631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:35:10.091631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:10.091631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:35:10.091631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:35:10.091631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:35:10.091631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:35:10.091631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:35:10.091631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:10.091631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:35:10.091631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:35:10.091631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:35:10.091631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:35:10.091631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:35:10.091631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:10.092962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:10.092962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:35:10.092962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:35:10.092962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:35:10.092962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:35:10.092962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:10.092962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:10.092962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:10.092962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:35:10.092962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:10.092962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:10.092962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:10.092962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:35:10.092962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:10.092962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:35:10.092962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:35:10.092962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:10.092962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:35:10.092962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:10.094629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:35:10.094629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:35:10.094629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:10.094629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:35:10.094629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:10.094629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:35:10.293573 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:35:10.294036 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:35:10.294223 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:35:10.295332 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:35:10.295687 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:35:10.295935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:10.296885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:35:10.296885 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:10.296885 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:35:10.296885 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:35:10.296885 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:10.296885 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:35:10.296885 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:10.296885 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:35:10.297914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:10.297914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:35:10.297914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:10.297914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:35:10.297914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:35:10.297914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:35:10.297914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:35:10.297914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:10.297914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:10.297914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:10.297914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:35:10.297914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:10.298278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:10.298278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:10.298278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:35:10.298278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:10.298278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:35:10.298278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:35:10.298278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:10.298278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:35:10.298278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:10.298278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:35:10.298278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:35:10.298278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:10.298278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:35:10.298647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:10.298647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:35:10.299144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:10.299144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:35:10.299144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:35:10.299144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:35:10.299144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:35:10.299144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:35:10.299144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:10.299144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:35:10.299144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:35:10.299144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:35:10.299144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:35:10.299144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:35:10.299144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:10.299144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:10.299144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:35:10.299144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:35:10.299800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:35:10.299800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:35:10.299800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:10.299800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:10.299800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:10.299800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:35:10.299800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:10.299800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:10.299800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:10.299800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:35:10.299800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:10.299800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:35:10.299800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:35:10.299800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:10.299800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:35:10.299800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:10.299800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:35:10.299800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:35:10.299800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:10.300277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:35:10.300277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:10.300277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:35:10.301034 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:35:10.301276 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:35:10.301374 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:35:10.301517 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:35:10.301977 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:35:10.302266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:10.302945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:35:10.302945 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:10.302945 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:35:10.302945 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:35:10.302945 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:10.302945 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:35:10.302945 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:10.302945 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:35:10.303747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:10.303747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:35:10.303747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:35:10.303747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:35:10.303747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:10.303747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:35:10.303747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:10.303747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:35:10.303747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:10.303747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:35:10.303747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:10.303747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:35:10.303747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:10.303747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:35:10.303747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:35:10.304229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:10.304229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:35:10.304229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:10.304229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:35:10.304229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:35:10.304229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:10.304229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:35:10.304229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:10.304229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:35:10.304575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:10.304575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:35:10.304575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:10.304575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:35:10.304575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:35:10.304575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:35:10.304575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:35:10.304575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:35:10.304575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:35:10.304575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:35:10.304575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:10.304575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:10.304951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:10.304951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:35:10.304951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:10.304951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:10.304951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:10.304951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:35:10.304951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:10.304951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:35:10.304951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:35:10.304951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:10.304951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:35:10.304951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:10.304951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:35:10.304951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:35:10.305337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:10.305337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:35:10.305337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:10.305337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:35:10.305601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:10.305601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:35:10.305601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:35:10.305601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:35:10.305601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:35:10.305601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:35:10.305601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:10.305601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:35:10.305601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:35:10.305601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:35:10.305601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:35:10.305601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:35:10.305601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:10.305601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:10.305601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:35:10.305601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:35:10.306054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:35:10.306054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:35:10.306054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:35:10.306054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:35:10.306054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:35:10.306054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:10.306054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:10.306054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:10.306054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:35:10.306054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:10.306054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:10.306054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:10.306054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:35:10.306054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:10.306054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:35:10.306054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:35:10.306054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:10.306054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:35:10.306559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:10.306559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:35:10.306559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:35:10.306559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:10.306559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:35:10.306559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:10.306559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:35:10.309113 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:35:10.309385 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:35:10.309526 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:35:10.309652 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:35:10.310098 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:35:10.310527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:10.311161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:35:10.311161 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:10.311161 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:35:10.311161 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:35:10.311161 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:10.311161 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:35:10.311161 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:10.311161 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:35:10.311901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:10.311901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:35:10.311901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:35:10.311901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:35:10.311901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:10.311901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:35:10.311901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:10.311901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:35:10.311901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:10.311901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:35:10.311901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:10.311901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:35:10.311901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:10.311901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:35:10.311901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:35:10.312433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:10.312433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:35:10.312433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:10.312433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:35:10.312433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:35:10.312433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:10.312433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:35:10.312433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:10.312433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:35:10.312786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:10.312786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:35:10.312786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:10.312786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:35:10.312786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:35:10.312786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:35:10.312786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:35:10.312786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:35:10.312786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:35:10.312786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:10.312786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:10.313248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:10.313248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:35:10.313248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:10.313248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:10.313248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:10.313248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:35:10.313248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:10.313248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:35:10.313248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:35:10.313248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:10.313248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:35:10.313248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:10.313248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:35:10.313248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:35:10.313669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:10.313669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:35:10.313669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:10.313669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:35:10.313870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:10.313870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:35:10.313870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:35:10.313870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:35:10.313870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:35:10.313870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:35:10.313870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:10.313870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:35:10.313870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:35:10.313870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:35:10.313870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:35:10.313870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:35:10.313870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:10.313870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:10.313870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:35:10.313870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:35:10.313870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:35:10.314346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:35:10.314346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:35:10.314346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:35:10.314346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:10.314346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:10.314346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:10.314346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:35:10.314346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:10.314346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:10.314346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:10.314346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:35:10.314346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:10.314346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:35:10.314346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:35:10.314346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:10.314346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:35:10.314346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:10.314346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:35:10.314346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:35:10.315292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:10.315292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:35:10.315292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:10.315292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:35:10.316128 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:35:10.316396 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:35:10.316539 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:35:10.316718 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:35:10.317144 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:35:10.317526 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:35:10.317761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:10.318460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:35:10.318460 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:10.318460 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:35:10.318460 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:35:10.318460 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:10.318460 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:35:10.318460 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:10.318460 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:35:10.319455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:10.319455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:35:10.319455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:10.319455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:35:10.319455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:35:10.319455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:35:10.319455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:35:10.319455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:10.319455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:10.319455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:10.319455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:35:10.319455 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:10.319868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:10.319868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:10.319868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:35:10.319868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:10.319868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:35:10.319868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:35:10.319868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:10.319868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:35:10.319868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:10.319868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:35:10.319868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:35:10.319868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:10.319868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:35:10.320203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:10.320203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:35:10.320362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:10.320362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:35:10.320362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:35:10.320362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:35:10.320362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:35:10.320362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:35:10.320362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:10.320362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:35:10.320362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:35:10.320362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:35:10.320362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:35:10.320362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:35:10.320362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:10.320362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:10.320362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:35:10.320362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:35:10.321054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:35:10.321054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:35:10.321054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:10.321054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:10.321054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:10.321054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:35:10.321054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:10.321054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:10.321054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:10.321054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:35:10.321054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:10.321054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:35:10.321054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:35:10.321054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:10.321054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:35:10.321054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:10.321054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:35:10.321054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:35:10.321054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:10.321618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:35:10.321618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:10.321618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:35:15.054196 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:35:15.055181 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:35:15.055181 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:35:15.056237 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:35:15.088351 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:35:15.088775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:15.089674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:35:15.089674 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:15.089674 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:35:15.089674 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:35:15.089674 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:15.089674 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:35:15.089674 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:15.089674 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:35:15.090387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:15.090387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:35:15.090387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:15.090387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:35:15.090387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:35:15.090387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:35:15.090387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:35:15.090387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:15.090387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:15.090387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:15.090387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:35:15.090387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:15.091105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:15.091105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:15.091105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:35:15.091105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:15.091105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:35:15.091105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:35:15.091105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:15.091105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:35:15.091105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:15.091105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:35:15.091105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:35:15.091105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:15.091105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:35:15.091960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:15.091960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:35:15.091960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:15.091960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:35:15.091960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:35:15.091960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:35:15.091960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:35:15.091960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:35:15.091960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:15.091960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:35:15.091960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:35:15.091960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:35:15.091960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:35:15.091960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:35:15.091960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:15.093520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:15.093520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:35:15.093520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:35:15.093520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:35:15.093520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:35:15.093520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:15.093520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:15.093520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:15.093520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:35:15.093520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:15.093520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:15.093520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:15.093520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:35:15.093520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:15.093520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:35:15.093520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:35:15.093520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:15.093520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:35:15.093520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:15.095095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:35:15.095095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:35:15.095095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:15.095095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:35:15.095095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:15.095095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:35:15.294258 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:35:15.295041 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:35:15.295260 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:35:15.296381 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:35:15.296787 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:35:15.297053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:15.298105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:35:15.298105 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:15.298105 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:35:15.298105 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:35:15.298105 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:15.298105 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:35:15.298105 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:15.298105 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:35:15.299589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:15.299589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:35:15.299589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:15.299589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:35:15.299589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:35:15.299589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:35:15.299589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:35:15.299589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:15.299589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:15.299589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:15.299589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:35:15.299589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:15.300158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:15.300158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:15.300158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:35:15.300158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:15.300158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:35:15.300158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:35:15.300158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:15.300158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:35:15.300158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:15.300158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:35:15.300158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:35:15.300158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:15.300158 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:35:15.300706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:15.300706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:35:15.301288 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:15.301288 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:35:15.301288 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:35:15.301288 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:35:15.301288 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:35:15.301288 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:35:15.301288 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:15.301288 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:35:15.301288 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:35:15.301288 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:35:15.301288 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:35:15.301288 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:35:15.301288 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:15.301288 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:15.301288 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:35:15.301288 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:35:15.302849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:35:15.302849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:35:15.302849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:15.302849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:15.302849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:15.302849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:35:15.302849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:15.302849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:15.302849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:15.302849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:35:15.302849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:15.302849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:35:15.302849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:35:15.302849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:15.302849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:35:15.302849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:15.302849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:35:15.302849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:35:15.302849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:15.304518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:35:15.304518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:15.304518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:35:15.304518 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:35:15.304518 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:35:15.304518 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:35:15.304518 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:35:15.304518 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:35:15.304999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:15.304999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:35:15.304999 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:15.304999 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:35:15.304999 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:35:15.304999 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:15.304999 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:35:15.304999 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:15.304999 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:35:15.305360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:15.305360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:35:15.305360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:35:15.305360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:35:15.305360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:15.305360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:35:15.305360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:15.305360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:35:15.305360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:15.305360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:35:15.305360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:15.305360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:35:15.305360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:15.305360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:35:15.305360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:35:15.307028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:15.307028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:35:15.307028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:15.307028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:35:15.307028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:35:15.307028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:15.307028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:35:15.307028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:15.307028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:35:15.307028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:15.307028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:35:15.307028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:15.307028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:35:15.307028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:35:15.309021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:35:15.309021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:35:15.309021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:35:15.309021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:35:15.309021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:35:15.309021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:15.309021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:15.309021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:15.309021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:35:15.309021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:15.309021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:15.309021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:15.309021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:35:15.309801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:15.309801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:35:15.309801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:35:15.309801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:15.309801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:35:15.309801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:15.309801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:35:15.309801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:35:15.309801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:15.309801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:35:15.309801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:15.309801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:35:15.310297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:15.310297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:35:15.310297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:35:15.310297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:35:15.310297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:35:15.310297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:35:15.310297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:15.310297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:35:15.310297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:35:15.310297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:35:15.310297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:35:15.310297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:35:15.310297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:15.310297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:15.310297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:35:15.310297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:35:15.311181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:35:15.311181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:35:15.311181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:35:15.311181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:35:15.311181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:35:15.311181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:15.311181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:15.311181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:15.311181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:35:15.311181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:15.311181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:15.311181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:15.311181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:35:15.311181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:15.311181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:35:15.311181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:35:15.311181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:15.311181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:35:15.311917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:15.311917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:35:15.311917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:35:15.311917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:15.311917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:35:15.311917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:15.311917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:35:15.311917 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:35:15.311917 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:35:15.311917 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:35:15.312320 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:35:15.312320 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:35:15.312320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:15.312320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:35:15.312320 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:15.312320 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:35:15.312320 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:35:15.312320 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:15.312320 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:35:15.312320 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:15.312320 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:35:15.312320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:15.312810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:35:15.312810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:35:15.312810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:35:15.312810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:15.312810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:35:15.312810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:15.312810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:35:15.312810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:15.312810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:35:15.312810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:15.312810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:35:15.312810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:15.312810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:35:15.312810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:35:15.312810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:15.312810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:35:15.312810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:15.313487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:35:15.313487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:35:15.313487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:15.313487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:35:15.313487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:15.313487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:35:15.313487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:15.313487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:35:15.313487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:15.313487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:35:15.313487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:35:15.313487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:35:15.313487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:35:15.313813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:35:15.313813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:35:15.313813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:15.313813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:15.313813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:15.313813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:35:15.313813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:15.313813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:15.313813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:15.313813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:35:15.313813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:15.313813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:35:15.313813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:35:15.314044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:15.314044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:35:15.314044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:15.314044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:35:15.314044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:35:15.314044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:15.314044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:35:15.314044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:15.314044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:35:15.314210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:15.314210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:35:15.314210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:35:15.314210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:35:15.314210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:35:15.314210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:35:15.314210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:15.314210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:35:15.314210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:35:15.314210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:35:15.314210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:35:15.314210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:35:15.314210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:15.314210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:15.314210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:35:15.314210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:35:15.314210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:35:15.314533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:35:15.314533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:35:15.314533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:35:15.314533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:15.314533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:15.314533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:15.314533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:35:15.314533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:15.314533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:15.314533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:15.314533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:35:15.314533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:15.314533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:35:15.314533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:35:15.314533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:15.314533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:35:15.314533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:15.314533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:35:15.314533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:35:15.314858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:15.314858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:35:15.314858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:15.314858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:35:15.316974 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:35:15.317315 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:35:15.317439 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:35:15.317589 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:35:15.318061 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:35:15.318387 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:35:15.318632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:15.319317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:35:15.319317 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:15.319317 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:35:15.319317 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:35:15.319317 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:15.319317 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:35:15.319317 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:15.319317 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:35:15.320166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:15.320166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:35:15.320166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:15.320166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:35:15.320166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:35:15.320166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:35:15.320166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:35:15.320166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:15.320166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:15.320166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:15.320166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:35:15.320166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:15.320531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:15.320531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:15.320531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:35:15.320531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:15.320531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:35:15.320531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:35:15.320531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:15.320531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:35:15.320531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:15.320531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:35:15.320531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:35:15.320531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:15.320531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:35:15.320867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:15.320867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:35:15.321137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:15.321137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:35:15.321137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:35:15.321137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:35:15.321137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:35:15.321137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:35:15.321137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:15.321137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:35:15.321137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:35:15.321137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:35:15.321137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:35:15.321137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:35:15.321137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:15.321137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:15.321137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:35:15.321137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:35:15.321605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:35:15.321605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:35:15.321605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:15.321605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:15.321605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:15.321605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:35:15.321605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:15.321605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:15.321605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:15.321605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:35:15.321605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:15.321605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:35:15.321605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:35:15.321605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:15.321605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:35:15.321605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:15.321605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:35:15.321605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:35:15.321605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:15.322182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:35:15.322182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:15.322182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:35:20.054780 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:35:20.055703 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:35:20.055703 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:35:20.056158 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:35:20.088051 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:35:20.088499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:20.089382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:35:20.089382 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:20.089382 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:35:20.089382 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:35:20.089382 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:20.089382 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:35:20.089382 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:20.089382 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:35:20.090259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:20.090259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:35:20.090259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:20.090259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:35:20.090259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:35:20.090259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:35:20.090259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:35:20.090259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:20.090259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:20.090259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:20.090259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:35:20.090259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:20.091165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:20.091165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:20.091165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:35:20.091165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:20.091165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:35:20.091165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:35:20.091165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:20.091165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:35:20.091165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:20.091165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:35:20.091165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:35:20.091165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:20.091165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:35:20.091853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:20.091853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:35:20.091853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:20.091853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:35:20.091853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:35:20.091853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:35:20.091853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:35:20.091853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:35:20.091853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:20.091853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:35:20.091853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:35:20.091853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:35:20.091853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:35:20.091853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:35:20.091853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:20.092765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:20.092765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:35:20.092765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:35:20.092765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:35:20.092765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:35:20.092765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:20.092765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:20.092765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:20.092765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:35:20.092765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:20.092765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:20.092765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:20.092765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:35:20.092765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:20.092765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:35:20.092765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:35:20.092765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:20.092765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:35:20.092765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:20.094483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:35:20.094483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:35:20.094483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:20.094483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:35:20.094483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:20.094483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:35:20.294890 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:35:20.295433 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:35:20.295624 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:35:20.296784 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:35:20.297126 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:35:20.297366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:20.298323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:35:20.298323 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:20.298323 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:35:20.298323 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:35:20.298323 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:20.298323 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:35:20.298323 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:20.298323 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:35:20.299821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:20.299821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:35:20.299821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:20.299821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:35:20.299821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:35:20.299821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:35:20.299821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:35:20.299821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:20.299821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:20.299821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:20.299821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:35:20.299821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:20.300466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:20.300466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:20.300466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:35:20.300466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:20.300466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:35:20.300466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:35:20.300466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:20.300466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:35:20.300466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:20.300466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:35:20.300466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:35:20.300466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:20.300466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:35:20.301074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:20.301074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:35:20.301430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:20.301430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:35:20.301430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:35:20.301430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:35:20.301430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:35:20.301430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:35:20.301430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:20.301430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:35:20.301430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:35:20.301430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:35:20.301430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:35:20.301430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:35:20.301430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:20.301430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:20.301430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:35:20.301430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:35:20.302250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:35:20.302250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:35:20.302250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:20.302250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:20.302250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:20.302250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:35:20.302250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:20.302250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:20.302250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:20.302250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:35:20.302250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:20.302250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:35:20.302250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:35:20.302250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:20.302250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:35:20.302250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:20.302250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:35:20.302250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:35:20.302250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:20.302904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:35:20.302904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:20.302904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:35:20.302904 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:35:20.302904 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:35:20.302904 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:35:20.302904 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:35:20.303145 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:35:20.303291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:20.303968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:35:20.303968 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:20.303968 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:35:20.303968 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:35:20.303968 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:20.303968 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:35:20.303968 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:20.303968 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:35:20.304709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:20.304709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:35:20.304709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:35:20.304709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:35:20.304709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:20.304709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:35:20.304709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:20.304709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:35:20.304709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:20.304709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:35:20.304709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:20.304709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:35:20.304709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:20.304709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:35:20.304709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:35:20.305281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:20.305281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:35:20.305281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:20.305281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:35:20.305281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:35:20.305281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:20.305281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:35:20.305281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:20.305281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:35:20.305635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:20.305635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:35:20.305635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:20.305635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:35:20.305635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:35:20.305635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:35:20.305635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:35:20.305635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:35:20.305635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:35:20.305635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:35:20.305635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:20.305635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:20.306074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:20.306074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:35:20.306074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:20.306074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:20.306074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:20.306074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:35:20.306074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:20.306074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:35:20.306074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:35:20.306074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:20.306074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:35:20.306074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:20.306074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:35:20.306074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:35:20.306543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:20.306543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:35:20.306543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:20.306543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:35:20.306681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:20.306681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:35:20.306681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:35:20.306681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:35:20.306681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:35:20.306681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:35:20.306681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:20.306681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:35:20.306681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:35:20.306681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:35:20.306681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:35:20.306681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:35:20.306681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:20.306681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:20.306681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:35:20.306681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:35:20.307123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:35:20.307123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:35:20.307123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:35:20.307123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:35:20.307123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:35:20.307123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:20.307123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:20.307123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:20.307123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:35:20.307123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:20.307123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:20.307123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:20.307123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:35:20.307123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:20.307123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:35:20.307123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:35:20.307123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:20.307123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:35:20.307677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:20.307677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:35:20.307677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:35:20.307677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:20.307677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:35:20.307677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:20.307677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:35:20.309911 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:35:20.310121 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:35:20.310267 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:35:20.310428 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:35:20.310883 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:35:20.311249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:20.311960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:35:20.311960 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:20.311960 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:35:20.311960 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:35:20.311960 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:20.311960 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:35:20.311960 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:20.311960 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:35:20.312747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:20.312747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:35:20.312747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:35:20.312747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:35:20.312747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:20.312747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:35:20.312747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:20.312747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:35:20.312747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:20.312747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:35:20.312747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:20.312747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:35:20.312747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:20.312747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:35:20.312747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:35:20.313178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:20.313178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:35:20.313178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:20.313178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:35:20.313178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:35:20.313178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:20.313178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:35:20.313178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:20.313178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:35:20.313507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:20.313507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:35:20.313507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:20.313507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:35:20.313507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:35:20.313507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:35:20.313507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:35:20.313507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:35:20.313507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:35:20.313507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:20.313507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:20.313851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:20.313851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:35:20.313851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:20.313851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:20.313851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:20.313851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:35:20.313851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:20.313851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:35:20.313851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:35:20.313851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:20.313851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:35:20.313851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:20.313851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:35:20.313851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:35:20.314278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:20.314278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:35:20.314278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:20.314278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:35:20.314551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:20.314551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:35:20.314551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:35:20.314551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:35:20.314551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:35:20.314551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:35:20.314551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:20.314551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:35:20.314551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:35:20.314551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:35:20.314551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:35:20.314551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:35:20.314551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:20.314551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:20.314551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:35:20.314551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:35:20.314551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:35:20.315063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:35:20.315063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:35:20.315063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:35:20.315063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:20.315063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:20.315063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:20.315063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:35:20.315063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:20.315063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:20.315063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:20.315063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:35:20.315063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:20.315063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:35:20.315063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:35:20.315063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:20.315063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:35:20.315063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:20.315063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:35:20.315063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:35:20.315625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:20.315625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:35:20.315625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:20.315625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:35:20.317323 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:35:20.317561 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:35:20.317662 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:35:20.317800 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:35:20.318465 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:35:20.318654 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:35:20.318963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:20.319302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:35:20.319302 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:20.319302 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:35:20.319302 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:35:20.319302 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:20.319302 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:35:20.319302 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:20.319302 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:35:20.320120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:20.320120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:35:20.320120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:20.320120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:35:20.320120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:35:20.320120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:35:20.320120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:35:20.320120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:20.320120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:20.320120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:20.320120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:35:20.320120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:20.320563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:20.320563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:20.320563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:35:20.320563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:20.320563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:35:20.320563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:35:20.320563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:20.320563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:35:20.320563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:20.320563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:35:20.320563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:35:20.320563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:20.320563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:35:20.321042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:20.321042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:35:20.321189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:20.321189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:35:20.321189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:35:20.321189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:35:20.321189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:35:20.321189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:35:20.321189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:20.321189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:35:20.321189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:35:20.321189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:35:20.321189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:35:20.321189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:35:20.321189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:20.321189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:20.321189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:35:20.321189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:35:20.321792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:35:20.321792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:35:20.321792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:20.321792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:20.321792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:20.321792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:35:20.321792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:20.321792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:20.321792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:20.321792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:35:20.321792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:20.321792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:35:20.321792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:35:20.321792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:20.321792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:35:20.321792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:20.321792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:35:20.321792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:35:20.321792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:20.322385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:35:20.322385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:20.322385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:35:25.056006 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:35:25.056598 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:35:25.056769 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:35:25.057974 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:35:25.060365 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG dbcounter [-] [105581] Writing DB stats neutron:SELECT=434 {{(pid=105581) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 14:35:25.090078 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:35:25.090494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:25.091278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:35:25.091278 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:25.091278 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:35:25.091278 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:35:25.091278 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:25.091278 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:35:25.091278 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:25.091278 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:35:25.092304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:25.092304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:35:25.092304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:25.092304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:35:25.092304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:35:25.092304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:35:25.092304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:35:25.092304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:25.092304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:25.092304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:25.092304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:35:25.092304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:25.092990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:25.092990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:25.092990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:35:25.092990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:25.092990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:35:25.092990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:35:25.092990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:25.092990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:35:25.092990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:25.092990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:35:25.092990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:35:25.092990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:25.092990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:35:25.093688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:25.093688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:35:25.093688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:25.093688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:35:25.093688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:35:25.093688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:35:25.093688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:35:25.093688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:35:25.093688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:25.093688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:35:25.093688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:35:25.093688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:35:25.093688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:35:25.093688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:35:25.093688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:25.094500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:25.094500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:35:25.094500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:35:25.094500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:35:25.094500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:35:25.094500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:25.094500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:25.094500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:25.094500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:35:25.094500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:25.094500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:25.094500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:25.094500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:35:25.094500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:25.094500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:35:25.094500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:35:25.094500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:25.094500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:35:25.094500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:25.095356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:35:25.095356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:35:25.095356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:25.095356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:35:25.095356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:25.095356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:35:25.295496 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:35:25.296119 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:35:25.296318 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:35:25.297584 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:35:25.298763 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:35:25.298935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:25.299605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:35:25.299605 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:25.299605 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:35:25.299605 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:35:25.299605 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:25.299605 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:35:25.299605 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:25.299605 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:35:25.301105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:25.301105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:35:25.301105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:25.301105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:35:25.301105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:35:25.301105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:35:25.301105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:35:25.301105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:25.301105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:25.301105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:25.301105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:35:25.301105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:25.302013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:25.302013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:25.302013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:35:25.302013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:25.302013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:35:25.302013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:35:25.302013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:25.302013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:35:25.302013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:25.302013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:35:25.302013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:35:25.302013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:25.302013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:35:25.302642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:25.302642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:35:25.302870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:25.302870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:35:25.302870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:35:25.302870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:35:25.302870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:35:25.302870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:35:25.302870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:25.302870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:35:25.302870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:35:25.302870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:35:25.302870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:35:25.302870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:35:25.302870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:25.302870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:25.302870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:35:25.302870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:35:25.303666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:35:25.303666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:35:25.303666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:25.303666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:25.303666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:25.303666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:35:25.303666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:25.303666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:25.303666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:25.303666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:35:25.303666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:25.303666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:35:25.303666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:35:25.303666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:25.303666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:35:25.303666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:25.303666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:35:25.303666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:35:25.303666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:25.304609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:35:25.304609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:25.304609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:35:25.304609 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:35:25.304609 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:35:25.304609 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:35:25.304609 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:35:25.304609 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:35:25.305093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:25.305093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:35:25.305093 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:25.305093 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:35:25.305093 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:35:25.305093 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:25.305093 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:35:25.305093 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:25.305093 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:35:25.305569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:25.305569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:35:25.305569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:35:25.305569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:35:25.305569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:25.305569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:35:25.305569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:25.305569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:35:25.305569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:25.305569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:35:25.305569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:25.305569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:35:25.305569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:25.305569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:35:25.305569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:35:25.306370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:25.306370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:35:25.306370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:25.306370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:35:25.306370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:35:25.306370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:25.306370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:35:25.306370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:25.306370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:35:25.306370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:25.306370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:35:25.306370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:25.306370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:35:25.306370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:35:25.307124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:35:25.307124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:35:25.307124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:35:25.307124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:35:25.307124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:35:25.307124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:25.307124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:25.307124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:25.307124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:35:25.307124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:25.307124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:25.307124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:25.307124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:35:25.307780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:25.307780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:35:25.307780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:35:25.307780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:25.307780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:35:25.307780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:25.307780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:35:25.307780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:35:25.307780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:25.307780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:35:25.307780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:25.307780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:35:25.308320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:25.308320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:35:25.308320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:35:25.308320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:35:25.308320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:35:25.308320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:35:25.308320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:25.308320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:35:25.308320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:35:25.308320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:35:25.308320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:35:25.308320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:35:25.308320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:25.308320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:25.308320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:35:25.308320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:35:25.309103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:35:25.309103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:35:25.309103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:35:25.309103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:35:25.309103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:35:25.309103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:25.309103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:25.309103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:25.309103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:35:25.309103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:25.309103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:25.309103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:25.309103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:35:25.309103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:25.309103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:35:25.309103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:35:25.309103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:25.309103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:35:25.310092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:25.310092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:35:25.310092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:35:25.310092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:25.310092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:35:25.310092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:25.310092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:35:25.310498 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:35:25.310498 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:35:25.310644 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:35:25.310770 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:35:25.311033 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:35:25.311248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:25.311698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:35:25.311698 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:25.311698 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:35:25.311698 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:35:25.311698 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:25.311698 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:35:25.311698 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:25.311698 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:35:25.312363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:25.312363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:35:25.312363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:35:25.312363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:35:25.312363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:25.312363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:35:25.312363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:25.312363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:35:25.312363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:25.312363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:35:25.312363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:25.312363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:35:25.312363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:25.312363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:35:25.312363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:35:25.312667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:25.312667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:35:25.312667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:25.312667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:35:25.312667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:35:25.312667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:25.312667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:35:25.312667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:25.312667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:35:25.313193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:25.313193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:35:25.313193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:25.313193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:35:25.313193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:35:25.313193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:35:25.313193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:35:25.313193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:35:25.313193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:35:25.313193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:25.313193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:25.313446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:25.313446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:35:25.313446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:25.313446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:25.313446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:25.313446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:35:25.313446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:25.313446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:35:25.313446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:35:25.313446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:25.313446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:35:25.313446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:25.313446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:35:25.313446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:35:25.313709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:25.313709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:35:25.313709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:25.313709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:35:25.314224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:25.314224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:35:25.314224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:35:25.314224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:35:25.314224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:35:25.314224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:35:25.314224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:25.314224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:35:25.314224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:35:25.314224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:35:25.314224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:35:25.314224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:35:25.314224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:25.314224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:25.314224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:35:25.314224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:35:25.314224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:35:25.314580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:35:25.314580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:35:25.314580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:35:25.314580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:25.314580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:25.314580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:25.314580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:35:25.314580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:25.314580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:25.314580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:25.314580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:35:25.314580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:25.314580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:35:25.314580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:35:25.314580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:25.314580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:35:25.314580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:25.314580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:35:25.314580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:35:25.314955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:25.314955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:35:25.314955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:25.314955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:35:25.317575 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:35:25.317795 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:35:25.317899 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:35:25.318044 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:35:25.318337 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:35:25.318563 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:35:25.318775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:25.319432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:35:25.319432 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:25.319432 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:35:25.319432 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:35:25.319432 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:25.319432 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:35:25.319432 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:25.319432 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:35:25.320193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:25.320193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:35:25.320193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:25.320193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:35:25.320193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:35:25.320193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:35:25.320193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:35:25.320193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:25.320193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:25.320193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:25.320193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:35:25.320193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:25.320740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:25.320740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:25.320740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:35:25.320740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:25.320740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:35:25.320740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:35:25.320740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:25.320740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:35:25.320740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:25.320740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:35:25.320740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:35:25.320740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:25.320740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:35:25.321104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:25.321104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:35:25.321169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:25.321169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:35:25.321169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:35:25.321169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:35:25.321169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:35:25.321169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:35:25.321169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:25.321169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:35:25.321169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:35:25.321169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:35:25.321169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:35:25.321169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:35:25.321169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:25.321169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:25.321169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:35:25.321169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:35:25.321694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:35:25.321694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:35:25.321694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:25.321694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:25.321694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:25.321694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:35:25.321694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:25.321694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:25.321694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:25.321694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:35:25.321694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:25.321694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:35:25.321694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:35:25.321694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:25.321694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:35:25.321694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:25.321694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:35:25.321694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:35:25.321694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:25.322361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:35:25.322361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:25.322361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:35:30.056841 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:35:30.057762 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:35:30.057762 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:35:30.058897 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:35:30.094530 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:35:30.094922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:30.095923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:35:30.095923 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:30.095923 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:35:30.095923 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:35:30.095923 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:30.095923 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:35:30.095923 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:30.095923 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:35:30.096842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:30.096842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:35:30.096842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:30.096842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:35:30.096842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:35:30.096842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:35:30.096842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:35:30.096842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:30.096842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:30.096842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:30.096842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:35:30.096842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:30.097835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:30.097835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:30.097835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:35:30.097835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:30.097835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:35:30.097835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:35:30.097835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:30.097835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:35:30.097835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:30.097835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:35:30.097835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:35:30.097835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:30.097835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:35:30.098678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:30.098678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:35:30.098678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:30.098678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:35:30.098678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:35:30.098678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:35:30.098678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:35:30.098678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:35:30.098678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:30.098678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:35:30.098678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:35:30.098678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:35:30.098678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:35:30.098678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:35:30.098678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:30.099651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:30.099651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:35:30.099651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:35:30.099651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:35:30.099651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:35:30.099651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:30.099651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:30.099651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:30.099651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:35:30.099651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:30.099651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:30.099651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:30.099651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:35:30.099651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:30.099651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:35:30.099651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:35:30.099651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:30.099651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:35:30.099651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:30.100314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:35:30.100314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:35:30.100314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:30.100314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:35:30.100314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:30.100314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:35:30.296103 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:35:30.296778 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:35:30.296973 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:35:30.297965 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:35:30.298248 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:35:30.298523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:30.299328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:35:30.299328 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:30.299328 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:35:30.299328 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:35:30.299328 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:30.299328 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:35:30.299328 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:30.299328 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:35:30.300955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:30.300955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:35:30.300955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:30.300955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:35:30.300955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:35:30.300955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:35:30.300955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:35:30.300955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:30.300955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:30.300955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:30.300955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:35:30.300955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:30.301653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:30.301653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:30.301653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:35:30.301653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:30.301653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:35:30.301653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:35:30.301653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:30.301653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:35:30.301653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:30.301653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:35:30.301653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:35:30.301653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:30.301653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:35:30.302226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:30.302226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:35:30.302554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:30.302554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:35:30.302554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:35:30.302554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:35:30.302554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:35:30.302554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:35:30.302554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:30.302554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:35:30.302554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:35:30.302554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:35:30.302554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:35:30.302554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:35:30.302554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:30.302554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:30.302554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:35:30.302554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:35:30.303312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:35:30.303312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:35:30.303312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:30.303312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:30.303312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:30.303312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:35:30.303312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:30.303312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:30.303312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:30.303312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:35:30.303312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:30.303312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:35:30.303312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:35:30.303312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:30.303312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:35:30.303312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:30.303312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:35:30.303312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:35:30.303312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:30.304200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:35:30.304200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:30.304200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:35:30.304200 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:35:30.304200 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:35:30.304200 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:35:30.304576 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:35:30.304764 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:35:30.304998 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:30.305447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:35:30.305447 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:30.305447 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:35:30.305447 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:35:30.305447 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:30.305447 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:35:30.305447 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:30.305447 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:35:30.306161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:30.306161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:35:30.306161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:35:30.306161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:35:30.306161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:30.306161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:35:30.306161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:30.306161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:35:30.306161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:30.306161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:35:30.306161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:30.306161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:35:30.306161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:30.306161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:35:30.306161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:35:30.306656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:30.306656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:35:30.306656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:30.306656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:35:30.306656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:35:30.306656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:30.306656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:35:30.306656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:30.306656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:35:30.307017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:30.307017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:35:30.307017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:30.307017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:35:30.307017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:35:30.307017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:35:30.307017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:35:30.307017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:35:30.307017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:35:30.307017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:35:30.307017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:30.307017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:30.307482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:30.307482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:35:30.307482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:30.307482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:30.307482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:30.307482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:35:30.307482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:30.307482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:35:30.307482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:35:30.307482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:30.307482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:35:30.307482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:30.307482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:35:30.307482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:35:30.307907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:30.307907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:35:30.307907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:30.307907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:35:30.308027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:30.308027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:35:30.308027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:35:30.308027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:35:30.308027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:35:30.308027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:35:30.308027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:30.308027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:35:30.308027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:35:30.308027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:35:30.308027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:35:30.308027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:35:30.308027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:30.308027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:30.308027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:35:30.308027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:35:30.308553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:35:30.308553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:35:30.308553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:35:30.308553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:35:30.308553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:35:30.308553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:30.308553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:30.308553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:30.308553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:35:30.308553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:30.308553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:30.308553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:30.308553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:35:30.308553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:30.308553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:35:30.308553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:35:30.308553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:30.308553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:35:30.309068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:30.309068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:35:30.309068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:35:30.309068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:30.309068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:35:30.309068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:30.309068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:35:30.310579 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:35:30.310824 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:35:30.310932 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:35:30.311055 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:35:30.311337 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:35:30.311584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:30.312045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:35:30.312045 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:30.312045 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:35:30.312045 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:35:30.312045 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:30.312045 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:35:30.312045 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:30.312045 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:35:30.312748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:30.312748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:35:30.312748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:35:30.312748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:35:30.312748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:30.312748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:35:30.312748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:30.312748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:35:30.312748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:30.312748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:35:30.312748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:30.312748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:35:30.312748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:30.312748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:35:30.312748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:35:30.313172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:30.313172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:35:30.313172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:30.313172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:35:30.313172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:35:30.313172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:30.313172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:35:30.313172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:30.313172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:35:30.313669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:30.313669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:35:30.313669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:30.313669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:35:30.313669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:35:30.313669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:35:30.313669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:35:30.313669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:35:30.313669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:35:30.313669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:30.313669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:30.314009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:30.314009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:35:30.314009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:30.314009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:30.314009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:30.314009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:35:30.314009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:30.314009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:35:30.314009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:35:30.314009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:30.314009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:35:30.314009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:30.314009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:35:30.314009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:35:30.314395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:30.314395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:35:30.314395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:30.314395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:35:30.315120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:30.315120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:35:30.315120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:35:30.315120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:35:30.315120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:35:30.315120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:35:30.315120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:30.315120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:35:30.315120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:35:30.315120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:35:30.315120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:35:30.315120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:35:30.315120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:30.315120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:30.315120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:35:30.315120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:35:30.315120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:35:30.315657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:35:30.315657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:35:30.315657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:35:30.315657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:30.315657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:30.315657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:30.315657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:35:30.315657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:30.315657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:30.315657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:30.315657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:35:30.315657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:30.315657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:35:30.315657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:35:30.315657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:30.315657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:35:30.315657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:30.315657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:35:30.315657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:35:30.316247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:30.316247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:35:30.316247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:30.316247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:35:30.317909 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:35:30.318212 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:35:30.318362 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:35:30.318600 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:35:30.319029 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:35:30.319419 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:35:30.319733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:30.320484 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:35:30.320484 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:30.320484 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:35:30.320484 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:35:30.320484 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:30.320484 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:35:30.320484 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:30.320484 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:35:30.321250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:30.321250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:35:30.321250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:30.321250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:35:30.321250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:35:30.321250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:35:30.321250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:35:30.321250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:30.321250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:30.321250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:30.321250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:35:30.321250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:30.321724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:30.321724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:30.321724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:35:30.321724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:30.321724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:35:30.321724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:35:30.321724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:30.321724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:35:30.321724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:30.321724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:35:30.321724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:35:30.321724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:30.321724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:35:30.322083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:30.322083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:35:30.322170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:30.322170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:35:30.322170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:35:30.322170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:35:30.322170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:35:30.322170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:35:30.322170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:30.322170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:35:30.322170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:35:30.322170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:35:30.322170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:35:30.322170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:35:30.322170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:30.322170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:30.322170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:35:30.322170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:35:30.322640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:35:30.322640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:35:30.322640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:30.322640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:30.322640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:30.322640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:35:30.322640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:30.322640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:30.322640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:30.322640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:35:30.322640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:30.322640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:35:30.322640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:35:30.322640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:30.322640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:35:30.322640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:30.322640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:35:30.322640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:35:30.322640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:30.323155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:35:30.323155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:30.323155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:35:35.057759 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:35:35.058509 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:35:35.058509 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:35:35.059040 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:35:35.093922 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:35:35.094351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:35.095532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:35:35.095532 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:35.095532 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:35:35.095532 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:35:35.095532 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:35.095532 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:35:35.095532 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:35.095532 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:35:35.096298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:35.096298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:35:35.096298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:35.096298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:35:35.096298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:35:35.096298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:35:35.096298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:35:35.096298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:35.096298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:35.096298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:35.096298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:35:35.096298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:35.096906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:35.096906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:35.096906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:35:35.096906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:35.096906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:35:35.096906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:35:35.096906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:35.096906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:35:35.096906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:35.096906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:35:35.096906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:35:35.096906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:35.096906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:35:35.097559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:35.097559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:35:35.097559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:35.097559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:35:35.097559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:35:35.097559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:35:35.097559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:35:35.097559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:35:35.097559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:35.097559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:35:35.097559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:35:35.097559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:35:35.097559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:35:35.097559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:35:35.097559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:35.098226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:35.098226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:35:35.098226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:35:35.098226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:35:35.098226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:35:35.098226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:35.098226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:35.098226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:35.098226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:35:35.098226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:35.098226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:35.098226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:35.098226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:35:35.098226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:35.098226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:35:35.098226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:35:35.098226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:35.098226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:35:35.098226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:35.099657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:35:35.099657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:35:35.099657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:35.099657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:35:35.099657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:35.099657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:35:35.296935 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:35:35.297552 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:35:35.297734 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:35:35.299501 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:35:35.299501 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:35:35.299501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:35.300430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:35:35.300430 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:35.300430 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:35:35.300430 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:35:35.300430 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:35.300430 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:35:35.300430 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:35.300430 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:35:35.301947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:35.301947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:35:35.301947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:35.301947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:35:35.301947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:35:35.301947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:35:35.301947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:35:35.301947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:35.301947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:35.301947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:35.301947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:35:35.301947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:35.302603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:35.302603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:35.302603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:35:35.302603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:35.302603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:35:35.302603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:35:35.302603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:35.302603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:35:35.302603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:35.302603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:35:35.302603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:35:35.302603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:35.302603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:35:35.303182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:35.303182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:35:35.303570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:35.303570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:35:35.303570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:35:35.303570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:35:35.303570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:35:35.303570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:35:35.303570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:35.303570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:35:35.303570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:35:35.303570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:35:35.303570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:35:35.303570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:35:35.303570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:35.303570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:35.303570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:35:35.303570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:35:35.304163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:35:35.304163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:35:35.304163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:35.304163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:35.304163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:35.304163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:35:35.304163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:35.304163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:35.304163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:35.304163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:35:35.304163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:35.304163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:35:35.304163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:35:35.304163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:35.304163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:35:35.304163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:35.304163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:35:35.304163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:35:35.304163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:35.304688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:35:35.304688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:35.304688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:35:35.304688 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:35:35.304688 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:35:35.304688 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:35:35.304688 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:35:35.305136 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:35:35.305516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:35.306153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:35:35.306153 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:35.306153 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:35:35.306153 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:35:35.306153 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:35.306153 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:35:35.306153 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:35.306153 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:35:35.306875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:35.306875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:35:35.306875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:35:35.306875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:35:35.306875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:35.306875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:35:35.306875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:35.306875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:35:35.306875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:35.306875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:35:35.306875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:35.306875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:35:35.306875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:35.306875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:35:35.306875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:35:35.307267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:35.307267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:35:35.307267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:35.307267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:35:35.307267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:35:35.307267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:35.307267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:35:35.307267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:35.307267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:35:35.307662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:35.307662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:35:35.307662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:35.307662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:35:35.307662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:35:35.307662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:35:35.307662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:35:35.307662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:35:35.307662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:35:35.307662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:35:35.307662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:35.307662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:35.308037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:35.308037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:35:35.308037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:35.308037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:35.308037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:35.308037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:35:35.308037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:35.308037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:35:35.308037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:35:35.308037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:35.308037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:35:35.308037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:35.308037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:35:35.308037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:35:35.308595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:35.308595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:35:35.308595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:35.308595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:35:35.308732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:35.308732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:35:35.308732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:35:35.308732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:35:35.308732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:35:35.308732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:35:35.308732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:35.308732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:35:35.308732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:35:35.308732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:35:35.308732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:35:35.308732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:35:35.308732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:35.308732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:35.308732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:35:35.308732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:35:35.309227 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:35:35.309227 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:35:35.309227 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:35:35.309227 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:35:35.309227 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:35:35.309227 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:35.309227 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:35.309227 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:35.309227 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:35:35.309227 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:35.309227 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:35.309227 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:35.309227 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:35:35.309227 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:35.309227 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:35:35.309227 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:35:35.309227 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:35.309227 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:35:35.310631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:35.310631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:35:35.310631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:35:35.310631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:35.310631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:35:35.310631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:35.310631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:35:35.310886 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:35:35.311233 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:35:35.311233 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:35:35.311392 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:35:35.311770 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:35:35.312141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:35.312840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:35:35.312840 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:35.312840 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:35:35.312840 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:35:35.312840 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:35.312840 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:35:35.312840 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:35.312840 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:35:35.313581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:35.313581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:35:35.313581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:35:35.313581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:35:35.313581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:35.313581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:35:35.313581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:35.313581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:35:35.313581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:35.313581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:35:35.313581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:35.313581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:35:35.313581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:35.313581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:35:35.313581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:35:35.314019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:35.314019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:35:35.314019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:35.314019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:35:35.314019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:35:35.314019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:35.314019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:35:35.314019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:35.314019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:35:35.314429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:35.314429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:35:35.314429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:35.314429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:35:35.314429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:35:35.314429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:35:35.314429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:35:35.314429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:35:35.314429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:35:35.314429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:35.314429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:35.314742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:35.314742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:35:35.314742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:35.314742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:35.314742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:35.314742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:35:35.314742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:35.314742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:35:35.314742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:35:35.314742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:35.314742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:35:35.314742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:35.314742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:35:35.314742 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:35:35.315126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:35.315126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:35:35.315126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:35.315126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:35:35.315471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:35.315471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:35:35.315471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:35:35.315471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:35:35.315471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:35:35.315471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:35:35.315471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:35.315471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:35:35.315471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:35:35.315471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:35:35.315471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:35:35.315471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:35:35.315471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:35.315471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:35.315471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:35:35.315471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:35:35.315471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:35:35.315938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:35:35.315938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:35:35.315938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:35:35.315938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:35.315938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:35.315938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:35.315938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:35:35.315938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:35.315938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:35.315938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:35.315938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:35:35.315938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:35.315938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:35:35.315938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:35:35.315938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:35.315938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:35:35.315938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:35.315938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:35:35.315938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:35:35.316500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:35.316500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:35:35.316500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:35.316500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:35:35.318221 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:35:35.318475 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:35:35.318602 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:35:35.318732 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:35:35.319148 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:35:35.319568 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:35:35.319775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:35.320428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:35:35.320428 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:35.320428 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:35:35.320428 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:35:35.320428 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:35.320428 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:35:35.320428 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:35.320428 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:35:35.321251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:35.321251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:35:35.321251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:35.321251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:35:35.321251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:35:35.321251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:35:35.321251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:35:35.321251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:35.321251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:35.321251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:35.321251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:35:35.321251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:35.321608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:35.321608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:35.321608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:35:35.321608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:35.321608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:35:35.321608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:35:35.321608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:35.321608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:35:35.321608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:35.321608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:35:35.321608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:35:35.321608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:35.321608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:35:35.321963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:35.321963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:35:35.322174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:35.322174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:35:35.322174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:35:35.322174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:35:35.322174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:35:35.322174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:35:35.322174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:35.322174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:35:35.322174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:35:35.322174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:35:35.322174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:35:35.322174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:35:35.322174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:35.322174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:35.322174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:35:35.322174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:35:35.322627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:35:35.322627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:35:35.322627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:35.322627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:35.322627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:35.322627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:35:35.322627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:35.322627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:35.322627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:35.322627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:35:35.322627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:35.322627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:35:35.322627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:35:35.322627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:35.322627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:35:35.322627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:35.322627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:35:35.322627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:35:35.322627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:35.323105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:35:35.323105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:35.323105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:35:40.058531 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:35:40.059281 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:35:40.059281 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:35:40.060609 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:35:40.096146 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:35:40.096780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:40.097616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:35:40.097616 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:40.097616 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:35:40.097616 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:35:40.097616 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:40.097616 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:35:40.097616 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:40.097616 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:35:40.098653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:40.098653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:35:40.098653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:40.098653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:35:40.098653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:35:40.098653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:35:40.098653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:35:40.098653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:40.098653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:40.098653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:40.098653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:35:40.098653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:40.099368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:40.099368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:40.099368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:35:40.099368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:40.099368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:35:40.099368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:35:40.099368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:40.099368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:35:40.099368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:40.099368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:35:40.099368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:35:40.099368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:40.099368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:35:40.100082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:40.100082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:35:40.100082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:40.100082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:35:40.100082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:35:40.100082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:35:40.100082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:35:40.100082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:35:40.100082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:40.100082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:35:40.100082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:35:40.100082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:35:40.100082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:35:40.100082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:35:40.100082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:40.101008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:40.101008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:35:40.101008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:35:40.101008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:35:40.101008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:35:40.101008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:40.101008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:40.101008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:40.101008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:35:40.101008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:40.101008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:40.101008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:40.101008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:35:40.101008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:40.101008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:35:40.101008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:35:40.101008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:40.101008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:35:40.101008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:40.102212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:35:40.102212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:35:40.102212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:40.102212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:35:40.102212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:40.102212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:35:40.297602 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:35:40.298009 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:35:40.298190 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:35:40.299293 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:35:40.299677 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:35:40.299980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:40.300907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:35:40.300907 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:40.300907 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:35:40.300907 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:35:40.300907 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:40.300907 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:35:40.300907 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:40.300907 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:35:40.302247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:40.302247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:35:40.302247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:40.302247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:35:40.302247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:35:40.302247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:35:40.302247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:35:40.302247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:40.302247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:40.302247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:40.302247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:35:40.302247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:40.302887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:40.302887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:40.302887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:35:40.302887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:40.302887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:35:40.302887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:35:40.302887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:40.302887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:35:40.302887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:40.302887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:35:40.302887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:35:40.302887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:40.302887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:35:40.303477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:40.303477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:35:40.303946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:40.303946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:35:40.303946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:35:40.303946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:35:40.303946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:35:40.303946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:35:40.303946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:40.303946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:35:40.303946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:35:40.303946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:35:40.303946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:35:40.303946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:35:40.303946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:40.303946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:40.303946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:35:40.303946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:35:40.304564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:35:40.304564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:35:40.304564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:40.304564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:40.304564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:40.304564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:35:40.304564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:40.304564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:40.304564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:40.304564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:35:40.304564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:40.304564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:35:40.304564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:35:40.304564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:40.304564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:35:40.304564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:40.304564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:35:40.304564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:35:40.304564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:40.305100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:35:40.305100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:40.305100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:35:40.305100 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:35:40.305100 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:35:40.305100 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:35:40.305100 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:35:40.305538 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:35:40.305780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:40.306358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:35:40.306358 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:40.306358 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:35:40.306358 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:35:40.306358 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:40.306358 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:35:40.306358 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:40.306358 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:35:40.307396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:40.307396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:35:40.307396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:35:40.307396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:35:40.307396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:40.307396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:35:40.307396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:40.307396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:35:40.307396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:40.307396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:35:40.307396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:40.307396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:35:40.307396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:40.307396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:35:40.307396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:35:40.308019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:40.308019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:35:40.308019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:40.308019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:35:40.308019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:35:40.308019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:40.308019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:35:40.308019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:40.308019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:35:40.308307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:40.308307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:35:40.308307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:40.308307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:35:40.308307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:35:40.308307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:35:40.308307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:35:40.308307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:35:40.308307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:35:40.308307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:35:40.308307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:40.308307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:40.308747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:40.308747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:35:40.308747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:40.308747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:40.308747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:40.308747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:35:40.308747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:40.308747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:35:40.308747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:35:40.308747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:40.308747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:35:40.308747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:40.308747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:35:40.308747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:35:40.309150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:40.309150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:35:40.309150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:40.309150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:35:40.309271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:40.309271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:35:40.309271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:35:40.309271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:35:40.309271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:35:40.309271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:35:40.309271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:40.309271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:35:40.309271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:35:40.309271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:35:40.309271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:35:40.309271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:35:40.309271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:40.309271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:40.309271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:35:40.309271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:35:40.309787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:35:40.309787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:35:40.309787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:35:40.309787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:35:40.309787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:35:40.309787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:40.309787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:40.309787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:40.309787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:35:40.309787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:40.309787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:40.309787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:40.309787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:35:40.309787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:40.309787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:35:40.309787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:35:40.309787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:40.309787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:35:40.310318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:40.310318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:35:40.310318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:35:40.310318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:40.310318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:35:40.310318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:40.310318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:35:40.311259 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:35:40.311535 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:35:40.311643 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:35:40.311801 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:35:40.312115 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:35:40.312330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:40.312810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:35:40.312810 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:40.312810 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:35:40.312810 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:35:40.312810 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:40.312810 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:35:40.312810 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:40.312810 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:35:40.313525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:40.313525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:35:40.313525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:35:40.313525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:35:40.313525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:40.313525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:35:40.313525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:40.313525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:35:40.313525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:40.313525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:35:40.313525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:40.313525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:35:40.313525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:40.313525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:35:40.313525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:35:40.313951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:40.313951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:35:40.313951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:40.313951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:35:40.313951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:35:40.313951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:40.313951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:35:40.313951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:40.313951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:35:40.314340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:40.314340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:35:40.314340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:40.314340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:35:40.314340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:35:40.314340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:35:40.314340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:35:40.314340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:35:40.314340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:35:40.314340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:40.314340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:40.314689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:40.314689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:35:40.314689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:40.314689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:40.314689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:40.314689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:35:40.314689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:40.314689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:35:40.314689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:35:40.314689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:40.314689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:35:40.314689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:40.314689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:35:40.314689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:35:40.315082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:40.315082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:35:40.315082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:40.315082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:35:40.315346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:40.315346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:35:40.315346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:35:40.315346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:35:40.315346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:35:40.315346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:35:40.315346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:40.315346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:35:40.315346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:35:40.315346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:35:40.315346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:35:40.315346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:35:40.315346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:40.315346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:40.315346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:35:40.315346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:35:40.315346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:35:40.315900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:35:40.315900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:35:40.315900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:35:40.315900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:40.315900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:40.315900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:40.315900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:35:40.315900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:40.315900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:40.315900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:40.315900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:35:40.315900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:40.315900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:35:40.315900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:35:40.315900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:40.315900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:35:40.315900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:40.315900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:35:40.315900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:35:40.316879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:40.316879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:35:40.316879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:40.316879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:35:40.319080 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:35:40.319080 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:35:40.319080 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:35:40.319239 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:35:40.319861 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:35:40.320193 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:35:40.320431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:40.321117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:35:40.321117 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:40.321117 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:35:40.321117 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:35:40.321117 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:40.321117 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:35:40.321117 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:40.321117 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:35:40.321912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:40.321912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:35:40.321912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:40.321912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:35:40.321912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:35:40.321912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:35:40.321912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:35:40.321912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:40.321912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:40.321912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:40.321912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:35:40.321912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:40.322516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:40.322516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:40.322516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:35:40.322516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:40.322516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:35:40.322516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:35:40.322516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:40.322516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:35:40.322516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:40.322516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:35:40.322516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:35:40.322516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:40.322516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:35:40.322897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:40.322897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:35:40.322897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:40.322897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:35:40.322897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:35:40.322897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:35:40.322897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:35:40.322897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:35:40.322897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:40.322897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:35:40.322897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:35:40.322897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:35:40.322897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:35:40.322897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:35:40.322897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:40.323519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:40.323519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:35:40.323519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:35:40.323519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:35:40.323519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:35:40.323519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:40.323519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:40.323519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:40.323519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:35:40.323519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:40.323519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:40.323519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:40.323519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:35:40.323519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:40.323519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:35:40.323519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:35:40.323519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:40.323519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:35:40.323519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:40.324099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:35:40.324099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:35:40.324099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:40.324099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:35:40.324099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:40.324099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:35:45.059161 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:35:45.060226 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:35:45.060226 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:35:45.061025 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:35:45.097508 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:35:45.097807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:45.098852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:35:45.098852 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:45.098852 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:35:45.098852 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:35:45.098852 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:45.098852 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:35:45.098852 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:45.098852 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:35:45.099855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:45.099855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:35:45.099855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:45.099855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:35:45.099855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:35:45.099855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:35:45.099855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:35:45.099855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:45.099855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:45.099855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:45.099855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:35:45.099855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:45.100622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:45.100622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:45.100622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:35:45.100622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:45.100622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:35:45.100622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:35:45.100622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:45.100622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:35:45.100622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:45.100622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:35:45.100622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:35:45.100622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:45.100622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:35:45.101325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:45.101325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:35:45.101325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:45.101325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:35:45.101325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:35:45.101325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:35:45.101325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:35:45.101325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:35:45.101325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:45.101325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:35:45.101325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:35:45.101325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:35:45.101325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:35:45.101325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:35:45.101325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:45.102207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:45.102207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:35:45.102207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:35:45.102207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:35:45.102207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:35:45.102207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:45.102207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:45.102207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:45.102207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:35:45.102207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:45.102207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:45.102207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:45.102207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:35:45.102207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:45.102207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:35:45.102207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:35:45.102207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:45.102207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:35:45.102207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:45.103072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:35:45.103072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:35:45.103072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:45.103072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:35:45.103072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:45.103072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:35:45.298122 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:35:45.298548 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:35:45.298734 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:35:45.299842 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:35:45.300171 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:35:45.300435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:45.301282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:35:45.301282 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:45.301282 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:35:45.301282 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:35:45.301282 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:45.301282 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:35:45.301282 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:45.301282 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:35:45.302926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:45.302926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:35:45.302926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:45.302926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:35:45.302926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:35:45.302926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:35:45.302926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:35:45.302926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:45.302926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:45.302926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:45.302926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:35:45.302926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:45.303539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:45.303539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:45.303539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:35:45.303539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:45.303539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:35:45.303539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:35:45.303539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:45.303539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:35:45.303539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:45.303539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:35:45.303539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:35:45.303539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:45.303539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:35:45.304153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:45.304153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:35:45.304509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:45.304509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:35:45.304509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:35:45.304509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:35:45.304509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:35:45.304509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:35:45.304509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:45.304509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:35:45.304509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:35:45.304509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:35:45.304509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:35:45.304509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:35:45.304509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:45.304509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:45.304509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:35:45.304509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:35:45.305210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:35:45.305210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:35:45.305210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:45.305210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:45.305210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:45.305210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:35:45.305210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:45.305210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:45.305210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:45.305210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:35:45.305210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:45.305210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:35:45.305210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:35:45.305210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:45.305210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:35:45.305210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:45.305210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:35:45.305210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:35:45.305210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:45.306075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:35:45.306075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:45.306075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:35:45.306075 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:35:45.306075 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:35:45.306075 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:35:45.306075 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:35:45.306075 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:35:45.306558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:45.307054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:35:45.307054 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:45.307054 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:35:45.307054 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:35:45.307054 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:45.307054 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:35:45.307054 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:45.307054 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:35:45.308245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:45.308245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:35:45.308245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:35:45.308245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:35:45.308245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:45.308245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:35:45.308245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:45.308245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:35:45.308245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:45.308245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:35:45.308245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:45.308245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:35:45.308245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:45.308245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:35:45.308245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:35:45.308813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:45.308813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:35:45.308813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:45.308813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:35:45.308813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:35:45.308813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:45.308813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:35:45.308813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:45.308813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:35:45.309538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:45.309538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:35:45.309538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:45.309538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:35:45.309538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:35:45.309538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:35:45.309538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:35:45.309538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:35:45.309538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:35:45.309538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:35:45.309538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:45.309538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:45.309871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:45.309871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:35:45.309871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:45.309871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:45.309871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:45.309871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:35:45.309871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:45.309871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:35:45.309871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:35:45.309871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:45.309871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:35:45.309871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:45.309871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:35:45.309871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:35:45.310347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:45.310347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:35:45.310347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:45.310347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:35:45.311155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:45.311155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:35:45.311155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:35:45.311155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:35:45.311155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:35:45.311155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:35:45.311155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:45.311155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:35:45.311155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:35:45.311155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:35:45.311155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:35:45.311155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:35:45.311155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:45.311155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:45.311155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:35:45.311155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:35:45.311634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:35:45.311634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:35:45.311634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:35:45.311634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:35:45.311634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:35:45.311634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:45.311634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:45.311634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:45.311634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:35:45.311634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:45.311634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:45.311634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:45.311634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:35:45.311634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:45.311634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:35:45.311634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:35:45.311634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:45.311634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:35:45.312079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:45.312079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:35:45.312079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:35:45.312079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:45.312079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:35:45.312079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:45.312079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:35:45.312079 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:35:45.312079 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:35:45.312079 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:35:45.312350 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:35:45.312350 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:35:45.312558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:45.313025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:35:45.313025 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:45.313025 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:35:45.313025 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:35:45.313025 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:45.313025 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:35:45.313025 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:45.313025 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:35:45.313774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:45.313774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:35:45.313774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:35:45.313774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:35:45.313774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:45.313774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:35:45.313774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:45.313774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:35:45.313774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:45.313774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:35:45.313774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:45.313774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:35:45.313774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:45.313774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:35:45.313774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:35:45.314162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:45.314162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:35:45.314162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:45.314162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:35:45.314162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:35:45.314162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:45.314162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:35:45.314162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:45.314162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:35:45.314654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:45.314654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:35:45.314654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:45.314654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:35:45.314654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:35:45.314654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:35:45.314654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:35:45.314654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:35:45.314654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:35:45.314654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:45.314654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:45.314945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:45.314945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:35:45.314945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:45.314945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:45.314945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:45.314945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:35:45.314945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:45.314945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:35:45.314945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:35:45.314945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:45.314945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:35:45.314945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:45.314945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:35:45.314945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:35:45.315318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:45.315318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:35:45.315318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:45.315318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:35:45.315715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:45.315715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:35:45.315715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:35:45.315715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:35:45.315715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:35:45.315715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:35:45.315715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:45.315715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:35:45.315715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:35:45.315715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:35:45.315715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:35:45.315715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:35:45.315715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:45.315715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:45.315715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:35:45.315715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:35:45.315715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:35:45.316179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:35:45.316179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:35:45.316179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:35:45.316179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:45.316179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:45.316179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:45.316179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:35:45.316179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:45.316179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:45.316179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:45.316179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:35:45.316179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:45.316179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:35:45.316179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:35:45.316179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:45.316179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:35:45.316179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:45.316179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:35:45.316179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:35:45.316687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:45.316687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:35:45.316687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:45.316687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:35:45.318953 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:35:45.319178 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:35:45.319275 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:35:45.319426 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:35:45.319874 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:35:45.320200 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:35:45.320423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:45.321131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:35:45.321131 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:45.321131 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:35:45.321131 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:35:45.321131 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:45.321131 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:35:45.321131 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:45.321131 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:35:45.321907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:45.321907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:35:45.321907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:45.321907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:35:45.321907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:35:45.321907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:35:45.321907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:35:45.321907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:45.321907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:45.321907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:45.321907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:35:45.321907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:45.322233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:45.322233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:45.322233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:35:45.322233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:45.322233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:35:45.322233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:35:45.322233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:45.322233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:35:45.322233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:45.322233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:35:45.322233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:35:45.322233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:45.322233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:35:45.322637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:45.322637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:35:45.322937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:45.322937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:35:45.322937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:35:45.322937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:35:45.322937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:35:45.322937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:35:45.322937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:45.322937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:35:45.322937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:35:45.322937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:35:45.322937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:35:45.322937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:35:45.322937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:45.322937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:45.322937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:35:45.322937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:35:45.323337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:35:45.323337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:35:45.323337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:45.323337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:45.323337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:45.323337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:35:45.323337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:45.323337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:45.323337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:45.323337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:35:45.323337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:45.323337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:35:45.323337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:35:45.323337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:45.323337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:35:45.323337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:45.323337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:35:45.323337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:35:45.323337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:45.323827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:35:45.323827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:45.323827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:35:50.059884 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:35:50.060939 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:35:50.060939 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:35:50.061771 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:35:50.095815 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:35:50.096037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:50.096782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:35:50.096782 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:50.096782 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:35:50.096782 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:35:50.096782 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:50.096782 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:35:50.096782 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:50.096782 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:35:50.097790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:50.097790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:35:50.097790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:50.097790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:35:50.097790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:35:50.097790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:35:50.097790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:35:50.097790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:50.097790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:50.097790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:50.097790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:35:50.097790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:50.098375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:50.098375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:50.098375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:35:50.098375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:50.098375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:35:50.098375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:35:50.098375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:50.098375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:35:50.098375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:50.098375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:35:50.098375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:35:50.098375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:50.098375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:35:50.099094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:50.099094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:35:50.099855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:50.099855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:35:50.099855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:35:50.099855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:35:50.099855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:35:50.099855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:35:50.099855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:50.099855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:35:50.099855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:35:50.099855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:35:50.099855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:35:50.099855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:35:50.099855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:50.099855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:50.099855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:35:50.099855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:35:50.100648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:35:50.100648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:35:50.100648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:50.100648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:50.100648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:50.100648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:35:50.100648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:50.100648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:50.100648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:50.100648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:35:50.100648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:50.100648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:35:50.100648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:35:50.100648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:50.100648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:35:50.100648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:50.100648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:35:50.100648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:35:50.100648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:50.101560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:35:50.101560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:50.101560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:35:50.298846 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:35:50.299533 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:35:50.299735 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:35:50.300807 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:35:50.301180 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:35:50.301462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:50.302317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:35:50.302317 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:50.302317 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:35:50.302317 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:35:50.302317 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:50.302317 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:35:50.302317 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:50.302317 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:35:50.304014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:50.304014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:35:50.304014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:50.304014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:35:50.304014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:35:50.304014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:35:50.304014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:35:50.304014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:50.304014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:50.304014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:50.304014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:35:50.304014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:50.304390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:50.304390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:50.304390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:35:50.304390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:50.304390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:35:50.304390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:35:50.304390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:50.304390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:35:50.304390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:50.304390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:35:50.304390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:35:50.304390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:50.304390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:35:50.304756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:50.304756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:35:50.305939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:50.305939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:35:50.305939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:35:50.305939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:35:50.305939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:35:50.305939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:35:50.305939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:50.305939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:35:50.305939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:35:50.305939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:35:50.305939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:35:50.305939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:35:50.305939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:50.305939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:50.305939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:35:50.305939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:35:50.306381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:35:50.306381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:35:50.306381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:50.306381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:50.306381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:50.306381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:35:50.306381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:50.306381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:50.306381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:50.306381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:35:50.306381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:50.306381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:35:50.306381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:35:50.306381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:50.306381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:35:50.306381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:50.306381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:35:50.306381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:35:50.306381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:50.306908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:35:50.306908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:50.306908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:35:50.306908 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:35:50.306908 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:35:50.306908 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:35:50.306908 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:35:50.307125 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:35:50.307236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:50.307732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:35:50.307732 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:50.307732 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:35:50.307732 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:35:50.307732 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:50.307732 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:35:50.307732 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:50.307732 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:35:50.308464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:50.308464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:35:50.308464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:35:50.308464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:35:50.308464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:50.308464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:35:50.308464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:50.308464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:35:50.308464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:50.308464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:35:50.308464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:50.308464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:35:50.308464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:50.308464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:35:50.308464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:35:50.308901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:50.308901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:35:50.308901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:50.308901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:35:50.308901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:35:50.308901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:50.308901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:35:50.308901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:50.308901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:35:50.309292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:50.309292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:35:50.309292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:50.309292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:35:50.309292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:35:50.309292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:35:50.309292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:35:50.309292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:35:50.309292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:35:50.309292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:35:50.309292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:50.309292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:50.309666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:50.309666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:35:50.309666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:50.309666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:50.309666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:50.309666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:35:50.309666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:50.309666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:35:50.309666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:35:50.309666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:50.309666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:35:50.309666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:50.309666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:35:50.309666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:35:50.310041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:50.310041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:35:50.310041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:50.310041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:35:50.310307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:50.310307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:35:50.310307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:35:50.310307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:35:50.310307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:35:50.310307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:35:50.310307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:50.310307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:35:50.310307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:35:50.310307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:35:50.310307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:35:50.310307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:35:50.310307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:50.310307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:50.310307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:35:50.310307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:35:50.310759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:35:50.310759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:35:50.310759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:35:50.310759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:35:50.310759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:35:50.310759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:50.310759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:50.310759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:50.310759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:35:50.310759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:50.310759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:50.310759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:50.310759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:35:50.310759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:50.310759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:35:50.310759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:35:50.310759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:50.310759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:35:50.311302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:50.311302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:35:50.311302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:35:50.311302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:50.311302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:35:50.311302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:50.311302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:35:50.311954 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:35:50.312222 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:35:50.312330 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:35:50.312486 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:35:50.312781 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:35:50.313030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:50.313550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:35:50.313550 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:50.313550 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:35:50.313550 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:35:50.313550 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:50.313550 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:35:50.313550 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:50.313550 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:35:50.314259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:50.314259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:35:50.314259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:35:50.314259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:35:50.314259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:50.314259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:35:50.314259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:50.314259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:35:50.314259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:50.314259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:35:50.314259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:50.314259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:35:50.314259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:50.314259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:35:50.314259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:35:50.314685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:50.314685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:35:50.314685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:50.314685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:35:50.314685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:35:50.314685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:50.314685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:35:50.314685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:50.314685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:35:50.315100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:50.315100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:35:50.315100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:50.315100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:35:50.315100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:35:50.315100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:35:50.315100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:35:50.315100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:35:50.315100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:35:50.315100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:50.315100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:50.315459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:50.315459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:35:50.315459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:50.315459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:50.315459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:50.315459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:35:50.315459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:50.315459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:35:50.315459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:35:50.315459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:50.315459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:35:50.315459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:50.315459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:35:50.315459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:35:50.315849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:50.315849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:35:50.315849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:50.315849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:35:50.316148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:50.316148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:35:50.316148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:35:50.316148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:35:50.316148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:35:50.316148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:35:50.316148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:50.316148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:35:50.316148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:35:50.316148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:35:50.316148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:35:50.316148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:35:50.316148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:50.316148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:50.316148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:35:50.316148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:35:50.316148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:35:50.316710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:35:50.316710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:35:50.316710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:35:50.316710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:50.316710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:50.316710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:50.316710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:35:50.316710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:50.316710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:50.316710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:50.316710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:35:50.316710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:50.316710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:35:50.316710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:35:50.316710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:50.316710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:35:50.316710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:50.316710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:35:50.316710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:35:50.317238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:50.317238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:35:50.317238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:50.317238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:35:50.319247 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:35:50.319513 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:35:50.319622 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:35:50.319767 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:35:50.320161 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:35:50.320548 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:35:50.320771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:50.321527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:35:50.321527 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:50.321527 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:35:50.321527 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:35:50.321527 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:50.321527 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:35:50.321527 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:50.321527 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:35:50.322317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:50.322317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:35:50.322317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:50.322317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:35:50.322317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:35:50.322317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:35:50.322317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:35:50.322317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:50.322317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:50.322317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:50.322317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:35:50.322317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:50.322770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:50.322770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:50.322770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:35:50.322770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:50.322770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:35:50.322770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:35:50.322770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:50.322770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:35:50.322770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:50.322770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:35:50.322770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:35:50.322770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:50.322770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:35:50.323119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:50.323119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:35:50.323299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:50.323299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:35:50.323299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:35:50.323299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:35:50.323299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:35:50.323299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:35:50.323299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:50.323299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:35:50.323299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:35:50.323299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:35:50.323299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:35:50.323299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:35:50.323299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:50.323299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:50.323299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:35:50.323299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:35:50.323776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:35:50.323776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:35:50.323776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:50.323776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:50.323776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:50.323776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:35:50.323776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:50.323776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:50.323776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:50.323776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:35:50.323776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:50.323776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:35:50.323776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:35:50.323776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:50.323776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:35:50.323776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:50.323776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:35:50.323776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:35:50.323776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:50.324392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:35:50.324392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:50.324392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:35:55.060611 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:35:55.061448 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:35:55.061519 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:35:55.062819 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:35:55.097975 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:35:55.098319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:55.099058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:35:55.099058 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:55.099058 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:35:55.099058 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:35:55.099058 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:55.099058 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:35:55.099058 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:55.099058 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:35:55.099868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:55.099868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:35:55.099868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:55.099868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:35:55.099868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:35:55.099868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:35:55.099868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:35:55.099868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:55.099868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:55.099868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:55.099868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:35:55.099868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:55.100477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:55.100477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:55.100477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:35:55.100477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:55.100477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:35:55.100477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:35:55.100477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:55.100477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:35:55.100477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:55.100477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:35:55.100477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:35:55.100477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:55.100477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:35:55.101023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:55.101023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:35:55.101023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:55.101023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:35:55.101023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:35:55.101023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:35:55.101023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:35:55.101023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:35:55.101023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:55.101023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:35:55.101023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:35:55.101023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:35:55.101023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:35:55.101023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:35:55.101023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:55.101733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:55.101733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:35:55.101733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:35:55.101733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:35:55.101733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:35:55.101733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:55.101733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:55.101733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:55.101733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:35:55.101733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:55.101733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:55.101733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:55.101733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:35:55.101733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:55.101733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:35:55.101733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:35:55.101733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:55.101733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:35:55.101733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:55.102568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:35:55.102568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:35:55.102568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:55.102568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:35:55.102568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:55.102568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:35:55.299565 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:35:55.300497 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:35:55.300735 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:35:55.301738 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:35:55.302011 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:35:55.302263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:55.303304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:35:55.303304 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:55.303304 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:35:55.303304 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:35:55.303304 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:55.303304 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:35:55.303304 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:55.303304 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:35:55.304926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:55.304926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:35:55.304926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:55.304926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:35:55.304926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:35:55.304926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:35:55.304926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:35:55.304926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:55.304926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:55.304926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:55.304926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:35:55.304926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:55.305495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:55.305495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:55.305495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:35:55.305495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:55.305495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:35:55.305495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:35:55.305495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:55.305495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:35:55.305495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:55.305495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:35:55.305495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:35:55.305495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:55.305495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:35:55.306011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:55.306011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:35:55.306945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:55.306945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:35:55.306945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:35:55.306945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:35:55.306945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:35:55.306945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:35:55.306945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:55.306945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:35:55.306945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:35:55.306945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:35:55.306945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:35:55.306945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:35:55.306945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:55.306945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:55.306945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:35:55.306945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:35:55.307668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:35:55.307668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:35:55.307668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:55.307668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:55.307668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:55.307668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:35:55.307668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:55.307668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:55.307668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:55.307668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:35:55.307668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:55.307668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:35:55.307668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:35:55.307668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:55.307668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:35:55.307668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:55.307668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:35:55.307668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:35:55.307668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:55.308465 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:35:55.308465 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:55.308465 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:35:55.308465 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:35:55.308465 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:35:55.308465 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:35:55.308465 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:35:55.308922 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:35:55.309203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:55.309872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:35:55.309872 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:55.309872 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:35:55.309872 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:35:55.309872 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:55.309872 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:35:55.309872 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:55.309872 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:35:55.310714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:55.310714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:35:55.310714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:35:55.310714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:35:55.310714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:55.310714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:35:55.310714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:55.310714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:35:55.310714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:55.310714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:35:55.310714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:55.310714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:35:55.310714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:55.310714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:35:55.310714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:35:55.311446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:55.311446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:35:55.311446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:55.311446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:35:55.311446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:35:55.311446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:55.311446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:35:55.311446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:55.311446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:35:55.311931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:55.311931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:35:55.311931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:55.311931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:35:55.311931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:35:55.311931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:35:55.311931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:35:55.311931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:35:55.311931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:35:55.311931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:35:55.311931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:55.311931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:55.312523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:55.312523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:35:55.312523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:55.312523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:55.312523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:55.312523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:35:55.312523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:55.312523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:35:55.312523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:35:55.312523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:55.312523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:35:55.312523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:55.312523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:35:55.312523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:35:55.313161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:55.313161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:35:55.313161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:55.313161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:35:55.313853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:55.313853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:35:55.313853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:35:55.313853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:35:55.313853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:35:55.313853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:35:55.313853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:55.313853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:35:55.313853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:35:55.313853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:35:55.313853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:35:55.313853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:35:55.313853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:55.313853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:55.313853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:35:55.313853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:35:55.314644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:35:55.314644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:35:55.314644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:35:55.314644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:35:55.314644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:35:55.314644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:55.314644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:55.314644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:55.314644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:35:55.314644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:55.314644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:55.314644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:55.314644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:35:55.314644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:55.314644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:35:55.314644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:35:55.314644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:55.314644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:35:55.315471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:55.315471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:35:55.315471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:35:55.315471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:55.315471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:35:55.315471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:55.315471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:35:55.315471 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:35:55.315471 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:35:55.315471 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:35:55.316096 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:35:55.316096 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:35:55.316096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:55.316096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:35:55.316096 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:55.316096 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:35:55.316096 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:35:55.316096 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:55.316096 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:35:55.316096 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:55.316096 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:35:55.317062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:55.317062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:35:55.317062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:35:55.317062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:35:55.317062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:55.317062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:35:55.317062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:55.317062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:35:55.317062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:55.317062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:35:55.317062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:55.317062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:35:55.317062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:55.317062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:35:55.317062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:35:55.317464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:55.317464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:35:55.317464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:55.317464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:35:55.317464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:35:55.317464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:55.317464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:35:55.317464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:55.317464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:35:55.318034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:55.318034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:35:55.318034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:55.318034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:35:55.318034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:35:55.318034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:35:55.318034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:35:55.318034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:35:55.318034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:35:55.318034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:55.318034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:55.318341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:55.318341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:35:55.318341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:55.318341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:55.318341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:55.318341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:35:55.318341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:55.318341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:35:55.318341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:35:55.318341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:55.318341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:35:55.318341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:55.318341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:35:55.318341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:35:55.318709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:55.318709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:35:55.318709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:55.318709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:35:55.319321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:55.319321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:35:55.319321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:35:55.319321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:35:55.319321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:35:55.319321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:35:55.319321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:55.319321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:35:55.319321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:35:55.319321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:35:55.319321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:35:55.319321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:35:55.319321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:55.319321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:55.319321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:35:55.319321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:35:55.319321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:35:55.319798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:35:55.319798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:35:55.319798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:35:55.319798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:55.319798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:55.319798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:55.319798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:35:55.319798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:55.319798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:55.319798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:55.319798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:35:55.319798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:55.319798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:35:55.319798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:35:55.319798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:55.319798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:35:55.319798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:55.319798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:35:55.319798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:35:55.320274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:55.320274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:35:55.320274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:55.320274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:35:55.320274 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:35:55.320274 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:35:55.320274 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:35:55.320483 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:35:55.320818 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:35:55.321160 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:35:55.321373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:55.322058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:35:55.322058 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:55.322058 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:35:55.322058 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:35:55.322058 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:55.322058 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:35:55.322058 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:55.322058 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:35:55.322940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:55.322940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:35:55.322940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:55.322940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:35:55.322940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:35:55.322940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:35:55.322940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:35:55.322940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:55.322940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:55.322940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:55.322940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:35:55.322940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:55.323259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:35:55.323259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:55.323259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:35:55.323259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:55.323259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:35:55.323259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:35:55.323259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:55.323259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:35:55.323259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:55.323259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:35:55.323259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:35:55.323259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:55.323259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:35:55.323687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:55.323687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:35:55.324374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:55.324374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:35:55.324374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:35:55.324374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:35:55.324374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:35:55.324374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:35:55.324374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:55.324374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:35:55.324374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:35:55.324374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:35:55.324374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:35:55.324374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:35:55.324374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:35:55.324374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:35:55.324374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:35:55.324374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:35:55.324832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:35:55.324832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:35:55.324832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:55.324832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:55.324832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:35:55.324832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:35:55.324832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:35:55.324832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:35:55.324832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:35:55.324832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:35:55.324832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:35:55.324832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:35:55.324832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:35:55.324832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:35:55.324832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:35:55.324832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:35:55.324832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:35:55.324832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:35:55.324832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:35:55.325315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:35:55.325315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:35:55.325315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:36:00.061388 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:36:00.062248 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:36:00.062248 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:36:00.063591 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:36:00.103359 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:36:00.103594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:00.104096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:36:00.104096 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:00.104096 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:36:00.104096 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:36:00.104096 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:00.104096 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:36:00.104096 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:00.104096 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:36:00.108884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:00.108884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:36:00.108884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:00.108884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:36:00.108884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:36:00.108884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:36:00.108884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:36:00.108884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:00.108884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:00.108884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:00.108884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:36:00.108884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:00.109549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:00.109549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:00.109549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:36:00.109549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:00.109549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:36:00.109549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:36:00.109549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:00.109549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:36:00.109549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:00.109549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:36:00.109549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:36:00.109549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:00.109549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:36:00.110278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:00.110278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:36:00.110843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:00.110843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:36:00.110843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:36:00.110843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:36:00.110843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:36:00.110843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:36:00.110843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:00.110843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:36:00.110843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:36:00.110843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:36:00.110843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:36:00.110843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:36:00.110843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:00.110843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:00.110843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:36:00.110843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:36:00.111972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:36:00.111972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:36:00.111972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:00.111972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:00.111972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:00.111972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:36:00.111972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:00.111972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:00.111972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:00.111972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:36:00.111972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:00.111972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:36:00.111972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:36:00.111972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:00.111972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:36:00.111972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:00.111972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:36:00.111972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:36:00.111972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:00.112899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:36:00.112899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:00.112899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:36:00.300619 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:36:00.301294 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:36:00.301642 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:36:00.302892 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:36:00.303256 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:36:00.303541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:00.304556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:36:00.304556 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:00.304556 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:36:00.304556 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:36:00.304556 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:00.304556 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:36:00.304556 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:00.304556 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:36:00.306109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:00.306109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:36:00.306109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:00.306109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:36:00.306109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:36:00.306109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:36:00.306109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:36:00.306109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:00.306109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:00.306109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:00.306109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:36:00.306109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:00.306824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:00.306824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:00.306824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:36:00.306824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:00.306824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:36:00.306824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:36:00.306824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:00.306824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:36:00.306824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:00.306824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:36:00.306824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:36:00.306824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:00.306824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:36:00.307488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:00.307488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:36:00.307946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:00.307946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:36:00.307946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:36:00.307946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:36:00.307946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:36:00.307946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:36:00.307946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:00.307946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:36:00.307946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:36:00.307946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:36:00.307946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:36:00.307946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:36:00.307946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:00.307946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:00.307946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:36:00.307946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:36:00.308771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:36:00.308771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:36:00.308771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:00.308771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:00.308771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:00.308771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:36:00.308771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:00.308771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:00.308771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:00.308771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:36:00.308771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:00.308771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:36:00.308771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:36:00.308771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:00.308771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:36:00.308771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:00.308771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:36:00.308771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:36:00.308771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:00.309787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:36:00.309787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:00.309787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:36:00.309787 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:36:00.309787 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:36:00.309787 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:36:00.309787 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:36:00.309787 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:36:00.310314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:00.310812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:36:00.310812 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:00.310812 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:36:00.310812 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:36:00.310812 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:00.310812 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:36:00.310812 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:00.310812 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:36:00.312051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:00.312051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:36:00.312051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:36:00.312051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:36:00.312051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:00.312051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:36:00.312051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:00.312051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:36:00.312051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:00.312051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:36:00.312051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:00.312051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:36:00.312051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:00.312051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:36:00.312051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:36:00.312550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:00.312550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:36:00.312550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:00.312550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:36:00.312550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:36:00.312550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:00.312550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:36:00.312550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:00.312550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:36:00.313349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:00.313349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:36:00.313349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:00.313349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:36:00.313349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:36:00.313349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:36:00.313349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:36:00.313349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:36:00.313349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:36:00.313349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:36:00.313349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:00.313349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:00.313749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:00.313749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:36:00.313749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:00.313749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:00.313749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:00.313749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:36:00.313749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:00.313749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:36:00.313749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:36:00.313749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:00.313749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:36:00.313749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:00.313749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:36:00.313749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:36:00.314103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:00.314103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:36:00.314103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:00.314103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:36:00.315255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:00.315255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:36:00.315255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:36:00.315255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:36:00.315255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:36:00.315255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:36:00.315255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:00.315255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:36:00.315255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:36:00.315255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:36:00.315255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:36:00.315255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:36:00.315255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:00.315255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:00.315255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:36:00.315255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:36:00.315714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:36:00.315714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:36:00.315714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:36:00.315714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:36:00.315714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:36:00.315714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:00.315714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:00.315714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:00.315714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:36:00.315714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:00.315714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:00.315714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:00.315714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:36:00.315714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:00.315714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:36:00.315714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:36:00.315714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:00.315714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:36:00.316198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:00.316198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:36:00.316198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:36:00.316198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:00.316198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:36:00.316198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:00.316198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:36:00.316198 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:36:00.316198 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:36:00.316198 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:36:00.316515 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:36:00.316515 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:36:00.316716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:00.317213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:36:00.317213 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:00.317213 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:36:00.317213 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:36:00.317213 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:00.317213 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:36:00.317213 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:00.317213 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:36:00.317970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:00.317970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:36:00.317970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:36:00.317970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:36:00.317970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:00.317970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:36:00.317970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:00.317970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:36:00.317970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:00.317970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:36:00.317970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:00.317970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:36:00.317970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:00.317970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:36:00.317970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:36:00.318453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:00.318453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:36:00.318453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:00.318453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:36:00.318453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:36:00.318453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:00.318453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:36:00.318453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:00.318453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:36:00.318876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:00.318876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:36:00.318876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:00.318876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:36:00.318876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:36:00.318876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:36:00.318876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:36:00.318876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:36:00.318876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:36:00.318876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:00.318876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:00.319251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:00.319251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:36:00.319251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:00.319251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:00.319251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:00.319251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:36:00.319251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:00.319251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:36:00.319251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:36:00.319251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:00.319251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:36:00.319251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:00.319251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:36:00.319251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:36:00.319630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:00.319630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:36:00.319630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:00.319630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:36:00.320184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:00.320184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:36:00.320184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:36:00.320184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:36:00.320184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:36:00.320184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:36:00.320184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:00.320184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:36:00.320184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:36:00.320184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:36:00.320184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:36:00.320184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:36:00.320184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:00.320184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:00.320184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:36:00.320184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:36:00.320184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:36:00.320674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:36:00.320674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:36:00.320674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:36:00.320674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:00.320674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:00.320674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:00.320674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:36:00.320674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:00.320674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:00.320674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:00.320674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:36:00.320674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:00.320674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:36:00.320674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:36:00.320674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:00.320674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:36:00.320674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:00.320674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:36:00.320674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:36:00.321172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:00.321172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:36:00.321172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:00.321172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:36:00.321172 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:36:00.321172 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:36:00.321172 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:36:00.321172 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:36:00.321634 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:36:00.322035 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:36:00.322314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:00.323083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:36:00.323083 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:00.323083 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:36:00.323083 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:36:00.323083 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:00.323083 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:36:00.323083 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:00.323083 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:36:00.323994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:00.323994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:36:00.323994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:00.323994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:36:00.323994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:36:00.323994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:36:00.323994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:36:00.323994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:00.323994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:00.323994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:00.323994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:36:00.323994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:00.324349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:00.324349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:00.324349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:36:00.324349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:00.324349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:36:00.324349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:36:00.324349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:00.324349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:36:00.324349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:00.324349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:36:00.324349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:36:00.324349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:00.324349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:36:00.324699 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:00.324699 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:36:00.325235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:00.325235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:36:00.325235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:36:00.325235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:36:00.325235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:36:00.325235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:36:00.325235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:00.325235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:36:00.325235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:36:00.325235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:36:00.325235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:36:00.325235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:36:00.325235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:00.325235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:00.325235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:36:00.325235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:36:00.325682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:36:00.325682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:36:00.325682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:00.325682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:00.325682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:00.325682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:36:00.325682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:00.325682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:00.325682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:00.325682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:36:00.325682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:00.325682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:36:00.325682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:36:00.325682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:00.325682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:36:00.325682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:00.325682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:36:00.325682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:36:00.325682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:00.326198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:36:00.326198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:00.326198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:36:05.062105 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:36:05.064612 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:36:05.064612 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:36:05.065433 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:36:05.100785 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:36:05.101174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:05.101972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:36:05.101972 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:05.101972 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:36:05.101972 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:36:05.101972 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:05.101972 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:36:05.101972 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:05.101972 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:36:05.103162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:05.103162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:36:05.103162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:05.103162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:36:05.103162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:36:05.103162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:36:05.103162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:36:05.103162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:05.103162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:05.103162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:05.103162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:36:05.103162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:05.103787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:05.103787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:05.103787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:36:05.103787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:05.103787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:36:05.103787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:36:05.103787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:05.103787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:36:05.103787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:05.103787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:36:05.103787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:36:05.103787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:05.103787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:36:05.104467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:05.104467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:36:05.104467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:05.104467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:36:05.104467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:36:05.104467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:36:05.104467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:36:05.104467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:36:05.104467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:05.104467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:36:05.104467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:36:05.104467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:36:05.104467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:36:05.104467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:36:05.104467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:05.105238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:05.105238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:36:05.105238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:36:05.105238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:36:05.105238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:36:05.105238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:05.105238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:05.105238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:05.105238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:36:05.105238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:05.105238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:05.105238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:05.105238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:36:05.105238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:05.105238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:36:05.105238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:36:05.105238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:05.105238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:36:05.105238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:05.106177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:36:05.106177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:36:05.106177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:05.106177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:36:05.106177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:05.106177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:36:05.301335 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:36:05.301906 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:36:05.302105 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:36:05.302976 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:36:05.303210 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:36:05.303362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:05.304052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:36:05.304052 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:05.304052 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:36:05.304052 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:36:05.304052 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:05.304052 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:36:05.304052 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:05.304052 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:36:05.305568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:05.305568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:36:05.305568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:05.305568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:36:05.305568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:36:05.305568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:36:05.305568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:36:05.305568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:05.305568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:05.305568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:05.305568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:36:05.305568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:05.306785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:05.306785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:05.306785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:36:05.306785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:05.306785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:36:05.306785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:36:05.306785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:05.306785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:36:05.306785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:05.306785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:36:05.306785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:36:05.306785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:05.306785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:36:05.307849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:05.307849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:36:05.307849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:05.307849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:36:05.307849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:36:05.307849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:36:05.307849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:36:05.307849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:36:05.307849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:05.307849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:36:05.307849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:36:05.307849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:36:05.307849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:36:05.307849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:36:05.307849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:05.309168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:05.309168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:36:05.309168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:36:05.309168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:36:05.309168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:36:05.309168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:05.309168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:05.309168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:05.309168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:36:05.309168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:05.309168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:05.309168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:05.309168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:36:05.309168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:05.309168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:36:05.309168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:36:05.309168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:05.309168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:36:05.309168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:05.310763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:36:05.310763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:36:05.310763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:05.310763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:36:05.310763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:05.310763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:36:05.310763 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:36:05.310763 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:36:05.310763 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:36:05.310763 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:36:05.311375 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:36:05.311375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:05.311375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:36:05.311375 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:05.311375 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:36:05.311375 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:36:05.311375 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:05.311375 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:36:05.311375 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:05.311375 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:36:05.312090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:05.312090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:36:05.312090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:36:05.312090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:36:05.312090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:05.312090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:36:05.312090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:05.312090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:36:05.312090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:05.312090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:36:05.312090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:05.312090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:36:05.312090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:05.312090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:36:05.312090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:36:05.312793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:05.312793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:36:05.312793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:05.312793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:36:05.312793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:36:05.312793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:05.312793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:36:05.312793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:05.312793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:36:05.313226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:05.313226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:36:05.313226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:05.313226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:36:05.313226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:36:05.313226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:36:05.313226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:36:05.313226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:36:05.313226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:36:05.313226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:36:05.313226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:05.313226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:05.313760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:05.313760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:36:05.313760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:05.313760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:05.313760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:05.313760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:36:05.313760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:05.313760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:36:05.313760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:36:05.313760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:05.313760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:36:05.313760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:05.313760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:36:05.313760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:36:05.314191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:05.314191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:36:05.314191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:05.314191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:36:05.314191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:05.314191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:36:05.314191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:36:05.314191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:36:05.314191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:36:05.314191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:36:05.314191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:05.314191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:36:05.314191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:36:05.314191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:36:05.314600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:36:05.314600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:36:05.314600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:05.314600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:05.314600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:36:05.314600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:36:05.314600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:36:05.314600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:36:05.314600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:36:05.314600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:36:05.314600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:36:05.314600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:05.314600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:05.314600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:05.314600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:36:05.314600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:05.314600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:05.314600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:05.314600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:36:05.315853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:05.315853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:36:05.315853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:36:05.315853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:05.315853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:36:05.315853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:05.315853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:36:05.315853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:36:05.315853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:05.315853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:36:05.315853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:05.315853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:36:05.315853 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:36:05.316603 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:36:05.316603 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:36:05.316603 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:36:05.316881 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:36:05.317241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:05.317916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:36:05.317916 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:05.317916 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:36:05.317916 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:36:05.317916 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:05.317916 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:36:05.317916 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:05.317916 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:36:05.318640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:05.318640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:36:05.318640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:36:05.318640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:36:05.318640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:05.318640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:36:05.318640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:05.318640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:36:05.318640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:05.318640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:36:05.318640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:05.318640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:36:05.318640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:05.318640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:36:05.318640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:36:05.319462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:05.319462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:36:05.319462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:05.319462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:36:05.319462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:36:05.319462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:05.319462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:36:05.319462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:05.319462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:36:05.319462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:05.319462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:36:05.319462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:05.319462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:36:05.319462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:36:05.320167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:36:05.320167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:36:05.320167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:36:05.320167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:36:05.320167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:05.320167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:05.320167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:05.320167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:36:05.320167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:05.320167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:05.320167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:05.320167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:36:05.320167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:05.321005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:36:05.321005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:36:05.321005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:05.321005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:36:05.321005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:05.321005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:36:05.321005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:36:05.321005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:05.321005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:36:05.321005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:05.321005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:36:05.321005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:05.321638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:36:05.321638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:36:05.321638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:36:05.321638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:36:05.321638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:36:05.321638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:05.321638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:36:05.321638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:36:05.321638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:36:05.321638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:36:05.321638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:36:05.321638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:05.321638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:05.321638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:36:05.321638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:36:05.321638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:36:05.321638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:36:05.321638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:36:05.321638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:36:05.322456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:05.322456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:05.322456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:05.322456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:36:05.322456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:05.322456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:05.322456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:05.322456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:36:05.322456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:05.322456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:36:05.322456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:36:05.322456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:05.322456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:36:05.322456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:05.322456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:36:05.322456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:36:05.322456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:05.322456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:36:05.322456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:05.322456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:36:05.323168 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:36:05.323168 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:36:05.323168 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:36:05.323168 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:36:05.323168 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:36:05.323168 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:36:05.323340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:05.323340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:36:05.323340 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:05.323340 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:36:05.323340 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:36:05.323340 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:05.323340 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:36:05.323340 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:05.323340 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:36:05.323827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:05.323827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:36:05.323827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:05.323827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:36:05.323827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:36:05.323827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:36:05.323827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:36:05.323827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:05.323827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:05.323827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:05.323827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:36:05.323827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:05.324055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:05.324055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:05.324055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:36:05.324055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:05.324055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:36:05.324055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:36:05.324055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:05.324055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:36:05.324055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:05.324055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:36:05.324055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:36:05.324055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:05.324055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:36:05.324281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:05.324281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:36:05.324758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:05.324758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:36:05.324758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:36:05.324758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:36:05.324758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:36:05.324758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:36:05.324758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:05.324758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:36:05.324758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:36:05.324758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:36:05.324758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:36:05.324758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:36:05.324758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:05.324758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:05.324758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:36:05.324758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:36:05.325064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:36:05.325064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:36:05.325064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:05.325064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:05.325064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:05.325064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:36:05.325064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:05.325064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:05.325064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:05.325064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:36:05.325064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:05.325064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:36:05.325064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:36:05.325064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:05.325064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:36:05.325064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:05.325064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:36:05.325064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:36:05.325064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:05.325435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:36:05.325435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:05.325435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:36:10.063858 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:36:10.064440 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:36:10.064654 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:36:10.065929 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:36:10.108062 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:36:10.108513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:10.109512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:36:10.109512 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:10.109512 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:36:10.109512 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:36:10.109512 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:10.109512 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:36:10.109512 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:10.109512 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:36:10.110675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.045 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:10.110675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:36:10.110675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:10.110675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:36:10.110675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:36:10.110675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:36:10.110675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:36:10.110675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:10.110675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:10.110675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:10.110675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:36:10.110675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:10.111227 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:10.111227 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:10.111227 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:36:10.111227 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:10.111227 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:36:10.111227 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:36:10.111227 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:10.111227 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:36:10.111227 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:10.111227 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:36:10.111227 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:36:10.111227 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:10.111227 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:36:10.111801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:10.111801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:36:10.112373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:10.112373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:36:10.112373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:36:10.112373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:36:10.112373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:36:10.112373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:36:10.112373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:10.112373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:36:10.112373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:36:10.112373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:36:10.112373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:36:10.112373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:36:10.112373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:10.112373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:10.112373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:36:10.112373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:36:10.112863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:36:10.112863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:36:10.112863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:10.112863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:10.112863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:10.112863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:36:10.112863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:10.112863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:10.112863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:10.112863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:36:10.112863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:10.112863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:36:10.112863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:36:10.112863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:10.112863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:36:10.112863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:10.112863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:36:10.112863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:36:10.112863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:10.113374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:36:10.113374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:10.113374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:36:10.302018 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:36:10.302668 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:36:10.302859 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:36:10.304074 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:36:10.304447 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:36:10.304682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:10.305578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:36:10.305578 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:10.305578 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:36:10.305578 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:36:10.305578 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:10.305578 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:36:10.305578 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:10.305578 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:36:10.307381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:10.307381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:36:10.307381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:10.307381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:36:10.307381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:36:10.307381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:36:10.307381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:36:10.307381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:10.307381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:10.307381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:10.307381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:36:10.307381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:10.308386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:10.308386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:10.308386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:36:10.308386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:10.308386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:36:10.308386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:36:10.308386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:10.308386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:36:10.308386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:10.308386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:36:10.308386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:36:10.308386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:10.308386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:36:10.308964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:10.308964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:36:10.309100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:10.309100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:36:10.309100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:36:10.309100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:36:10.309100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:36:10.309100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:36:10.309100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:10.309100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:36:10.309100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:36:10.309100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:36:10.309100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:36:10.309100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:36:10.309100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:10.309100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:10.309100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:36:10.309100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:36:10.309835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:36:10.309835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:36:10.309835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:10.309835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:10.309835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:10.309835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:36:10.309835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:10.309835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:10.309835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:10.309835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:36:10.309835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:10.309835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:36:10.309835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:36:10.309835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:10.309835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:36:10.309835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:10.309835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:36:10.309835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:36:10.309835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:10.310666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:36:10.310666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:10.310666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:36:10.310666 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:36:10.310666 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:36:10.310666 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:36:10.310666 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:36:10.310666 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:36:10.311070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:10.311440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:36:10.311440 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:10.311440 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:36:10.311440 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:36:10.311440 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:10.311440 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:36:10.311440 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:10.311440 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:36:10.312457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:10.312457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:36:10.312457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:36:10.312457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:36:10.312457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:10.312457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:36:10.312457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:10.312457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:36:10.312457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:10.312457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:36:10.312457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:10.312457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:36:10.312457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:10.312457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:36:10.312457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:36:10.313151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:10.313151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:36:10.313151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:10.313151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:36:10.313151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:36:10.313151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:10.313151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:36:10.313151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:10.313151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:36:10.313397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:10.313397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:36:10.313397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:10.313397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:36:10.313397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:36:10.313397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:36:10.313397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:36:10.313397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:36:10.313397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:36:10.313397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:36:10.313397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:10.313397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:10.313734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:10.313734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:36:10.313734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:10.313734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:10.313734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:10.313734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:36:10.313734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:10.313734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:36:10.313734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:36:10.313734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:10.313734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:36:10.313734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:10.313734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:36:10.313734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:36:10.314077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:10.314077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:36:10.314077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:10.314077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:36:10.314184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:10.314184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:36:10.314184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:36:10.314184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:36:10.314184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:36:10.314184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:36:10.314184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:10.314184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:36:10.314184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:36:10.314184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:36:10.314184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:36:10.314184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:36:10.314184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:10.314184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:10.314184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:36:10.314184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:36:10.314592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:36:10.314592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:36:10.314592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:36:10.314592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:36:10.314592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:36:10.314592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:10.314592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:10.314592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:10.314592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:36:10.314592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:10.314592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:10.314592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:10.314592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:36:10.314592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:10.314592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:36:10.314592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:36:10.314592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:10.314592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:36:10.315036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:10.315036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:36:10.315036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:36:10.315036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:10.315036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:36:10.315036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:10.315036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:36:10.316430 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:36:10.316645 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:36:10.316790 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:36:10.316948 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:36:10.317484 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:36:10.317747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:10.318440 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:36:10.318440 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:10.318440 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:36:10.318440 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:36:10.318440 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:10.318440 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:36:10.318440 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:10.318440 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:36:10.319575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:10.319575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:36:10.319575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:36:10.319575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:36:10.319575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:10.319575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:36:10.319575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:10.319575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:36:10.319575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:10.319575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:36:10.319575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:10.319575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:36:10.319575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:10.319575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:36:10.319575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:36:10.319998 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:10.319998 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:36:10.319998 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:10.319998 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:36:10.319998 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:36:10.319998 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:10.319998 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:36:10.319998 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:10.319998 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:36:10.320942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:10.320942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:36:10.320942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:10.320942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:36:10.320942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:36:10.320942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:36:10.320942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:36:10.320942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:36:10.320942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:36:10.320942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:10.320942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:10.321302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:10.321302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:36:10.321302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:10.321302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:10.321302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:10.321302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:36:10.321302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:10.321302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:36:10.321302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:36:10.321302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:10.321302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:36:10.321302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:10.321302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:36:10.321302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:36:10.321727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:10.321727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:36:10.321727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:10.321727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:36:10.322645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:10.322645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:36:10.322645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:36:10.322645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:36:10.322645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:36:10.322645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:36:10.322645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:10.322645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:36:10.322645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:36:10.322645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:36:10.322645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:36:10.322645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:36:10.322645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:10.322645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:10.322645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:36:10.322645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:36:10.322645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:36:10.323149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:36:10.323149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:36:10.323149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:36:10.323149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:10.323149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:10.323149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:10.323149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:36:10.323149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:10.323149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:10.323149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:10.323149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:36:10.323149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:10.323149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:36:10.323149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:36:10.323149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:10.323149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:36:10.323149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:10.323149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:36:10.323149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:36:10.323745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:10.323745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:36:10.323745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:10.323745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:36:10.323745 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:36:10.323745 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:36:10.323745 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:36:10.323745 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:36:10.324172 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:36:10.324532 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:36:10.324746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:10.325454 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:36:10.325454 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:10.325454 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:36:10.325454 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:36:10.325454 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:10.325454 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:36:10.325454 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:10.325454 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:36:10.326570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:10.326570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:36:10.326570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:10.326570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:36:10.326570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:36:10.326570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:36:10.326570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:36:10.326570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:10.326570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:10.326570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:10.326570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:36:10.326570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:10.326908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:10.326908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:10.326908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:36:10.326908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:10.326908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:36:10.326908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:36:10.326908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:10.326908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:36:10.326908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:10.326908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:36:10.326908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:36:10.326908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:10.326908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:36:10.327300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:10.327300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:36:10.327743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:10.327743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:36:10.327743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:36:10.327743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:36:10.327743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:36:10.327743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:36:10.327743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:10.327743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:36:10.327743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:36:10.327743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:36:10.327743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:36:10.327743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:36:10.327743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:10.327743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:10.327743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:36:10.327743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:36:10.328185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:36:10.328185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:36:10.328185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:10.328185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:10.328185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:10.328185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:36:10.328185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:10.328185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:10.328185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:10.328185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:36:10.328185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:10.328185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:36:10.328185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:36:10.328185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:10.328185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:36:10.328185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:10.328185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:36:10.328185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:36:10.328185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:10.328765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:36:10.328765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:10.328765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:36:15.064499 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:36:15.065270 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:36:15.065378 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:36:15.066546 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:36:15.100826 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:36:15.101280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:15.102149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:36:15.102149 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:15.102149 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:36:15.102149 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:36:15.102149 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:15.102149 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:36:15.102149 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:15.102149 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:36:15.102909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:15.102909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:36:15.102909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:15.102909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:36:15.102909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:36:15.102909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:36:15.102909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:36:15.102909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:15.102909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:15.102909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:15.102909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:36:15.102909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:15.103594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:15.103594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:15.103594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:36:15.103594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:15.103594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:36:15.103594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:36:15.103594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:15.103594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:36:15.103594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:15.103594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:36:15.103594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:36:15.103594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:15.103594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:36:15.104256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:15.104256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:36:15.104256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:15.104256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:36:15.104256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:36:15.104256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:36:15.104256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:36:15.104256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:36:15.104256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:15.104256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:36:15.104256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:36:15.104256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:36:15.104256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:36:15.104256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:36:15.104256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:15.105053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:15.105053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:36:15.105053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:36:15.105053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:36:15.105053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:36:15.105053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:15.105053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:15.105053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:15.105053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:36:15.105053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:15.105053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:15.105053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:15.105053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:36:15.105053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:15.105053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:36:15.105053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:36:15.105053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:15.105053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:36:15.105053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:15.106034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:36:15.106034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:36:15.106034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:15.106034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:36:15.106034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:15.106034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:36:15.302587 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:36:15.302948 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:36:15.303130 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:36:15.304286 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:36:15.304671 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:36:15.304909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:15.305905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:36:15.305905 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:15.305905 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:36:15.305905 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:36:15.305905 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:15.305905 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:36:15.305905 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:15.305905 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:36:15.307497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:15.307497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:36:15.307497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:15.307497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:36:15.307497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:36:15.307497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:36:15.307497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:36:15.307497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:15.307497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:15.307497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:15.307497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:36:15.307497 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:15.308366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:15.308366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:15.308366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:36:15.308366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:15.308366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:36:15.308366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:36:15.308366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:15.308366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:36:15.308366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:15.308366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:36:15.308366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:36:15.308366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:15.308366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:36:15.309084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:15.309084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:36:15.309084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:15.309084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:36:15.309084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:36:15.309084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:36:15.309084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:36:15.309084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:36:15.309084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:15.309084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:36:15.309084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:36:15.309084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:36:15.309084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:36:15.309084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:36:15.309084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:15.309808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:15.309808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:36:15.309808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:36:15.309808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:36:15.309808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:36:15.309808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:15.309808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:15.309808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:15.309808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:36:15.309808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:15.309808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:15.309808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:15.309808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:36:15.309808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:15.309808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:36:15.309808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:36:15.309808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:15.309808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:36:15.309808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:15.311666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:36:15.311666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:36:15.311666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:15.311666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:36:15.311666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:15.311666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:36:15.311666 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:36:15.311666 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:36:15.311666 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:36:15.311666 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:36:15.312127 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:36:15.312127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:15.312127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:36:15.312127 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:15.312127 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:36:15.312127 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:36:15.312127 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:15.312127 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:36:15.312127 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:15.312127 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:36:15.312584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:15.312584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:36:15.312584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:36:15.312584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:36:15.312584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:15.312584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:36:15.312584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:15.312584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:36:15.312584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:15.312584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:36:15.312584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:15.312584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:36:15.312584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:15.312584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:36:15.312584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:36:15.313235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:15.313235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:36:15.313235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:15.313235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:36:15.313235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:36:15.313235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:15.313235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:36:15.313235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:15.313235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:36:15.313235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:15.313235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:36:15.313235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:15.313235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:36:15.313235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:36:15.314044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:36:15.314044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:36:15.314044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:36:15.314044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:36:15.314044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:36:15.314044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:15.314044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:15.314044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:15.314044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:36:15.314044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:15.314044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:15.314044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:15.314044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:36:15.315865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:15.315865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:36:15.315865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:36:15.315865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:15.315865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:36:15.315865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:15.315865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:36:15.315865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:36:15.315865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:15.315865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:36:15.315865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:15.315865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:36:15.317056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:15.317056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:36:15.317056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:36:15.317056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:36:15.317056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:36:15.317056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:36:15.317056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:15.317056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:36:15.317056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:36:15.317056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:36:15.317056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:36:15.317056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:36:15.317056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:15.317056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:15.317056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:36:15.317056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:36:15.320301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:36:15.320301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:36:15.320301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:36:15.320301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:36:15.320301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:36:15.320301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:15.320301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:15.320301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:15.320301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:36:15.320301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:15.320301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:15.320301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:15.320301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:36:15.320301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:15.320301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:36:15.320301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:36:15.320301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:15.320301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:36:15.320659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:15.320659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:36:15.320659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:36:15.320659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:15.320659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:36:15.320659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:15.320659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:36:15.320659 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:36:15.320659 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:36:15.320659 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:36:15.321189 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:36:15.321189 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:36:15.321189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:15.321189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:36:15.321189 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:15.321189 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:36:15.321189 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:36:15.321189 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:15.321189 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:36:15.321189 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:15.321189 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:36:15.321189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:15.321837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:36:15.321837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:36:15.321837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:36:15.321837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:15.321837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:36:15.321837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:15.321837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:36:15.321837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:15.321837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:36:15.321837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:15.321837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:36:15.321837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:15.321837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:36:15.321837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:36:15.321837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:15.321837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:36:15.321837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:15.322602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:36:15.322602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:36:15.322602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:15.322602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:36:15.322602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:15.322602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:36:15.322602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:15.322602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:36:15.322602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:15.322602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:36:15.322602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:36:15.322602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:36:15.322602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:36:15.323143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:36:15.323143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:36:15.323143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:15.323143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:15.323143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:15.323143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:36:15.323143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:15.323143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:15.323143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:15.323143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:36:15.323143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:15.323143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:36:15.323143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:36:15.323546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:15.323546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:36:15.323546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:15.323546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:36:15.323546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:36:15.323546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:15.323546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:36:15.323546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:15.323546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:36:15.323546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:15.323546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:36:15.323546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:36:15.323991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:36:15.323991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:36:15.323991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:36:15.323991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:15.323991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:36:15.323991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:36:15.323991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:36:15.323991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:36:15.323991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:36:15.323991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:15.323991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:15.323991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:36:15.323991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:36:15.323991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:36:15.323991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:36:15.323991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:36:15.323991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:36:15.323991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:15.323991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:15.324697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:15.324697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:36:15.324697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:15.324697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:15.324697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:15.324697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:36:15.324697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:15.324697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:36:15.324697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:36:15.324697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:15.324697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:36:15.324697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:15.324697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:36:15.324697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:36:15.324697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:15.324697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:36:15.324697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:15.324697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:36:15.325059 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:36:15.325059 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:36:15.325059 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:36:15.325059 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:36:15.325059 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:36:15.325059 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:36:15.325217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:15.325241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:36:15.325241 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:15.325241 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:36:15.325241 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:36:15.325241 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:15.325241 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:36:15.325241 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:15.325241 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:36:15.325872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:15.325872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:36:15.325872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:15.325872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:36:15.325872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:36:15.325872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:36:15.325872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:36:15.325872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:15.325872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:15.325872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:15.325872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:36:15.325872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:15.326113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:15.326113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:15.326113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:36:15.326113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:15.326113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:36:15.326113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:36:15.326113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:15.326113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:36:15.326113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:15.326113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:36:15.326113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:36:15.326113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:15.326113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:36:15.326344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:15.326344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:36:15.326559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:15.326559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:36:15.326559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:36:15.326559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:36:15.326559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:36:15.326559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:36:15.326559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:15.326559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:36:15.326559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:36:15.326559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:36:15.326559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:36:15.326559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:36:15.326559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:15.326559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:15.326559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:36:15.326559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:36:15.326860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:36:15.326860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:36:15.326860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:15.326860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:15.326860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:15.326860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:36:15.326860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:15.326860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:15.326860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:15.326860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:36:15.326860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:15.326860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:36:15.326860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:36:15.326860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:15.326860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:36:15.326860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:15.326860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:36:15.326860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:36:15.326860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:15.327595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:36:15.327595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:15.327595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:36:20.065131 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:36:20.065841 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:36:20.065841 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:36:20.066928 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:36:20.102724 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:36:20.103112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:20.103937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:36:20.103937 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:20.103937 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:36:20.103937 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:36:20.103937 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:20.103937 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:36:20.103937 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:20.103937 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:36:20.104691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:20.104691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:36:20.104691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:20.104691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:36:20.104691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:36:20.104691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:36:20.104691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:36:20.104691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:20.104691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:20.104691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:20.104691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:36:20.104691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:20.105260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:20.105260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:20.105260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:36:20.105260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:20.105260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:36:20.105260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:36:20.105260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:20.105260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:36:20.105260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:20.105260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:36:20.105260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:36:20.105260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:20.105260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:36:20.105798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:20.105798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:36:20.105798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:20.105798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:36:20.105798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:36:20.105798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:36:20.105798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:36:20.105798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:36:20.105798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:20.105798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:36:20.105798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:36:20.105798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:36:20.105798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:36:20.105798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:36:20.105798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:20.106472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:20.106472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:36:20.106472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:36:20.106472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:36:20.106472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:36:20.106472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:20.106472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:20.106472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:20.106472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:36:20.106472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:20.106472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:20.106472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:20.106472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:36:20.106472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:20.106472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:36:20.106472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:36:20.106472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:20.106472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:36:20.106472 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:20.107282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:36:20.107282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:36:20.107282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:20.107282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:36:20.107282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:20.107282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:36:20.303309 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:36:20.303821 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:36:20.304010 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:36:20.305212 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:36:20.305597 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:36:20.305834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:20.306728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:36:20.306728 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:20.306728 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:36:20.306728 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:36:20.306728 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:20.306728 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:36:20.306728 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:20.306728 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:36:20.308543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:20.308543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:36:20.308543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:20.308543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:36:20.308543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:36:20.308543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:36:20.308543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:36:20.308543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:20.308543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:20.308543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:20.308543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:36:20.308543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:20.309276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:20.309276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:20.309276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:36:20.309276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:20.309276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:36:20.309276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:36:20.309276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:20.309276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:36:20.309276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:20.309276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:36:20.309276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:36:20.309276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:20.309276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:36:20.310066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:20.310066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:36:20.311622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:20.311622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:36:20.311622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:36:20.311622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:36:20.311622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:36:20.311622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:36:20.311622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:20.311622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:36:20.311622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:36:20.311622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:36:20.311622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:36:20.311622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:36:20.311622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:20.311622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:20.311622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:36:20.311622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:36:20.312485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:36:20.312485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:36:20.312485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:20.312485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:20.312485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:20.312485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:36:20.312485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:20.312485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:20.312485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:20.312485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:36:20.312485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:20.312485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:36:20.312485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:36:20.312485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:20.312485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:36:20.312485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:20.312485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:36:20.312485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:36:20.312485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:20.315115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:36:20.315115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:20.315115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:36:20.315115 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:36:20.315115 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:36:20.315115 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:36:20.315115 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:36:20.315115 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:36:20.315602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:20.315602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:36:20.315602 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:20.315602 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:36:20.315602 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:36:20.315602 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:20.315602 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:36:20.315602 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:20.315602 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:36:20.316050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:20.316050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:36:20.316050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:36:20.316050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:36:20.316050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:20.316050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:36:20.316050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:20.316050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:36:20.316050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:20.316050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:36:20.316050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:20.316050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:36:20.316050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:20.316050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:36:20.316050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:36:20.316797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:20.316797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:36:20.316797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:20.316797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:36:20.316797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:36:20.316797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:20.316797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:36:20.316797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:20.316797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:36:20.316797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:20.316797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:36:20.316797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:20.316797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:36:20.316797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:36:20.317600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:36:20.317600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:36:20.317600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:36:20.317600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:36:20.317600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:36:20.317600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:20.317600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:20.317600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:20.317600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:36:20.317600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:20.317600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:20.317600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:20.317600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:36:20.318263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:20.318263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:36:20.318263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:36:20.318263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:20.318263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:36:20.318263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:20.318263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:36:20.318263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:36:20.318263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:20.318263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:36:20.318263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:20.318263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:36:20.319035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:20.319035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:36:20.319035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:36:20.319035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:36:20.319035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:36:20.319035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:36:20.319035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:20.319035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:36:20.319035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:36:20.319035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:36:20.319035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:36:20.319035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:36:20.319035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:20.319035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:20.319035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:36:20.319035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:36:20.319971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:36:20.319971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:36:20.319971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:36:20.319971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:36:20.319971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:36:20.319971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:20.319971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:20.319971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:20.319971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:36:20.319971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:20.319971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:20.319971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:20.319971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:36:20.319971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:20.319971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:36:20.319971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:36:20.319971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:20.319971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:36:20.320610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:20.320610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:36:20.320610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:36:20.320610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:20.320610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:36:20.320610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:20.320610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:36:20.320610 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:36:20.320610 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:36:20.320610 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:36:20.320905 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:36:20.320905 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:36:20.320905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:20.320905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:36:20.320905 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:20.320905 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:36:20.320905 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:36:20.320905 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:20.320905 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:36:20.320905 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:20.320905 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:36:20.321176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:20.321176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:36:20.321176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:36:20.321176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:36:20.321176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:20.321176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:36:20.321176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:20.321176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:36:20.321176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:20.321176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:36:20.321176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:20.321176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:36:20.321176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:20.321176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:36:20.321176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:36:20.321562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:20.321562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:36:20.321562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:20.321562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:36:20.321562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:36:20.321562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:20.321562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:36:20.321562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:20.321562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:36:20.322025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:20.322025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:36:20.322025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:20.322025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:36:20.322025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:36:20.322025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:36:20.322025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:36:20.322025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:36:20.322025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:36:20.322025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:20.322025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:20.322239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:20.322239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:36:20.322239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:20.322239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:20.322239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:20.322239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:36:20.322239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:20.322239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:36:20.322239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:36:20.322239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:20.322239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:36:20.322239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:20.322239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:36:20.322239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:36:20.322506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:20.322506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:36:20.322506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:20.322506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:36:20.323051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:20.323051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:36:20.323051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:36:20.323051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:36:20.323051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:36:20.323051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:36:20.323051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:20.323051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:36:20.323051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:36:20.323051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:36:20.323051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:36:20.323051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:36:20.323051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:20.323051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:20.323051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:36:20.323051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:36:20.323051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:36:20.323374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:36:20.323374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:36:20.323374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:36:20.323374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:20.323374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:20.323374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:20.323374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:36:20.323374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:20.323374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:20.323374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:20.323374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:36:20.323374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:20.323374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:36:20.323374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:36:20.323374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:20.323374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:36:20.323374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:20.323374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:36:20.323374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:36:20.323762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:20.323762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:36:20.323762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:20.323762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:36:20.323762 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:36:20.323762 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:36:20.323895 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:36:20.324020 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:36:20.324538 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:36:20.324887 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:36:20.325106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:20.325917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:36:20.325917 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:20.325917 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:36:20.325917 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:36:20.325917 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:20.325917 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:36:20.325917 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:20.325917 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:36:20.326791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:20.326791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:36:20.326791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:20.326791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:36:20.326791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:36:20.326791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:36:20.326791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:36:20.326791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:20.326791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:20.326791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:20.326791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:36:20.326791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:20.327035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:20.327035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:20.327035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:36:20.327035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:20.327035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:36:20.327035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:36:20.327035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:20.327035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:36:20.327035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:20.327035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:36:20.327035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:36:20.327035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:20.327035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:36:20.327260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:20.327260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:36:20.327782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:20.327782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:36:20.327782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:36:20.327782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:36:20.327782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:36:20.327782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:36:20.327782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:20.327782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:36:20.327782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:36:20.327782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:36:20.327782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:36:20.327782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:36:20.327782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:20.327782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:20.327782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:36:20.327782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:36:20.328079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:36:20.328079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:36:20.328079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:20.328079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:20.328079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:20.328079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:36:20.328079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:20.328079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:20.328079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:20.328079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:36:20.328079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:20.328079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:36:20.328079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:36:20.328079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:20.328079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:36:20.328079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:20.328079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:36:20.328079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:36:20.328079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:20.328446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:36:20.328446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:20.328446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:36:25.065797 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:36:25.066455 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:36:25.066455 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:36:25.067878 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:36:25.069548 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG dbcounter [-] [105581] Writing DB stats neutron:SELECT=240 {{(pid=105581) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 14:36:25.101180 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:36:25.101487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:25.102151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:36:25.102151 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:25.102151 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:36:25.102151 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:36:25.102151 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:25.102151 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:36:25.102151 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:25.102151 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:36:25.103127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:25.103127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:36:25.103127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:25.103127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:36:25.103127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:36:25.103127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:36:25.103127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:36:25.103127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:25.103127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:25.103127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:25.103127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:36:25.103127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:25.103857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:25.103857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:25.103857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:36:25.103857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:25.103857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:36:25.103857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:36:25.103857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:25.103857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:36:25.103857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:25.103857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:36:25.103857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:36:25.103857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:25.103857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:36:25.104776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:25.104776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:36:25.104776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:25.104776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:36:25.104776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:36:25.104776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:36:25.104776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:36:25.104776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:36:25.104776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:25.104776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:36:25.104776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:36:25.104776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:36:25.104776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:36:25.104776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:36:25.104776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:25.105646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:25.105646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:36:25.105646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:36:25.105646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:36:25.105646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:36:25.105646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:25.105646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:25.105646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:25.105646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:36:25.105646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:25.105646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:25.105646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:25.105646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:36:25.105646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:25.105646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:36:25.105646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:36:25.105646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:25.105646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:36:25.105646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:25.106824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:36:25.106824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:36:25.106824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:25.106824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:36:25.106824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:25.106824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:36:25.303843 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:36:25.304176 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:36:25.304347 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:36:25.305421 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:36:25.305809 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:36:25.306072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:25.306986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:36:25.306986 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:25.306986 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:36:25.306986 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:36:25.306986 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:25.306986 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:36:25.306986 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:25.306986 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:36:25.308325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:25.308325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:36:25.308325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:25.308325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:36:25.308325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:36:25.308325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:36:25.308325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:36:25.308325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:25.308325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:25.308325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:25.308325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:36:25.308325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:25.309163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:25.309163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:25.309163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:36:25.309163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:25.309163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:36:25.309163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:36:25.309163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:25.309163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:36:25.309163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:25.309163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:36:25.309163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:36:25.309163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:25.309163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:36:25.309791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:25.309791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:36:25.309904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:25.309904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:36:25.309904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:36:25.309904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:36:25.309904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:36:25.309904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:36:25.309904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:25.309904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:36:25.309904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:36:25.309904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:36:25.309904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:36:25.309904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:36:25.309904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:25.309904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:25.309904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:36:25.309904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:36:25.310763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:36:25.310763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:36:25.310763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:25.310763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:25.310763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:25.310763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:36:25.310763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:25.310763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:25.310763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:25.310763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:36:25.310763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:25.310763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:36:25.310763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:36:25.310763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:25.310763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:36:25.310763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:25.310763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:36:25.310763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:36:25.310763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:25.311783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:36:25.311783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:25.311783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:36:25.312916 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:36:25.313120 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:36:25.313225 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:36:25.313337 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:36:25.313815 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:36:25.314135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:25.314818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:36:25.314818 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:25.314818 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:36:25.314818 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:36:25.314818 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:25.314818 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:36:25.314818 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:25.314818 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:36:25.315567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:25.315567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:36:25.315567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:36:25.315567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:36:25.315567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:25.315567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:36:25.315567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:25.315567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:36:25.315567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:25.315567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:36:25.315567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:25.315567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:36:25.315567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:25.315567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:36:25.315567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:36:25.316071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:25.316071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:36:25.316071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:25.316071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:36:25.316071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:36:25.316071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:25.316071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:36:25.316071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:25.316071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:36:25.316348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:25.316348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:36:25.316348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:25.316348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:36:25.316348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:36:25.316348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:36:25.316348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:36:25.316348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:36:25.316348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:36:25.316348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:36:25.316348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:25.316348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:25.316755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:25.316755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:36:25.316755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:25.316755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:25.316755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:25.316755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:36:25.316755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:25.316755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:36:25.316755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:36:25.316755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:25.316755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:36:25.316755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:25.316755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:36:25.316755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:36:25.317123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:25.317123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:36:25.317123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:25.317123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:36:25.317337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:25.317337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:36:25.317337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:36:25.317337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:36:25.317337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:36:25.317337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:36:25.317337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:25.317337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:36:25.317337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:36:25.317337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:36:25.317337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:36:25.317337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:36:25.317337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:25.317337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:25.317337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:36:25.317337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:36:25.317794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:36:25.317794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:36:25.317794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:36:25.317794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:36:25.317794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:36:25.317794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:25.317794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:25.317794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:25.317794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:36:25.317794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:25.317794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:25.317794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:25.317794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:36:25.317794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:25.317794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:36:25.317794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:36:25.317794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:25.317794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:36:25.318432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:25.318432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:36:25.318432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:36:25.318432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:25.318432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:36:25.318432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:25.318432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:36:25.318432 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:36:25.318432 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:36:25.318432 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:36:25.319134 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:36:25.319134 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:36:25.319260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:25.319972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:36:25.319972 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:25.319972 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:36:25.319972 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:36:25.319972 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:25.319972 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:36:25.319972 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:25.319972 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:36:25.320710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:25.320710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:36:25.320710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:36:25.320710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:36:25.320710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:25.320710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:36:25.320710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:25.320710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:36:25.320710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:25.320710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:36:25.320710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:25.320710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:36:25.320710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:25.320710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:36:25.320710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:36:25.321105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:25.321105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:36:25.321105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:25.321105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:36:25.321105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:36:25.321105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:25.321105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:36:25.321105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:25.321105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:36:25.321559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:25.321559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:36:25.321559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:25.321559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:36:25.321559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:36:25.321559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:36:25.321559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:36:25.321559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:36:25.321559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:36:25.321559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:25.321559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:25.321907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:25.321907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:36:25.321907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:25.321907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:25.321907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:25.321907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:36:25.321907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:25.321907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:36:25.321907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:36:25.321907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:25.321907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:36:25.321907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:25.321907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:36:25.321907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:36:25.322311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:25.322311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:36:25.322311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:25.322311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:36:25.322672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:25.322672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:36:25.322672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:36:25.322672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:36:25.322672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:36:25.322672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:36:25.322672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:25.322672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:36:25.322672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:36:25.322672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:36:25.322672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:36:25.322672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:36:25.322672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:25.322672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:25.322672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:36:25.322672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:36:25.322672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:36:25.323159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:36:25.323159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:36:25.323159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:36:25.323159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:25.323159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:25.323159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:25.323159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:36:25.323159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:25.323159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:25.323159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:25.323159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:36:25.323159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:25.323159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:36:25.323159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:36:25.323159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:25.323159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:36:25.323159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:25.323159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:36:25.323159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:36:25.323661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:25.323661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:36:25.323661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:25.323661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:36:25.323787 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:36:25.323956 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:36:25.324058 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:36:25.324198 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:36:25.324659 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:36:25.325011 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:36:25.325233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:25.325929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:36:25.325929 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:25.325929 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:36:25.325929 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:36:25.325929 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:25.325929 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:36:25.325929 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:25.325929 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:36:25.326766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:25.326766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:36:25.326766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:25.326766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:36:25.326766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:36:25.326766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:36:25.326766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:36:25.326766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:25.326766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:25.326766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:25.326766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:36:25.326766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:25.327167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:25.327167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:25.327167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:36:25.327167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:25.327167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:36:25.327167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:36:25.327167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:25.327167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:36:25.327167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:25.327167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:36:25.327167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:36:25.327167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:25.327167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:36:25.327527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:25.327527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:36:25.327716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:25.327716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:36:25.327716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:36:25.327716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:36:25.327716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:36:25.327716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:36:25.327716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:25.327716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:36:25.327716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:36:25.327716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:36:25.327716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:36:25.327716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:36:25.327716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:25.327716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:25.327716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:36:25.327716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:36:25.328164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:36:25.328164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:36:25.328164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:25.328164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:25.328164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:25.328164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:36:25.328164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:25.328164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:25.328164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:25.328164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:36:25.328164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:25.328164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:36:25.328164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:36:25.328164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:25.328164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:36:25.328164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:25.328164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:36:25.328164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:36:25.328164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:25.328988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:36:25.328988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:25.328988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:36:30.066561 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:36:30.067275 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:36:30.067275 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:36:30.068354 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:36:30.100607 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:36:30.101003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:30.101804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:36:30.101804 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:30.101804 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:36:30.101804 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:36:30.101804 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:30.101804 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:36:30.101804 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:30.101804 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:36:30.102582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:30.102582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:36:30.102582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:30.102582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:36:30.102582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:36:30.102582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:36:30.102582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:36:30.102582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:30.102582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:30.102582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:30.102582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:36:30.102582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:30.103164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:30.103164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:30.103164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:36:30.103164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:30.103164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:36:30.103164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:36:30.103164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:30.103164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:36:30.103164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:30.103164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:36:30.103164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:36:30.103164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:30.103164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:36:30.103783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:30.103783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:36:30.103783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:30.103783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:36:30.103783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:36:30.103783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:36:30.103783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:36:30.103783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:36:30.103783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:30.103783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:36:30.103783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:36:30.103783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:36:30.103783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:36:30.103783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:36:30.103783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:30.104762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:30.104762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:36:30.104762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:36:30.104762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:36:30.104762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:36:30.104762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:30.104762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:30.104762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:30.104762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:36:30.104762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:30.104762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:30.104762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:30.104762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:36:30.104762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:30.104762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:36:30.104762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:36:30.104762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:30.104762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:36:30.104762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:30.105341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:36:30.105341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:36:30.105341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:30.105341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:36:30.105341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:30.105341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:36:30.304588 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:36:30.305003 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:36:30.305184 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:36:30.306173 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:36:30.306518 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:36:30.306754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:30.307628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:36:30.307628 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:30.307628 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:36:30.307628 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:36:30.307628 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:30.307628 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:36:30.307628 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:30.307628 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:36:30.309167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:30.309167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:36:30.309167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:30.309167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:36:30.309167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:36:30.309167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:36:30.309167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:36:30.309167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:30.309167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:30.309167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:30.309167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:36:30.309167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:30.309667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:30.309667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:30.309667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:36:30.309667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:30.309667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:36:30.309667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:36:30.309667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:30.309667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:36:30.309667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:30.309667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:36:30.309667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:36:30.309667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:30.309667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:36:30.310033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:30.310033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:36:30.310794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:30.310794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:36:30.310794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:36:30.310794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:36:30.310794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:36:30.310794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:36:30.310794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:30.310794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:36:30.310794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:36:30.310794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:36:30.310794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:36:30.310794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:36:30.310794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:30.310794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:30.310794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:36:30.310794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:36:30.311233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:36:30.311233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:36:30.311233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:30.311233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:30.311233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:30.311233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:36:30.311233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:30.311233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:30.311233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:30.311233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:36:30.311233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:30.311233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:36:30.311233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:36:30.311233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:30.311233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:36:30.311233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:30.311233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:36:30.311233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:36:30.311233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:30.311802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:36:30.311802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:30.311802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:36:30.313221 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:36:30.313382 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:36:30.313501 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:36:30.313634 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:36:30.314122 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:36:30.314441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:30.315148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:36:30.315148 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:30.315148 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:36:30.315148 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:36:30.315148 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:30.315148 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:36:30.315148 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:30.315148 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:36:30.315921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:30.315921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:36:30.315921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:36:30.315921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:36:30.315921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:30.315921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:36:30.315921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:30.315921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:36:30.315921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:30.315921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:36:30.315921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:30.315921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:36:30.315921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:30.315921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:36:30.315921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:36:30.316942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:30.316942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:36:30.316942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:30.316942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:36:30.316942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:36:30.316942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:30.316942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:36:30.316942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:30.316942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:36:30.316942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:30.316942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:36:30.316942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:30.316942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:36:30.316942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:36:30.317612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:36:30.317612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:36:30.317612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:36:30.317612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:36:30.317612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:36:30.317612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:30.317612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:30.317612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:30.317612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:36:30.317612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:30.317612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:30.317612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:30.317612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:36:30.318090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:30.318090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:36:30.318090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:36:30.318090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:30.318090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:36:30.318090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:30.318090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:36:30.318090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:36:30.318090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:30.318090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:36:30.318090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:30.318090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:36:30.318611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:30.318611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:36:30.318611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:36:30.318611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:36:30.318611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:36:30.318611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:36:30.318611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:30.318611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:36:30.318611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:36:30.318611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:36:30.318611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:36:30.318611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:36:30.318611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:30.318611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:30.318611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:36:30.318611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:36:30.319193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:36:30.319193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:36:30.319193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:36:30.319193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:36:30.319193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:36:30.319193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:30.319193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:30.319193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:30.319193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:36:30.319193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:30.319193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:30.319193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:30.319193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:36:30.319193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:30.319193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:36:30.319193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:36:30.319193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:30.319193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:36:30.320045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:30.320045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:36:30.320045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:36:30.320045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:30.320045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:36:30.320045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:30.320045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:36:30.320045 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:36:30.320045 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:36:30.320045 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:36:30.321081 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:36:30.321081 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:36:30.321081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:30.321081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:36:30.321081 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:30.321081 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:36:30.321081 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:36:30.321081 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:30.321081 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:36:30.321081 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:30.321081 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:36:30.321563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:30.321563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:36:30.321563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:36:30.321563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:36:30.321563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:30.321563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:36:30.321563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:30.321563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:36:30.321563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:30.321563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:36:30.321563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:30.321563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:36:30.321563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:30.321563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:36:30.321563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:36:30.322004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:30.322004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:36:30.322004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:30.322004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:36:30.322004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:36:30.322004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:30.322004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:36:30.322004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:30.322004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:36:30.322004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:30.322004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:36:30.322004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:30.322004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:36:30.322004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:36:30.322505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:36:30.322505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:36:30.322505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:36:30.322505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:36:30.322505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:30.322505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:30.322505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:30.322505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:36:30.322505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:30.322505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:30.322505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:30.322505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:36:30.322505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:30.322843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:36:30.322843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:36:30.322843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:30.322843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:36:30.322843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:30.322843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:36:30.322843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:36:30.322843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:30.322843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:36:30.322843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:30.322843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:36:30.323159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:30.323159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:36:30.323159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:36:30.323159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:36:30.323159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:36:30.323159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:36:30.323159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:30.323159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:36:30.323159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:36:30.323159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:36:30.323159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:36:30.323159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:36:30.323159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:30.323159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:30.323159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:36:30.323159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:36:30.323159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:36:30.323701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:36:30.323701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:36:30.323701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:36:30.323701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:30.323701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:30.323701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:30.323701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:36:30.323701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:30.323701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:30.323701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:30.323701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:36:30.323701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:30.323701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:36:30.323701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:36:30.323701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:30.323701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:36:30.323701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:30.323701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:36:30.323701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:36:30.324232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:30.324232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:36:30.324232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:30.324232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:36:30.324232 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:36:30.324232 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:36:30.324426 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:36:30.324467 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:36:30.325000 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:36:30.325370 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:36:30.325624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:30.326325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:36:30.326325 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:30.326325 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:36:30.326325 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:36:30.326325 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:30.326325 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:36:30.326325 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:30.326325 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:36:30.327171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:30.327171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:36:30.327171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:30.327171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:36:30.327171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:36:30.327171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:36:30.327171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:36:30.327171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:30.327171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:30.327171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:30.327171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:36:30.327171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:30.327422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:30.327422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:30.327422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:36:30.327422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:30.327422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:36:30.327422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:36:30.327422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:30.327422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:36:30.327422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:30.327422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:36:30.327422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:36:30.327422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:30.327422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:36:30.327681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:30.327681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:36:30.328151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:30.328151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:36:30.328151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:36:30.328151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:36:30.328151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:36:30.328151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:36:30.328151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:30.328151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:36:30.328151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:36:30.328151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:36:30.328151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:36:30.328151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:36:30.328151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:30.328151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:30.328151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:36:30.328151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:36:30.328513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:36:30.328513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:36:30.328513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:30.328513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:30.328513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:30.328513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:36:30.328513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:30.328513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:30.328513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:30.328513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:36:30.328513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:30.328513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:36:30.328513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:36:30.328513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:30.328513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:36:30.328513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:30.328513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:36:30.328513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:36:30.328513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:30.329021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:36:30.329021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:30.329021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:36:35.067150 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:36:35.068192 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:36:35.068192 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:36:35.068899 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:36:35.100154 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:36:35.100569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:35.101425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:36:35.101425 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:35.101425 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:36:35.101425 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:36:35.101425 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:35.101425 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:36:35.101425 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:35.101425 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:36:35.102557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:35.102557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:36:35.102557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:35.102557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:36:35.102557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:36:35.102557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:36:35.102557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:36:35.102557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:35.102557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:35.102557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:35.102557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:36:35.102557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:35.103765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:35.103765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:35.103765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:36:35.103765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:35.103765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:36:35.103765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:36:35.103765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:35.103765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:36:35.103765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:35.103765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:36:35.103765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:36:35.103765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:35.103765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:36:35.104737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:35.104737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:36:35.104737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:35.104737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:36:35.104737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:36:35.104737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:36:35.104737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:36:35.104737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:36:35.104737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:35.104737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:36:35.104737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:36:35.104737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:36:35.104737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:36:35.104737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:36:35.104737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:35.106649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:35.106649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:36:35.106649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:36:35.106649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:36:35.106649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:36:35.106649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:35.106649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:35.106649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:35.106649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:36:35.106649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:35.106649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:35.106649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:35.106649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:36:35.106649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:35.106649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:36:35.106649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:36:35.106649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:35.106649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:36:35.106649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:35.108034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:36:35.108034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:36:35.108034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:35.108034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:36:35.108034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:35.108034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:36:35.305207 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:36:35.305675 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:36:35.305856 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:36:35.307017 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:36:35.307434 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:36:35.307758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:35.308645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:36:35.308645 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:35.308645 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:36:35.308645 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:36:35.308645 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:35.308645 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:36:35.308645 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:35.308645 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:36:35.310021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:35.310021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:36:35.310021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:35.310021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:36:35.310021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:36:35.310021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:36:35.310021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:36:35.310021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:35.310021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:35.310021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:35.310021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:36:35.310021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:35.310948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:35.310948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:35.310948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:36:35.310948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:35.310948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:36:35.310948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:36:35.310948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:35.310948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:36:35.310948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:35.310948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:36:35.310948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:36:35.310948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:35.310948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:36:35.311749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:35.311749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:36:35.311749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:35.311749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:36:35.311749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:36:35.311749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:36:35.311749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:36:35.311749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:36:35.311749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:35.311749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:36:35.311749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:36:35.311749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:36:35.311749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:36:35.311749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:36:35.311749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:35.312784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:35.312784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:36:35.312784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:36:35.312784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:36:35.312784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:36:35.312784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:35.312784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:35.312784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:35.312784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:36:35.312784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:35.312784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:35.312784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:35.312784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:36:35.312784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:35.312784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:36:35.312784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:36:35.312784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:35.312784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:36:35.312784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:35.313644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:36:35.313644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:36:35.313644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:35.313644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:36:35.313644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:35.313644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:36:35.313644 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:36:35.313644 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:36:35.313969 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:36:35.313969 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:36:35.314176 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:36:35.314421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:35.314889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:36:35.314889 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:35.314889 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:36:35.314889 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:36:35.314889 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:35.314889 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:36:35.314889 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:35.314889 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:36:35.315595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:35.315595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:36:35.315595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:36:35.315595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:36:35.315595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:35.315595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:36:35.315595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:35.315595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:36:35.315595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:35.315595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:36:35.315595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:35.315595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:36:35.315595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:35.315595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:36:35.315595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:36:35.316243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:35.316243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:36:35.316243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:35.316243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:36:35.316243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:36:35.316243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:35.316243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:36:35.316243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:35.316243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:36:35.316656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:35.316656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:36:35.316656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:35.316656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:36:35.316656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:36:35.316656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:36:35.316656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:36:35.316656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:36:35.316656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:36:35.316656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:36:35.316656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:35.316656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:35.317138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:35.317138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:36:35.317138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:35.317138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:35.317138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:35.317138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:36:35.317138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:35.317138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:36:35.317138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:36:35.317138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:35.317138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:36:35.317138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:35.317138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:36:35.317138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:36:35.317649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:35.317649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:36:35.317649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:35.317649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:36:35.317649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:35.317649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:36:35.317649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:36:35.317649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:36:35.317649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:36:35.317649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:36:35.317649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:35.317649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:36:35.317649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:36:35.317649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:36:35.318174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:36:35.318174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:36:35.318174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:35.318174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:35.318174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:36:35.318174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:36:35.318174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:36:35.318174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:36:35.318174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:36:35.318174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:36:35.318174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:36:35.318174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:35.318174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:35.318174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:35.318174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:36:35.318174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:35.318174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:35.318174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:35.318174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:36:35.318923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:35.318923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:36:35.318923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:36:35.318923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:35.318923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:36:35.318923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:35.318923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:36:35.318923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:36:35.318923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:35.318923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:36:35.318923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:35.318923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:36:35.318923 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:36:35.318923 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:36:35.319520 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:36:35.319520 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:36:35.319520 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:36:35.319664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:35.320169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:36:35.320169 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:35.320169 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:36:35.320169 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:36:35.320169 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:35.320169 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:36:35.320169 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:35.320169 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:36:35.320853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:35.320853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:36:35.320853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:36:35.320853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:36:35.320853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:35.320853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:36:35.320853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:35.320853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:36:35.320853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:35.320853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:36:35.320853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:35.320853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:36:35.320853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:35.320853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:36:35.320853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:36:35.321142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:35.321142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:36:35.321142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:35.321142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:36:35.321142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:36:35.321142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:35.321142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:36:35.321142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:35.321142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:36:35.321747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:35.321747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:36:35.321747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:35.321747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:36:35.321747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:36:35.321747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:36:35.321747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:36:35.321747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:36:35.321747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:36:35.321747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:35.321747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:35.321982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:35.321982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:36:35.321982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:35.321982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:35.321982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:35.321982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:36:35.321982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:35.321982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:36:35.321982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:36:35.321982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:35.321982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:36:35.321982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:35.321982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:36:35.321982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:36:35.322220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:35.322220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:36:35.322220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:35.322220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:36:35.322828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:35.322828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:36:35.322828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:36:35.322828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:36:35.322828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:36:35.322828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:36:35.322828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:35.322828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:36:35.322828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:36:35.322828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:36:35.322828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:36:35.322828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:36:35.322828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:35.322828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:35.322828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:36:35.322828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:36:35.322828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:36:35.323300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:36:35.323300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:36:35.323300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:36:35.323300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:35.323300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:35.323300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:35.323300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:36:35.323300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:35.323300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:35.323300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:35.323300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:36:35.323300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:35.323300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:36:35.323300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:36:35.323300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:35.323300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:36:35.323300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:35.323300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:36:35.323300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:36:35.323807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:35.323807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:36:35.323807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:35.323807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:36:35.324269 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:36:35.324475 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:36:35.324581 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:36:35.324726 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:36:35.324956 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:36:35.325158 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:36:35.325284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:35.325689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:36:35.325689 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:35.325689 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:36:35.325689 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:36:35.325689 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:35.325689 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:36:35.325689 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:35.325689 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:36:35.326496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:35.326496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:36:35.326496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:35.326496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:36:35.326496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:36:35.326496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:36:35.326496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:36:35.326496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:35.326496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:35.326496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:35.326496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:36:35.326496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:35.326835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:35.326835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:35.326835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:36:35.326835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:35.326835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:36:35.326835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:36:35.326835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:35.326835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:36:35.326835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:35.326835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:36:35.326835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:36:35.326835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:35.326835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:36:35.327186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:35.327186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:36:35.327527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:35.327527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:36:35.327527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:36:35.327527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:36:35.327527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:36:35.327527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:36:35.327527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:35.327527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:36:35.327527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:36:35.327527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:36:35.327527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:36:35.327527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:36:35.327527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:35.327527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:35.327527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:36:35.327527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:36:35.327954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:36:35.327954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:36:35.327954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:35.327954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:35.327954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:35.327954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:36:35.327954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:35.327954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:35.327954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:35.327954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:36:35.327954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:35.327954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:36:35.327954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:36:35.327954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:35.327954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:36:35.327954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:35.327954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:36:35.327954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:36:35.327954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:35.328457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:36:35.328457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:35.328457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:36:40.067808 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:36:40.068525 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:36:40.068664 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:36:40.070072 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:36:40.102897 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:36:40.103351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:40.104150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:36:40.104150 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:40.104150 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:36:40.104150 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:36:40.104150 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:40.104150 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:36:40.104150 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:40.104150 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:36:40.104879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:40.104879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:36:40.104879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:40.104879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:36:40.104879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:36:40.104879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:36:40.104879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:36:40.104879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:40.104879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:40.104879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:40.104879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:36:40.104879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:40.105500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:40.105500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:40.105500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:36:40.105500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:40.105500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:36:40.105500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:36:40.105500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:40.105500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:36:40.105500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:40.105500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:36:40.105500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:36:40.105500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:40.105500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:36:40.106114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:40.106114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:36:40.106114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:40.106114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:36:40.106114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:36:40.106114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:36:40.106114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:36:40.106114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:36:40.106114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:40.106114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:36:40.106114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:36:40.106114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:36:40.106114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:36:40.106114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:36:40.106114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:40.106825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:40.106825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:36:40.106825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:36:40.106825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:36:40.106825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:36:40.106825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:40.106825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:40.106825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:40.106825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:36:40.106825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:40.106825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:40.106825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:40.106825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:36:40.106825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:40.106825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:36:40.106825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:36:40.106825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:40.106825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:36:40.106825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:40.107707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:36:40.107707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:36:40.107707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:40.107707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:36:40.107707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:40.107707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:36:40.305894 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:36:40.306317 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:36:40.306523 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:36:40.307757 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:36:40.308155 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:36:40.308494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:40.309427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:36:40.309427 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:40.309427 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:36:40.309427 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:36:40.309427 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:40.309427 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:36:40.309427 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:40.309427 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:36:40.310989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:40.310989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:36:40.310989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:40.310989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:36:40.310989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:36:40.310989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:36:40.310989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:36:40.310989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:40.310989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:40.310989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:40.310989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:36:40.310989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:40.311674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:40.311674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:40.311674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:36:40.311674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:40.311674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:36:40.311674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:36:40.311674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:40.311674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:36:40.311674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:40.311674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:36:40.311674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:36:40.311674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:40.311674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:36:40.312294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:40.312294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:36:40.312603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:40.312603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:36:40.312603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:36:40.312603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:36:40.312603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:36:40.312603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:36:40.312603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:40.312603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:36:40.312603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:36:40.312603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:36:40.312603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:36:40.312603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:36:40.312603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:40.312603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:40.312603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:36:40.312603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:36:40.313511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:36:40.313511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:36:40.313511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:40.313511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:40.313511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:40.313511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:36:40.313511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:40.313511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:40.313511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:40.313511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:36:40.313511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:40.313511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:36:40.313511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:36:40.313511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:40.313511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:36:40.313511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:40.313511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:36:40.313511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:36:40.313511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:40.314269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:36:40.314269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:40.314269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:36:40.314269 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:36:40.314269 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:36:40.314269 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:36:40.314593 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:36:40.315004 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:36:40.315249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:40.315957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:36:40.315957 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:40.315957 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:36:40.315957 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:36:40.315957 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:40.315957 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:36:40.315957 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:40.315957 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:36:40.316768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:40.316768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:36:40.316768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:36:40.316768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:36:40.316768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:40.316768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:36:40.316768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:40.316768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:36:40.316768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:40.316768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:36:40.316768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:40.316768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:36:40.316768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:40.316768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:36:40.316768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:36:40.317168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:40.317168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:36:40.317168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:40.317168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:36:40.317168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:36:40.317168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:40.317168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:36:40.317168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:40.317168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:36:40.317569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:40.317569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:36:40.317569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:40.317569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:36:40.317569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:36:40.317569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:36:40.317569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:36:40.317569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:36:40.317569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:36:40.317569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:36:40.317569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:40.317569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:40.317899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:40.317899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:36:40.317899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:40.317899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:40.317899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:40.317899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:36:40.317899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:40.317899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:36:40.317899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:36:40.317899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:40.317899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:36:40.317899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:40.317899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:36:40.317899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:36:40.318276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:40.318276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:36:40.318276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:40.318276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:36:40.318873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:40.318873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:36:40.318873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:36:40.318873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:36:40.318873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:36:40.318873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:36:40.318873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:40.318873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:36:40.318873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:36:40.318873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:36:40.318873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:36:40.318873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:36:40.318873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:40.318873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:40.318873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:36:40.318873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:36:40.319679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:36:40.319679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:36:40.319679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:36:40.319679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:36:40.319679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:36:40.319679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:40.319679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:40.319679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:40.319679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:36:40.319679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:40.319679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:40.319679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:40.319679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:36:40.319679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:40.319679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:36:40.319679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:36:40.319679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:40.319679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:36:40.320144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:40.320144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:36:40.320144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:36:40.320144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:40.320144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:36:40.320144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:40.320144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:36:40.320144 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:36:40.320144 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:36:40.320144 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:36:40.320500 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:36:40.320500 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:36:40.320500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:40.320500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:36:40.320500 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:40.320500 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:36:40.320500 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:36:40.320500 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:40.320500 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:36:40.320500 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:40.320500 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:36:40.321211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:40.321211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:36:40.321211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:36:40.321211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:36:40.321211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:40.321211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:36:40.321211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:40.321211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:36:40.321211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:40.321211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:36:40.321211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:40.321211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:36:40.321211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:40.321211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:36:40.321211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:36:40.321675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:40.321675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:36:40.321675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:40.321675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:36:40.321675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:36:40.321675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:40.321675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:36:40.321675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:40.321675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:36:40.322066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:40.322066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:36:40.322066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:40.322066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:36:40.322066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:36:40.322066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:36:40.322066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:36:40.322066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:36:40.322066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:36:40.322066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:40.322066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:40.322376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:40.322376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:36:40.322376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:40.322376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:40.322376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:40.322376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:36:40.322376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:40.322376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:36:40.322376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:36:40.322376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:40.322376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:36:40.322376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:40.322376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:36:40.322376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:36:40.322760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:40.322760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:36:40.322760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:40.322760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:36:40.323116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:40.323116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:36:40.323116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:36:40.323116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:36:40.323116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:36:40.323116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:36:40.323116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:40.323116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:36:40.323116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:36:40.323116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:36:40.323116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:36:40.323116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:36:40.323116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:40.323116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:40.323116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:36:40.323116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:36:40.323116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:36:40.323590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:36:40.323590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:36:40.323590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:36:40.323590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:40.323590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:40.323590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:40.323590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:36:40.323590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:40.323590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:40.323590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:40.323590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:36:40.323590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:40.323590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:36:40.323590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:36:40.323590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:40.323590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:36:40.323590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:40.323590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:36:40.323590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:36:40.324100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:40.324100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:36:40.324100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:40.324100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:36:40.324533 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:36:40.324708 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:36:40.324810 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:36:40.324944 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:36:40.325275 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:36:40.325568 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:36:40.325758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:40.326201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:36:40.326201 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:40.326201 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:36:40.326201 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:36:40.326201 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:40.326201 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:36:40.326201 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:40.326201 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:36:40.326982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:40.326982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:36:40.326982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:40.326982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:36:40.326982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:36:40.326982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:36:40.326982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:36:40.326982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:40.326982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:40.326982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:40.326982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:36:40.326982 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:40.327345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:40.327345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:40.327345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:36:40.327345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:40.327345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:36:40.327345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:36:40.327345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:40.327345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:36:40.327345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:40.327345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:36:40.327345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:36:40.327345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:40.327345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:36:40.327716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:40.327716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:36:40.327896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:40.327896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:36:40.327896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:36:40.327896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:36:40.327896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:36:40.327896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:36:40.327896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:40.327896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:36:40.327896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:36:40.327896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:36:40.327896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:36:40.327896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:36:40.327896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:40.327896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:40.327896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:36:40.327896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:36:40.328368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:36:40.328368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:36:40.328368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:40.328368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:40.328368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:40.328368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:36:40.328368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:40.328368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:40.328368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:40.328368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:36:40.328368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:40.328368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:36:40.328368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:36:40.328368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:40.328368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:36:40.328368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:40.328368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:36:40.328368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:36:40.328368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:40.328909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:36:40.328909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:40.328909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:36:45.068742 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:36:45.069630 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:36:45.069630 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:36:45.070576 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:36:45.114582 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:36:45.114964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:45.115973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:36:45.115973 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:45.115973 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:36:45.115973 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:36:45.115973 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:45.115973 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:36:45.115973 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:45.115973 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:36:45.117076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.047 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:45.117076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:36:45.117076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:45.117076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:36:45.117076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:36:45.117076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:36:45.117076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:36:45.117076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:45.117076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:45.117076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:45.117076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:36:45.117076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:45.117850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:45.117850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:45.117850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:36:45.117850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:45.117850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:36:45.117850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:36:45.117850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:45.117850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:36:45.117850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:45.117850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:36:45.117850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:36:45.117850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:45.117850 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:36:45.118523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:45.118523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:36:45.118523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:45.118523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:36:45.118523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:36:45.118523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:36:45.118523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:36:45.118523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:36:45.118523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:45.118523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:36:45.118523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:36:45.118523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:36:45.118523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:36:45.118523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:36:45.118523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:45.119395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:45.119395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:36:45.119395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:36:45.119395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:36:45.119395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:36:45.119395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:45.119395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:45.119395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:45.119395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:36:45.119395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:45.119395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:45.119395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:45.119395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:36:45.119395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:45.119395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:36:45.119395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:36:45.119395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:45.119395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:36:45.119395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:45.120371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:36:45.120371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:36:45.120371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:45.120371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:36:45.120371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:45.120371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:36:45.306615 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:36:45.307302 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:36:45.307564 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:36:45.308639 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:36:45.308986 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:36:45.309230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:45.310086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:36:45.310086 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:45.310086 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:36:45.310086 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:36:45.310086 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:45.310086 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:36:45.310086 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:45.310086 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:36:45.311660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:45.311660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:36:45.311660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:45.311660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:36:45.311660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:36:45.311660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:36:45.311660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:36:45.311660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:45.311660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:45.311660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:45.311660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:36:45.311660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:45.311912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:45.311912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:45.311912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:36:45.311912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:45.311912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:36:45.311912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:36:45.311912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:45.311912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:36:45.311912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:45.311912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:36:45.311912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:36:45.311912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:45.311912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:36:45.312164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:45.312164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:36:45.313378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:45.313378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:36:45.313378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:36:45.313378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:36:45.313378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:36:45.313378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:36:45.313378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:45.313378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:36:45.313378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:36:45.313378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:36:45.313378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:36:45.313378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:36:45.313378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:45.313378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:45.313378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:36:45.313378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:36:45.313701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:36:45.313701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:36:45.313701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:45.313701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:45.313701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:45.313701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:36:45.313701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:45.313701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:45.313701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:45.313701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:36:45.313701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:45.313701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:36:45.313701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:36:45.313701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:45.313701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:36:45.313701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:45.313701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:36:45.313701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:36:45.313701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:45.314044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:36:45.314044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:45.314044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:36:45.314369 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:36:45.314748 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:36:45.314893 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:36:45.315048 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:36:45.315537 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:36:45.315814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:45.316556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:36:45.316556 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:45.316556 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:36:45.316556 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:36:45.316556 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:45.316556 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:36:45.316556 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:45.316556 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:36:45.317274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:45.317274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:36:45.317274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:36:45.317274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:36:45.317274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:45.317274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:36:45.317274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:45.317274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:36:45.317274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:45.317274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:36:45.317274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:45.317274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:36:45.317274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:45.317274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:36:45.317274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:36:45.317610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:45.317610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:36:45.317610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:45.317610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:36:45.317610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:36:45.317610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:45.317610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:36:45.317610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:45.317610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:36:45.318097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:45.318097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:36:45.318097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:45.318097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:36:45.318097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:36:45.318097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:36:45.318097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:36:45.318097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:36:45.318097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:36:45.318097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:36:45.318097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:45.318097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:45.318353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:45.318353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:36:45.318353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:45.318353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:45.318353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:45.318353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:36:45.318353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:45.318353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:36:45.318353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:36:45.318353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:45.318353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:36:45.318353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:45.318353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:36:45.318353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:36:45.318613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:45.318613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:36:45.318613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:45.318613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:36:45.319080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:45.319080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:36:45.319080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:36:45.319080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:36:45.319080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:36:45.319080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:36:45.319080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:45.319080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:36:45.319080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:36:45.319080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:36:45.319080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:36:45.319080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:36:45.319080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:45.319080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:45.319080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:36:45.319080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:36:45.319379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:36:45.319379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:36:45.319379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:36:45.319379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:36:45.319379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:36:45.319379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:45.319379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:45.319379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:45.319379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:36:45.319379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:45.319379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:45.319379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:45.319379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:36:45.319379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:45.319379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:36:45.319379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:36:45.319379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:45.319379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:36:45.319727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:45.319727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:36:45.319727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:36:45.319727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:45.319727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:36:45.319727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:45.319727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:36:45.319727 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:36:45.319881 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:36:45.319912 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:36:45.320034 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:36:45.320549 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:36:45.320883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:45.321563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:36:45.321563 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:45.321563 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:36:45.321563 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:36:45.321563 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:45.321563 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:36:45.321563 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:45.321563 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:36:45.322257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:45.322257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:36:45.322257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:36:45.322257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:36:45.322257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:45.322257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:36:45.322257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:45.322257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:36:45.322257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:45.322257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:36:45.322257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:45.322257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:36:45.322257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:45.322257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:36:45.322257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:36:45.322553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:45.322553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:36:45.322553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:45.322553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:36:45.322553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:36:45.322553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:45.322553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:36:45.322553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:45.322553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:36:45.323095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:45.323095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:36:45.323095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:45.323095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:36:45.323095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:36:45.323095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:36:45.323095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:36:45.323095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:36:45.323095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:36:45.323095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:45.323095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:45.323310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:45.323310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:36:45.323310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:45.323310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:45.323310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:45.323310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:36:45.323310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:45.323310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:36:45.323310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:36:45.323310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:45.323310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:36:45.323310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:45.323310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:36:45.323310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:36:45.323572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:45.323572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:36:45.323572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:45.323572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:36:45.324105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:45.324105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:36:45.324105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:36:45.324105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:36:45.324105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:36:45.324105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:36:45.324105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:45.324105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:36:45.324105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:36:45.324105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:36:45.324105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:36:45.324105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:36:45.324105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:45.324105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:45.324105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:36:45.324105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:36:45.324105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:36:45.324470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:36:45.324470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:36:45.324470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:36:45.324470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:45.324470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:45.324470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:45.324470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:36:45.324470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:45.324470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:45.324470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:45.324470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:36:45.324470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:45.324470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:36:45.324470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:36:45.324470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:45.324470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:36:45.324470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:45.324470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:36:45.324470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:36:45.324826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:45.324826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:36:45.324826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:45.324826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:36:45.324904 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:36:45.325172 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:36:45.325278 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:36:45.325418 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:36:45.325850 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:36:45.326267 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:36:45.326597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:45.327248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:36:45.327248 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:45.327248 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:36:45.327248 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:36:45.327248 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:45.327248 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:36:45.327248 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:45.327248 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:36:45.328030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:45.328030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:36:45.328030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:45.328030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:36:45.328030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:36:45.328030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:36:45.328030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:36:45.328030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:45.328030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:45.328030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:45.328030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:36:45.328030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:45.328291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:45.328291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:45.328291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:36:45.328291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:45.328291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:36:45.328291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:36:45.328291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:45.328291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:36:45.328291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:45.328291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:36:45.328291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:36:45.328291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:45.328291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:36:45.328556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:45.328556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:36:45.328976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:45.328976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:36:45.328976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:36:45.328976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:36:45.328976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:36:45.328976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:36:45.328976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:45.328976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:36:45.328976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:36:45.328976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:36:45.328976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:36:45.328976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:36:45.328976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:45.328976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:45.328976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:36:45.328976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:36:45.329296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:36:45.329296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:36:45.329296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:45.329296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:45.329296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:45.329296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:36:45.329296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:45.329296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:45.329296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:45.329296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:36:45.329296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:45.329296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:36:45.329296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:36:45.329296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:45.329296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:36:45.329296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:45.329296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:36:45.329296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:36:45.329296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:45.329687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:36:45.329687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:45.329687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:36:50.069348 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:36:50.070084 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:36:50.070172 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:36:50.071755 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:36:50.104886 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:36:50.105085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:50.105522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:36:50.105522 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:50.105522 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:36:50.105522 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:36:50.105522 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:50.105522 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:36:50.105522 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:50.105522 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:36:50.106153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:50.106153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:36:50.106153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:50.106153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:36:50.106153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:36:50.106153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:36:50.106153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:36:50.106153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:50.106153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:50.106153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:50.106153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:36:50.106153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:50.106855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:50.106855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:50.106855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:36:50.106855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:50.106855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:36:50.106855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:36:50.106855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:50.106855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:36:50.106855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:50.106855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:36:50.106855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:36:50.106855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:50.106855 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:36:50.107546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:50.107546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:36:50.107546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:50.107546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:36:50.107546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:36:50.107546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:36:50.107546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:36:50.107546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:36:50.107546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:50.107546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:36:50.107546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:36:50.107546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:36:50.107546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:36:50.107546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:36:50.107546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:50.108254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:50.108254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:36:50.108254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:36:50.108254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:36:50.108254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:36:50.108254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:50.108254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:50.108254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:50.108254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:36:50.108254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:50.108254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:50.108254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:50.108254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:36:50.108254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:50.108254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:36:50.108254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:36:50.108254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:50.108254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:36:50.108254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:50.109186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:36:50.109186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:36:50.109186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:50.109186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:36:50.109186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:50.109186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:36:50.307394 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:36:50.307992 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:36:50.308256 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:36:50.309171 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:36:50.309541 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:36:50.309777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:50.310601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:36:50.310601 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:50.310601 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:36:50.310601 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:36:50.310601 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:50.310601 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:36:50.310601 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:50.310601 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:36:50.312318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:50.312318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:36:50.312318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:50.312318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:36:50.312318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:36:50.312318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:36:50.312318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:36:50.312318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:50.312318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:50.312318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:50.312318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:36:50.312318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:50.313015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:50.313015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:50.313015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:36:50.313015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:50.313015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:36:50.313015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:36:50.313015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:50.313015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:36:50.313015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:50.313015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:36:50.313015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:36:50.313015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:50.313015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:36:50.313644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:50.313644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:36:50.314096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:50.314096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:36:50.314096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:36:50.314096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:36:50.314096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:36:50.314096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:36:50.314096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:50.314096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:36:50.314096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:36:50.314096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:36:50.314096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:36:50.314096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:36:50.314096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:50.314096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:50.314096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:36:50.314096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:36:50.314956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:36:50.314956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:36:50.314956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:50.314956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:50.314956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:50.314956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:36:50.314956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:50.314956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:50.314956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:50.314956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:36:50.314956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:50.314956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:36:50.314956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:36:50.314956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:50.314956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:36:50.314956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:50.314956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:36:50.314956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:36:50.314956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:50.315859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:36:50.315859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:50.315859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:36:50.315859 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:36:50.315859 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:36:50.315859 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:36:50.315859 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:36:50.316253 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:36:50.316327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:50.316929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:36:50.316929 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:50.316929 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:36:50.316929 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:36:50.316929 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:50.316929 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:36:50.316929 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:50.316929 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:36:50.318431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:50.318431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:36:50.318431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:36:50.318431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:36:50.318431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:50.318431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:36:50.318431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:50.318431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:36:50.318431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:50.318431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:36:50.318431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:50.318431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:36:50.318431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:50.318431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:36:50.318431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:36:50.318957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:50.318957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:36:50.318957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:50.318957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:36:50.318957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:36:50.318957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:50.318957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:36:50.318957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:50.318957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:36:50.319725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:50.319725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:36:50.319725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:50.319725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:36:50.319725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:36:50.319725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:36:50.319725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:36:50.319725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:36:50.319725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:36:50.319725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:36:50.319725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:50.319725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:50.320117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:50.320117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:36:50.320117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:50.320117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:50.320117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:50.320117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:36:50.320117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:50.320117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:36:50.320117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:36:50.320117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:50.320117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:36:50.320117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:50.320117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:36:50.320117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:36:50.320503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:50.320503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:36:50.320503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:50.320503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:36:50.320916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:50.320916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:36:50.320916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:36:50.320916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:36:50.320916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:36:50.320916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:36:50.320916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:50.320916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:36:50.320916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:36:50.320916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:36:50.320916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:36:50.320916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:36:50.320916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:50.320916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:50.320916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:36:50.320916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:36:50.321378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:36:50.321378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:36:50.321378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:36:50.321378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:36:50.321378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:36:50.321378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:50.321378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:50.321378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:50.321378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:36:50.321378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:50.321378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:50.321378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:50.321378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:36:50.321378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:50.321378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:36:50.321378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:36:50.321378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:50.321378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:36:50.321865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:50.321865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:36:50.321865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:36:50.321865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:50.321865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:36:50.321865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:50.321865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:36:50.321865 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:36:50.321865 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:36:50.321865 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:36:50.322148 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:36:50.322148 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:36:50.322220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:50.322686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:36:50.322686 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:50.322686 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:36:50.322686 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:36:50.322686 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:50.322686 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:36:50.322686 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:50.322686 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:36:50.323399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:50.323399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:36:50.323399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:36:50.323399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:36:50.323399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:50.323399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:36:50.323399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:50.323399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:36:50.323399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:50.323399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:36:50.323399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:50.323399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:36:50.323399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:50.323399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:36:50.323399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:36:50.323949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:50.323949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:36:50.323949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:50.323949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:36:50.323949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:36:50.323949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:50.323949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:36:50.323949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:50.323949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:36:50.324244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:50.324244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:36:50.324244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:50.324244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:36:50.324244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:36:50.324244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:36:50.324244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:36:50.324244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:36:50.324244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:36:50.324244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:50.324244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:50.324719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:50.324719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:36:50.324719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:50.324719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:50.324719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:50.324719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:36:50.324719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:50.324719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:36:50.324719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:36:50.324719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:50.324719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:36:50.324719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:50.324719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:36:50.324719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:36:50.325210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:50.325210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:36:50.325210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:50.325210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:36:50.325358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:50.325358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:36:50.325358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:36:50.325358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:36:50.325358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:36:50.325358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:36:50.325358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:50.325358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:36:50.325358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:36:50.325358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:36:50.325358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:36:50.325358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:36:50.325358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:50.325358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:50.325358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:36:50.325358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:36:50.325358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:36:50.325963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:36:50.325963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:36:50.325963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:36:50.325963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:50.325963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:50.325963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:50.325963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:36:50.325963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:50.325963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:50.325963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:50.325963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:36:50.325963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:50.325963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:36:50.325963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:36:50.325963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:50.325963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:36:50.325963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:50.325963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:36:50.325963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:36:50.326664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:50.326664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:36:50.326664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:50.326664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:36:50.326664 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:36:50.326664 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:36:50.326664 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:36:50.326664 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:36:50.327042 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:36:50.327042 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:36:50.327042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:50.327180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:36:50.327180 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:50.327180 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:36:50.327180 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:36:50.327180 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:50.327180 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:36:50.327180 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:50.327180 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:36:50.327927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:50.327927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:36:50.327927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:50.327927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:36:50.327927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:36:50.327927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:36:50.327927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:36:50.327927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:50.327927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:50.327927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:50.327927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:36:50.327927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:50.328387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:50.328387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:50.328387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:36:50.328387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:50.328387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:36:50.328387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:36:50.328387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:50.328387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:36:50.328387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:50.328387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:36:50.328387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:36:50.328387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:50.328387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:36:50.328887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:50.328887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:36:50.328996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:50.328996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:36:50.328996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:36:50.328996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:36:50.328996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:36:50.328996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:36:50.328996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:50.328996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:36:50.328996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:36:50.328996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:36:50.328996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:36:50.328996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:36:50.328996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:50.328996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:50.328996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:36:50.328996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:36:50.329632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:36:50.329632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:36:50.329632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:50.329632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:50.329632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:50.329632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:36:50.329632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:50.329632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:50.329632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:50.329632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:36:50.329632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:50.329632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:36:50.329632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:36:50.329632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:50.329632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:36:50.329632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:50.329632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:36:50.329632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:36:50.329632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:50.330235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:36:50.330235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:50.330235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:36:55.070041 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:36:55.070749 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:36:55.070971 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:36:55.072371 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:36:55.104115 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:36:55.104498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:55.105240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:36:55.105240 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:55.105240 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:36:55.105240 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:36:55.105240 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:55.105240 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:36:55.105240 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:55.105240 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:36:55.106210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:55.106210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:36:55.106210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:55.106210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:36:55.106210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:36:55.106210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:36:55.106210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:36:55.106210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:55.106210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:55.106210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:55.106210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:36:55.106210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:55.107074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:55.107074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:55.107074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:36:55.107074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:55.107074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:36:55.107074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:36:55.107074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:55.107074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:36:55.107074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:55.107074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:36:55.107074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:36:55.107074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:55.107074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:36:55.108056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:55.108056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:36:55.108056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:55.108056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:36:55.108056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:36:55.108056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:36:55.108056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:36:55.108056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:36:55.108056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:55.108056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:36:55.108056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:36:55.108056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:36:55.108056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:36:55.108056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:36:55.108056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:55.108975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:55.108975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:36:55.108975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:36:55.108975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:36:55.108975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:36:55.108975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:55.108975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:55.108975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:55.108975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:36:55.108975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:55.108975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:55.108975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:55.108975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:36:55.108975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:55.108975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:36:55.108975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:36:55.108975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:55.108975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:36:55.108975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:55.109871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:36:55.109871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:36:55.109871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:55.109871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:36:55.109871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:55.109871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:36:55.308064 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:36:55.308834 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:36:55.309029 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:36:55.310081 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:36:55.310428 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:36:55.310703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:55.311538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:36:55.311538 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:55.311538 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:36:55.311538 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:36:55.311538 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:55.311538 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:36:55.311538 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:55.311538 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:36:55.313041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:55.313041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:36:55.313041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:55.313041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:36:55.313041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:36:55.313041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:36:55.313041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:36:55.313041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:55.313041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:55.313041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:55.313041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:36:55.313041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:55.313436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:55.313436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:55.313436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:36:55.313436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:55.313436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:36:55.313436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:36:55.313436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:55.313436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:36:55.313436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:55.313436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:36:55.313436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:36:55.313436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:55.313436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:36:55.313932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:55.313932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:36:55.314776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:55.314776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:36:55.314776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:36:55.314776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:36:55.314776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:36:55.314776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:36:55.314776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:55.314776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:36:55.314776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:36:55.314776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:36:55.314776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:36:55.314776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:36:55.314776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:55.314776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:55.314776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:36:55.314776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:36:55.315678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:36:55.315678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:36:55.315678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:55.315678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:55.315678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:55.315678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:36:55.315678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:55.315678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:55.315678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:55.315678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:36:55.315678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:55.315678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:36:55.315678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:36:55.315678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:55.315678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:36:55.315678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:55.315678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:36:55.315678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:36:55.315678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:55.316518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:36:55.316518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:55.316518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:36:55.316518 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:36:55.316518 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:36:55.316518 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:36:55.316518 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:36:55.316518 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:36:55.316954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:55.317431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:36:55.317431 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:55.317431 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:36:55.317431 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:36:55.317431 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:55.317431 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:36:55.317431 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:55.317431 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:36:55.318148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:55.318148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:36:55.318148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:36:55.318148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:36:55.318148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:55.318148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:36:55.318148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:55.318148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:36:55.318148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:55.318148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:36:55.318148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:55.318148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:36:55.318148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:55.318148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:36:55.318148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:36:55.318984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:55.318984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:36:55.318984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:55.318984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:36:55.318984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:36:55.318984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:55.318984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:36:55.318984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:55.318984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:36:55.318984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:55.318984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:36:55.318984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:55.318984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:36:55.318984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:36:55.319726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:36:55.319726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:36:55.319726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:36:55.319726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:36:55.319726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:36:55.319726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:55.319726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:55.319726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:55.319726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:36:55.319726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:55.319726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:55.319726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:55.319726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:36:55.320361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:55.320361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:36:55.320361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:36:55.320361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:55.320361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:36:55.320361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:55.320361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:36:55.320361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:36:55.320361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:55.320361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:36:55.320361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:55.320361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:36:55.321765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:55.321765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:36:55.321765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:36:55.321765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:36:55.321765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:36:55.321765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:36:55.321765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:55.321765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:36:55.321765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:36:55.321765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:36:55.321765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:36:55.321765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:36:55.321765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:55.321765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:55.321765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:36:55.321765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:36:55.322077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:36:55.322077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:36:55.322077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:36:55.322077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:36:55.322077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:36:55.322077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:55.322077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:55.322077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:55.322077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:36:55.322077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:55.322077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:55.322077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:55.322077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:36:55.322077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:55.322077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:36:55.322077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:36:55.322077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:55.322077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:36:55.322465 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:55.322465 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:36:55.322465 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:36:55.322465 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:55.322465 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:36:55.322465 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:55.322465 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:36:55.322465 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:36:55.322465 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:36:55.322465 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:36:55.322688 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:36:55.322688 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:36:55.322914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:55.323641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:36:55.323641 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:55.323641 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:36:55.323641 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:36:55.323641 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:55.323641 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:36:55.323641 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:55.323641 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:36:55.324351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:55.324351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:36:55.324351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:36:55.324351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:36:55.324351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:55.324351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:36:55.324351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:55.324351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:36:55.324351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:55.324351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:36:55.324351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:55.324351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:36:55.324351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:55.324351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:36:55.324351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:36:55.324668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:55.324668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:36:55.324668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:55.324668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:36:55.324668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:36:55.324668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:55.324668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:36:55.324668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:55.324668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:36:55.325210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:55.325210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:36:55.325210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:55.325210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:36:55.325210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:36:55.325210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:36:55.325210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:36:55.325210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:36:55.325210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:36:55.325210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:55.325210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:55.325479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:55.325479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:36:55.325479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:55.325479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:55.325479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:55.325479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:36:55.325479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:55.325479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:36:55.325479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:36:55.325479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:55.325479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:36:55.325479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:55.325479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:36:55.325479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:36:55.325734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:55.325734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:36:55.325734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:55.325734 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:36:55.326533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:55.326533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:36:55.326533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:36:55.326533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:36:55.326533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:36:55.326533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:36:55.326533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:55.326533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:36:55.326533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:36:55.326533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:36:55.326533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:36:55.326533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:36:55.326533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:55.326533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:55.326533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:36:55.326533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:36:55.326533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:36:55.327009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:36:55.327009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:36:55.327009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:36:55.327009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:55.327009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:55.327009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:55.327009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:36:55.327009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:55.327009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:55.327009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:55.327009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:36:55.327009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:55.327009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:36:55.327009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:36:55.327009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:55.327009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:36:55.327009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:55.327009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:36:55.327009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:36:55.327364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:55.327364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:36:55.327364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:55.327364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:36:55.327364 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:36:55.327364 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:36:55.327364 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:36:55.327364 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:36:55.327568 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:36:55.327777 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:36:55.328147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:55.328748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:36:55.328748 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:55.328748 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:36:55.328748 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:36:55.328748 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:55.328748 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:36:55.328748 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:55.328748 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:36:55.330090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:55.330090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:36:55.330090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:55.330090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:36:55.330090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:36:55.330090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:36:55.330090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:36:55.330090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:55.330090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:55.330090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:55.330090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:36:55.330090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:55.330499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:36:55.330499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:55.330499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:36:55.330499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:55.330499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:36:55.330499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:36:55.330499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:55.330499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:36:55.330499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:55.330499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:36:55.330499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:36:55.330499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:55.330499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:36:55.330862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:55.330862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:36:55.331115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:55.331115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:36:55.331115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:36:55.331115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:36:55.331115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:36:55.331115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:36:55.331115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:55.331115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:36:55.331115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:36:55.331115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:36:55.331115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:36:55.331115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:36:55.331115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:36:55.331115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:36:55.331115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:36:55.331115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:36:55.331640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:36:55.331640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:36:55.331640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:55.331640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:55.331640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:36:55.331640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:36:55.331640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:36:55.331640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:36:55.331640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:36:55.331640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:36:55.331640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:36:55.331640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:36:55.331640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:36:55.331640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:36:55.331640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:36:55.331640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:36:55.331640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:36:55.331640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:36:55.331640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:36:55.332232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:36:55.332232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:36:55.332232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:37:00.070846 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:37:00.071657 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:37:00.071857 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:37:00.073151 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:37:00.103331 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:37:00.103721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:00.104707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:37:00.104707 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:00.104707 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:37:00.104707 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:37:00.104707 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:00.104707 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:37:00.104707 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:00.104707 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:37:00.105607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:00.105607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:37:00.105607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:00.105607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:37:00.105607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:37:00.105607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:37:00.105607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:37:00.105607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:00.105607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:00.105607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:00.105607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:37:00.105607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:00.106302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:00.106302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:00.106302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:37:00.106302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:00.106302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:37:00.106302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:37:00.106302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:00.106302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:37:00.106302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:00.106302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:37:00.106302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:37:00.106302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:00.106302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:37:00.107066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:00.107066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:37:00.107066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:00.107066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:37:00.107066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:37:00.107066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:37:00.107066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:37:00.107066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:37:00.107066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:00.107066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:37:00.107066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:37:00.107066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:37:00.107066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:37:00.107066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:37:00.107066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:00.107923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:00.107923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:37:00.107923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:37:00.107923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:37:00.107923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:37:00.107923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:00.107923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:00.107923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:00.107923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:37:00.107923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:00.107923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:00.107923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:00.107923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:37:00.107923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:00.107923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:37:00.107923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:37:00.107923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:00.107923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:37:00.107923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:00.108988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:37:00.108988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:37:00.108988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:00.108988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:37:00.108988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:00.108988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:37:00.308940 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:37:00.309553 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:37:00.309775 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:37:00.311707 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:37:00.312235 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:37:00.312431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:00.313419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:37:00.313419 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:00.313419 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:37:00.313419 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:37:00.313419 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:00.313419 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:37:00.313419 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:00.313419 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:37:00.315146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:00.315146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:37:00.315146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:00.315146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:37:00.315146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:37:00.315146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:37:00.315146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:37:00.315146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:00.315146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:00.315146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:00.315146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:37:00.315146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:00.315808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:00.315808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:00.315808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:37:00.315808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:00.315808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:37:00.315808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:37:00.315808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:00.315808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:37:00.315808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:00.315808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:37:00.315808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:37:00.315808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:00.315808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:37:00.316445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:00.316445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:37:00.317068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:00.317068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:37:00.317068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:37:00.317068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:37:00.317068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:37:00.317068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:37:00.317068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:00.317068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:37:00.317068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:37:00.317068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:37:00.317068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:37:00.317068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:37:00.317068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:00.317068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:00.317068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:37:00.317068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:37:00.317798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:37:00.317798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:37:00.317798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:00.317798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:00.317798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:00.317798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:37:00.317798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:00.317798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:00.317798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:00.317798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:37:00.317798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:00.317798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:37:00.317798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:37:00.317798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:00.317798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:37:00.317798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:00.317798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:37:00.317798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:37:00.317798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:00.318290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:37:00.318290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:00.318290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:37:00.318290 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:37:00.318290 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:37:00.318290 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:37:00.318290 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:37:00.318290 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:37:00.318666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:00.319369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:37:00.319369 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:00.319369 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:37:00.319369 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:37:00.319369 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:00.319369 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:37:00.319369 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:00.319369 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:37:00.320174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:00.320174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:37:00.320174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:37:00.320174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:37:00.320174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:00.320174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:37:00.320174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:00.320174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:37:00.320174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:00.320174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:37:00.320174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:00.320174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:37:00.320174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:00.320174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:37:00.320174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:37:00.320694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:00.320694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:37:00.320694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:00.320694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:37:00.320694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:37:00.320694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:00.320694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:37:00.320694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:00.320694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:37:00.320979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:00.320979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:37:00.320979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:00.320979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:37:00.320979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:37:00.320979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:37:00.320979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:37:00.320979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:37:00.320979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:37:00.320979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:37:00.320979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:00.320979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:00.321335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:00.321335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:37:00.321335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:00.321335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:00.321335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:00.321335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:37:00.321335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:00.321335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:37:00.321335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:37:00.321335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:00.321335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:37:00.321335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:00.321335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:37:00.321335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:37:00.321768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:00.321768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:37:00.321768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:00.321768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:37:00.322160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:00.322160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:37:00.322160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:37:00.322160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:37:00.322160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:37:00.322160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:37:00.322160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:00.322160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:37:00.322160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:37:00.322160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:37:00.322160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:37:00.322160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:37:00.322160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:00.322160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:00.322160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:37:00.322160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:37:00.322636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:37:00.322636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:37:00.322636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:37:00.322636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:37:00.322636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:37:00.322636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:00.322636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:00.322636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:00.322636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:37:00.322636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:00.322636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:00.322636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:00.322636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:37:00.322636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:00.322636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:37:00.322636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:37:00.322636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:00.322636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:37:00.323141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:00.323141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:37:00.323141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:37:00.323141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:00.323141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:37:00.323141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:00.323141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:37:00.323141 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:37:00.323141 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:37:00.323141 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:37:00.323478 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:37:00.323478 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:37:00.323710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:00.324334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:37:00.324334 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:00.324334 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:37:00.324334 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:37:00.324334 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:00.324334 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:37:00.324334 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:00.324334 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:37:00.325112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:00.325112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:37:00.325112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:37:00.325112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:37:00.325112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:00.325112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:37:00.325112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:00.325112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:37:00.325112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:00.325112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:37:00.325112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:00.325112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:37:00.325112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:00.325112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:37:00.325112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:37:00.325628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:00.325628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:37:00.325628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:00.325628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:37:00.325628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:37:00.325628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:00.325628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:37:00.325628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:00.325628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:37:00.325963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:00.325963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:37:00.325963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:00.325963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:37:00.325963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:37:00.325963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:37:00.325963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:37:00.325963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:37:00.325963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:37:00.325963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:00.325963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:00.326307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:00.326307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:37:00.326307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:00.326307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:00.326307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:00.326307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:37:00.326307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:00.326307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:37:00.326307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:37:00.326307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:00.326307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:37:00.326307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:00.326307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:37:00.326307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:37:00.326714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:00.326714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:37:00.326714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:00.326714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:37:00.327038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:00.327038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:37:00.327038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:37:00.327038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:37:00.327038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:37:00.327038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:37:00.327038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:00.327038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:37:00.327038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:37:00.327038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:37:00.327038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:37:00.327038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:37:00.327038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:00.327038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:00.327038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:37:00.327038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:37:00.327038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:37:00.327768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:37:00.327768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:37:00.327768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:37:00.327768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:00.327768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:00.327768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:00.327768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:37:00.327768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:00.327768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:00.327768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:00.327768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:37:00.327768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:00.327768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:37:00.327768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:37:00.327768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:00.327768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:37:00.327768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:00.327768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:37:00.327768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:37:00.328389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:00.328389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:37:00.328389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:00.328389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:37:00.328389 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:37:00.328389 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:37:00.328389 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:37:00.328389 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:37:00.328721 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:37:00.328721 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:37:00.328721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:00.329212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:37:00.329212 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:00.329212 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:37:00.329212 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:37:00.329212 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:00.329212 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:37:00.329212 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:00.329212 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:37:00.330107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:00.330107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:37:00.330107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:00.330107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:37:00.330107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:37:00.330107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:37:00.330107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:37:00.330107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:00.330107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:00.330107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:00.330107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:37:00.330107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:00.330521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:00.330521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:00.330521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:37:00.330521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:00.330521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:37:00.330521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:37:00.330521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:00.330521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:37:00.330521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:00.330521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:37:00.330521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:37:00.330521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:00.330521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:37:00.330864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:00.330864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:37:00.331145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:00.331145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:37:00.331145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:37:00.331145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:37:00.331145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:37:00.331145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:37:00.331145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:00.331145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:37:00.331145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:37:00.331145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:37:00.331145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:37:00.331145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:37:00.331145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:00.331145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:00.331145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:37:00.331145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:37:00.331614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:37:00.331614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:37:00.331614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:00.331614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:00.331614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:00.331614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:37:00.331614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:00.331614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:00.331614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:00.331614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:37:00.331614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:00.331614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:37:00.331614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:37:00.331614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:00.331614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:37:00.331614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:00.331614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:37:00.331614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:37:00.331614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:00.332154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:37:00.332154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:00.332154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:37:05.071836 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:37:05.072885 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:37:05.072885 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:37:05.074239 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:37:05.110577 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:37:05.111028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:05.111952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:37:05.111952 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:05.111952 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:37:05.111952 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:37:05.111952 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:05.111952 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:37:05.111952 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:05.111952 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:37:05.112746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:05.112746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:37:05.112746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:05.112746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:37:05.112746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:37:05.112746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:37:05.112746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:37:05.112746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:05.112746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:05.112746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:05.112746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:37:05.112746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:05.113348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:05.113348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:05.113348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:37:05.113348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:05.113348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:37:05.113348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:37:05.113348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:05.113348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:37:05.113348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:05.113348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:37:05.113348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:37:05.113348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:05.113348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:37:05.113970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:05.113970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:37:05.113970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:05.113970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:37:05.113970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:37:05.113970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:37:05.113970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:37:05.113970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:37:05.113970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:05.113970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:37:05.113970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:37:05.113970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:37:05.113970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:37:05.113970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:37:05.113970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:05.114797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:05.114797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:37:05.114797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:37:05.114797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:37:05.114797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:37:05.114797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:05.114797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:05.114797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:05.114797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:37:05.114797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:05.114797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:05.114797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:05.114797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:37:05.114797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:05.114797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:37:05.114797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:37:05.114797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:05.114797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:37:05.114797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:05.116852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:37:05.116852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:37:05.116852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:05.116852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:37:05.116852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:05.116852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:37:05.309833 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:37:05.310611 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:37:05.310808 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:37:05.311880 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:37:05.312085 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:37:05.312215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:05.312831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:37:05.312831 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:05.312831 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:37:05.312831 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:37:05.312831 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:05.312831 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:37:05.312831 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:05.312831 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:37:05.314691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:05.314691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:37:05.314691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:05.314691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:37:05.314691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:37:05.314691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:37:05.314691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:37:05.314691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:05.314691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:05.314691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:05.314691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:37:05.314691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:05.315302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:05.315302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:05.315302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:37:05.315302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:05.315302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:37:05.315302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:37:05.315302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:05.315302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:37:05.315302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:05.315302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:37:05.315302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:37:05.315302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:05.315302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:37:05.315894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:05.315894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:37:05.315894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:05.315894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:37:05.315894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:37:05.315894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:37:05.315894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:37:05.315894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:37:05.315894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:05.315894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:37:05.315894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:37:05.315894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:37:05.315894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:37:05.315894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:37:05.315894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:05.316625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:05.316625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:37:05.316625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:37:05.316625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:37:05.316625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:37:05.316625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:05.316625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:05.316625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:05.316625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:37:05.316625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:05.316625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:05.316625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:05.316625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:37:05.316625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:05.316625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:37:05.316625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:37:05.316625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:05.316625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:37:05.316625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:05.317503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:37:05.317503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:37:05.317503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:05.317503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:37:05.317503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:05.317503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:37:05.317771 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:37:05.317980 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:37:05.318063 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:37:05.318228 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:37:05.318538 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:37:05.318658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:05.319094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:37:05.319094 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:05.319094 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:37:05.319094 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:37:05.319094 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:05.319094 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:37:05.319094 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:05.319094 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:37:05.322494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:05.322494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:37:05.322494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:37:05.322494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:37:05.322494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:05.322494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:37:05.322494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:05.322494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:37:05.322494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:05.322494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:37:05.322494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:05.322494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:37:05.322494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:05.322494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:37:05.322494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:37:05.322979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:05.322979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:37:05.322979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:05.322979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:37:05.322979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:37:05.322979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:05.322979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:37:05.322979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:05.322979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:37:05.322979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:05.322979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:37:05.322979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:05.322979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:37:05.322979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:37:05.323386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:37:05.323386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:37:05.323386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:37:05.323386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:37:05.323386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:37:05.323386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:05.323386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:05.323386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:05.323386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:37:05.323386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:05.323386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:05.323386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:05.323386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:37:05.323769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:05.323769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:37:05.323769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:37:05.323769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:05.323769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:37:05.323769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:05.323769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:37:05.323769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:37:05.323769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:05.323769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:37:05.323769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:05.323769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:37:05.324079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:05.324079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:37:05.324079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:37:05.324079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:37:05.324079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:37:05.324079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:37:05.324079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:05.324079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:37:05.324079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:37:05.324079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:37:05.324079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:37:05.324079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:37:05.324079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:05.324079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:05.324079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:37:05.324079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:37:05.324494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:37:05.324494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:37:05.324494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:37:05.324494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:37:05.324494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:37:05.324494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:05.324494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:05.324494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:05.324494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:37:05.324494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:05.324494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:05.324494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:05.324494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:37:05.324494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:05.324494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:37:05.324494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:37:05.324494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:05.324494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:37:05.324960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:05.324960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:37:05.324960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:37:05.324960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:05.324960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:37:05.324960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:05.324960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:37:05.324960 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:37:05.324960 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:37:05.324960 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:37:05.325262 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:37:05.325262 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:37:05.325262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:05.325262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:37:05.325262 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:05.325262 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:37:05.325262 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:37:05.325262 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:05.325262 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:37:05.325262 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:05.325262 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:37:05.326665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:05.326665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:37:05.326665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:37:05.326665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:37:05.326665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:05.326665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:37:05.326665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:05.326665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:37:05.326665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:05.326665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:37:05.326665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:05.326665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:37:05.326665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:05.326665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:37:05.326665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:37:05.327084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:05.327084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:37:05.327084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:05.327084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:37:05.327084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:37:05.327084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:05.327084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:37:05.327084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:05.327084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:37:05.327084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:05.327084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:37:05.327084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:05.327084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:37:05.327084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:37:05.327543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:37:05.327543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:37:05.327543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:37:05.327543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:37:05.327543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:05.327543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:05.327543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:05.327543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:37:05.327543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:05.327543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:05.327543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:05.327543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:37:05.327543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:05.327899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:37:05.327899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:37:05.327899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:05.327899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:37:05.327899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:05.327899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:37:05.327899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:37:05.327899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:05.327899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:37:05.327899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:05.327899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:37:05.327899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:05.328217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:37:05.328217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:37:05.328217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:37:05.328217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:37:05.328217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:37:05.328217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:05.328217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:37:05.328217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:37:05.328217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:37:05.328217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:37:05.328217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:37:05.328217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:05.328217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:05.328217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:37:05.328217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:37:05.328217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:37:05.328217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:37:05.328217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:37:05.328217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:37:05.328763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:05.328763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:05.328763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:05.328763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:37:05.328763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:05.328763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:05.328763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:05.328763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:37:05.328763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:05.328763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:37:05.328763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:37:05.328763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:05.328763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:37:05.328763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:05.328763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:37:05.328763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:37:05.328763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:05.328763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:37:05.328763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:05.328763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:37:05.329260 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:37:05.329260 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:37:05.329260 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:37:05.329260 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:37:05.329260 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:37:05.329260 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:37:05.329486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:05.329486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:37:05.329486 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:05.329486 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:37:05.329486 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:37:05.329486 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:05.329486 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:37:05.329486 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:05.329486 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:37:05.330123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:05.330123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:37:05.330123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:05.330123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:37:05.330123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:37:05.330123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:37:05.330123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:37:05.330123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:05.330123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:05.330123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:05.330123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:37:05.330123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:05.330549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:05.330549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:05.330549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:37:05.330549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:05.330549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:37:05.330549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:37:05.330549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:05.330549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:37:05.330549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:05.330549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:37:05.330549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:37:05.330549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:05.330549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:37:05.330897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:05.330897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:37:05.331022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:05.331022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:37:05.331022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:37:05.331022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:37:05.331022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:37:05.331022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:37:05.331022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:05.331022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:37:05.331022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:37:05.331022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:37:05.331022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:37:05.331022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:37:05.331022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:05.331022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:05.331022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:37:05.331022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:37:05.331370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:37:05.331370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:37:05.331370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:05.331370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:05.331370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:05.331370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:37:05.331370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:05.331370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:05.331370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:05.331370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:37:05.331370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:05.331370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:37:05.331370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:37:05.331370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:05.331370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:37:05.331370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:05.331370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:37:05.331370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:37:05.331370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:05.331804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:37:05.331804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:05.331804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:37:10.072752 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:37:10.073924 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:37:10.074073 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:37:10.075430 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:37:10.111004 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:37:10.111246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:10.111870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:37:10.111870 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:10.111870 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:37:10.111870 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:37:10.111870 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:10.111870 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:37:10.111870 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:10.111870 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:37:10.112622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:10.112622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:37:10.112622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:10.112622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:37:10.112622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:37:10.112622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:37:10.112622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:37:10.112622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:10.112622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:10.112622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:10.112622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:37:10.112622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:10.113396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:10.113396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:10.113396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:37:10.113396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:10.113396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:37:10.113396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:37:10.113396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:10.113396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:37:10.113396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:10.113396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:37:10.113396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:37:10.113396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:10.113396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:37:10.114260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:10.114260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:37:10.114260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:10.114260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:37:10.114260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:37:10.114260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:37:10.114260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:37:10.114260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:37:10.114260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:10.114260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:37:10.114260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:37:10.114260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:37:10.114260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:37:10.114260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:37:10.114260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:10.115183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:10.115183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:37:10.115183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:37:10.115183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:37:10.115183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:37:10.115183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:10.115183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:10.115183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:10.115183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:37:10.115183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:10.115183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:10.115183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:10.115183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:37:10.115183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:10.115183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:37:10.115183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:37:10.115183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:10.115183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:37:10.115183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:10.116183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:37:10.116183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:37:10.116183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:10.116183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:37:10.116183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:10.116183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:37:10.310624 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:37:10.311337 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:37:10.311635 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:37:10.313027 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:37:10.313520 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:37:10.313789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:10.314706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:37:10.314706 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:10.314706 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:37:10.314706 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:37:10.314706 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:10.314706 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:37:10.314706 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:10.314706 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:37:10.316161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:10.316161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:37:10.316161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:10.316161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:37:10.316161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:37:10.316161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:37:10.316161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:37:10.316161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:10.316161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:10.316161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:10.316161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:37:10.316161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:10.316846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:10.316846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:10.316846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:37:10.316846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:10.316846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:37:10.316846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:37:10.316846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:10.316846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:37:10.316846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:10.316846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:37:10.316846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:37:10.316846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:10.316846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:37:10.317464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:10.317464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:37:10.318681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:10.318681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:37:10.318681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:37:10.318681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:37:10.318681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:37:10.318681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:37:10.318681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:10.318681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:37:10.318681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:37:10.318681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:37:10.318681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:37:10.318681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:37:10.318681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:10.318681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:10.318681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:37:10.318681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:37:10.319893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:37:10.319893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:37:10.319893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:10.319893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:10.319893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:10.319893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:37:10.319893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:10.319893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:10.319893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:10.319893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:37:10.319893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:10.319893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:37:10.319893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:37:10.319893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:10.319893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:37:10.319893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:10.319893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:37:10.319893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:37:10.319893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:10.320888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:37:10.320888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:10.320888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:37:10.320888 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:37:10.320888 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:37:10.320888 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:37:10.320888 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:37:10.320888 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:37:10.322020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:10.322020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:37:10.322020 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:10.322020 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:37:10.322020 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:37:10.322020 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:10.322020 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:37:10.322020 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:10.322020 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:37:10.322336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:10.322336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:37:10.322336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:37:10.322336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:37:10.322336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:10.322336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:37:10.322336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:10.322336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:37:10.322336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:10.322336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:37:10.322336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:10.322336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:37:10.322336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:10.322336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:37:10.322336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:37:10.322882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:10.322882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:37:10.322882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:10.322882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:37:10.322882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:37:10.322882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:10.322882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:37:10.322882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:10.322882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:37:10.323183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:10.323183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:37:10.323183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:10.323183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:37:10.323183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:37:10.323183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:37:10.323183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:37:10.323183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:37:10.323183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:37:10.323183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:37:10.323183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:10.323183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:10.323542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:10.323542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:37:10.323542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:10.323542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:10.323542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:10.323542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:37:10.323542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:10.323542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:37:10.323542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:37:10.323542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:10.323542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:37:10.323542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:10.323542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:37:10.323542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:37:10.324076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:10.324076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:37:10.324076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:10.324076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:37:10.324076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:10.324076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:37:10.324076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:37:10.324076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:37:10.324076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:37:10.324076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:37:10.324076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:10.324076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:37:10.324076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:37:10.324076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:37:10.324494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:37:10.324494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:37:10.324494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:10.324494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:10.324494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:37:10.324494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:37:10.324494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:37:10.324494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:37:10.324494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:37:10.324494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:37:10.324494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:37:10.324494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:10.324494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:10.324494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:10.324494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:37:10.324494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:10.324494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:10.324494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:10.324494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:37:10.325147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:10.325147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:37:10.325147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:37:10.325147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:10.325147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:37:10.325147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:10.325147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:37:10.325147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:37:10.325147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:10.325147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:37:10.325147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:10.325147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:37:10.325147 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:37:10.325147 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:37:10.325623 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:37:10.325623 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:37:10.325623 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:37:10.325835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:10.326329 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:37:10.326329 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:10.326329 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:37:10.326329 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:37:10.326329 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:10.326329 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:37:10.326329 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:10.326329 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:37:10.327022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:10.327022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:37:10.327022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:37:10.327022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:37:10.327022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:10.327022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:37:10.327022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:10.327022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:37:10.327022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:10.327022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:37:10.327022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:10.327022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:37:10.327022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:10.327022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:37:10.327022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:37:10.327498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:10.327498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:37:10.327498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:10.327498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:37:10.327498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:37:10.327498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:10.327498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:37:10.327498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:10.327498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:37:10.327869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:10.327869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:37:10.327869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:10.327869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:37:10.327869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:37:10.327869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:37:10.327869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:37:10.327869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:37:10.327869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:37:10.327869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:10.327869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:10.328324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:10.328324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:37:10.328324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:10.328324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:10.328324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:10.328324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:37:10.328324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:10.328324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:37:10.328324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:37:10.328324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:10.328324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:37:10.328324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:10.328324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:37:10.328324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:37:10.328888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:10.328888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:37:10.328888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:10.328888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:37:10.329045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:10.329045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:37:10.329045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:37:10.329045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:37:10.329045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:37:10.329045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:37:10.329045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:10.329045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:37:10.329045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:37:10.329045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:37:10.329045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:37:10.329045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:37:10.329045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:10.329045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:10.329045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:37:10.329045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:37:10.329045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:37:10.329704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:37:10.329704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:37:10.329704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:37:10.329704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:10.329704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:10.329704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:10.329704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:37:10.329704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:10.329704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:10.329704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:10.329704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:37:10.329704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:10.329704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:37:10.329704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:37:10.329704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:10.329704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:37:10.329704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:10.329704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:37:10.329704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:37:10.330367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:10.330367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:37:10.330367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:10.330367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:37:10.330367 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:37:10.330367 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:37:10.330367 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:37:10.330367 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:37:10.330581 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:37:10.330581 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:37:10.330581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:10.330951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:37:10.330951 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:10.330951 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:37:10.330951 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:37:10.330951 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:10.330951 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:37:10.330951 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:10.330951 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:37:10.331784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:10.331784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:37:10.331784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:10.331784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:37:10.331784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:37:10.331784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:37:10.331784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:37:10.331784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:10.331784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:10.331784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:10.331784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:37:10.331784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:10.332075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:10.332075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:10.332075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:37:10.332075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:10.332075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:37:10.332075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:37:10.332075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:10.332075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:37:10.332075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:10.332075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:37:10.332075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:37:10.332075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:10.332075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:37:10.332338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:10.332338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:37:10.332775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:10.332775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:37:10.332775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:37:10.332775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:37:10.332775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:37:10.332775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:37:10.332775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:10.332775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:37:10.332775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:37:10.332775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:37:10.332775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:37:10.332775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:37:10.332775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:10.332775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:10.332775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:37:10.332775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:37:10.333148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:37:10.333148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:37:10.333148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:10.333148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:10.333148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:10.333148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:37:10.333148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:10.333148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:10.333148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:10.333148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:37:10.333148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:10.333148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:37:10.333148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:37:10.333148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:10.333148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:37:10.333148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:10.333148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:37:10.333148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:37:10.333148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:10.333591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:37:10.333591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:10.333591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:37:15.073841 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:37:15.074676 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:37:15.074676 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:37:15.075852 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:37:15.104365 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:37:15.104652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:15.105272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:37:15.105272 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:15.105272 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:37:15.105272 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:37:15.105272 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:15.105272 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:37:15.105272 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:15.105272 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:37:15.106005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.031 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:15.106005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:37:15.106005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:15.106005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:37:15.106005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:37:15.106005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:37:15.106005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:37:15.106005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:15.106005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:15.106005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:15.106005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:37:15.106005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:15.106676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:15.106676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:15.106676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:37:15.106676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:15.106676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:37:15.106676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:37:15.106676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:15.106676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:37:15.106676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:15.106676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:37:15.106676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:37:15.106676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:15.106676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:37:15.107358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:15.107358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:37:15.107358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:15.107358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:37:15.107358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:37:15.107358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:37:15.107358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:37:15.107358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:37:15.107358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:15.107358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:37:15.107358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:37:15.107358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:37:15.107358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:37:15.107358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:37:15.107358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:15.108086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:15.108086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:37:15.108086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:37:15.108086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:37:15.108086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:37:15.108086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:15.108086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:15.108086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:15.108086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:37:15.108086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:15.108086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:15.108086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:15.108086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:37:15.108086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:15.108086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:37:15.108086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:37:15.108086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:15.108086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:37:15.108086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:15.109033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:37:15.109033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:37:15.109033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:15.109033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:37:15.109033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:15.109033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:37:15.311495 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:37:15.312172 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:37:15.312172 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:37:15.313117 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:37:15.313309 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:37:15.313518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:15.314092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:37:15.314092 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:15.314092 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:37:15.314092 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:37:15.314092 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:15.314092 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:37:15.314092 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:15.314092 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:37:15.315540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:15.315540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:37:15.315540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:15.315540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:37:15.315540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:37:15.315540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:37:15.315540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:37:15.315540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:15.315540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:15.315540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:15.315540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:37:15.315540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:15.316302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:15.316302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:15.316302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:37:15.316302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:15.316302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:37:15.316302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:37:15.316302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:15.316302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:37:15.316302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:15.316302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:37:15.316302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:37:15.316302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:15.316302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:37:15.317005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:15.317005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:37:15.317005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:15.317005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:37:15.317005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:37:15.317005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:37:15.317005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:37:15.317005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:37:15.317005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:15.317005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:37:15.317005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:37:15.317005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:37:15.317005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:37:15.317005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:37:15.317005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:15.317751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:15.317751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:37:15.317751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:37:15.317751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:37:15.317751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:37:15.317751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:15.317751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:15.317751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:15.317751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:37:15.317751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:15.317751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:15.317751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:15.317751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:37:15.317751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:15.317751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:37:15.317751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:37:15.317751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:15.317751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:37:15.317751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:15.318761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:37:15.318761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:37:15.318761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:15.318761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:37:15.318761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:15.318761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:37:15.319701 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:37:15.320236 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:37:15.320349 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:37:15.320498 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:37:15.322717 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:37:15.322917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:15.323427 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:37:15.323427 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:15.323427 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:37:15.323427 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:37:15.323427 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:15.323427 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:37:15.323427 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:15.323427 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:37:15.324251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:15.324251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:37:15.324251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:37:15.324251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:37:15.324251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:15.324251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:37:15.324251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:15.324251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:37:15.324251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:15.324251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:37:15.324251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:15.324251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:37:15.324251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:15.324251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:37:15.324251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:37:15.324576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:15.324576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:37:15.324576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:15.324576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:37:15.324576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:37:15.324576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:15.324576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:37:15.324576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:15.324576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:37:15.325052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:15.325052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:37:15.325052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:15.325052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:37:15.325052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:37:15.325052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:37:15.325052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:37:15.325052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:37:15.325052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:37:15.325052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:37:15.325052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:15.325052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:15.325339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:15.325339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:37:15.325339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:15.325339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:15.325339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:15.325339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:37:15.325339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:15.325339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:37:15.325339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:37:15.325339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:15.325339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:37:15.325339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:15.325339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:37:15.325339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:37:15.325710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:15.325710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:37:15.325710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:15.325710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:37:15.326209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:15.326209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:37:15.326209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:37:15.326209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:37:15.326209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:37:15.326209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:37:15.326209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:15.326209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:37:15.326209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:37:15.326209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:37:15.326209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:37:15.326209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:37:15.326209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:15.326209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:15.326209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:37:15.326209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:37:15.326537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:37:15.326537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:37:15.326537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:37:15.326537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:37:15.326537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:37:15.326537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:15.326537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:15.326537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:15.326537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:37:15.326537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:15.326537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:15.326537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:15.326537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:37:15.326537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:15.326537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:37:15.326537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:37:15.326537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:15.326537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:37:15.326894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:15.326894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:37:15.326894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:37:15.326894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:15.326894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:37:15.326894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:15.326894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:37:15.327172 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:37:15.327399 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:37:15.327540 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:37:15.327668 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:37:15.328074 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:37:15.328305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:15.328794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:37:15.328794 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:15.328794 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:37:15.328794 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:37:15.328794 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:15.328794 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:37:15.328794 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:15.328794 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:37:15.329551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:15.329551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:37:15.329551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:37:15.329551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:37:15.329551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:15.329551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:37:15.329551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:15.329551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:37:15.329551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:15.329551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:37:15.329551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:15.329551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:37:15.329551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:15.329551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:37:15.329551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:37:15.329972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:15.329972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:37:15.329972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:15.329972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:37:15.329972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:37:15.329972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:15.329972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:37:15.329972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:15.329972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:37:15.330437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:15.330437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:37:15.330437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:15.330437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:37:15.330437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:37:15.330437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:37:15.330437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:37:15.330437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:37:15.330437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:37:15.330437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:15.330437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:15.330747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:15.330747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:37:15.330747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:15.330747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:15.330747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:15.330747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:37:15.330747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:15.330747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:37:15.330747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:37:15.330747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:15.330747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:37:15.330747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:15.330747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:37:15.330747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:37:15.331253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:15.331253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:37:15.331253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:15.331253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:37:15.331795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:15.331795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:37:15.331795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:37:15.331795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:37:15.331795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:37:15.331795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:37:15.331795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:15.331795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:37:15.331795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:37:15.331795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:37:15.331795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:37:15.331795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:37:15.331795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:15.331795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:15.331795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:37:15.331795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:37:15.331795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:37:15.332511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:37:15.332511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:37:15.332511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:37:15.332511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:15.332511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:15.332511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:15.332511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:37:15.332511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:15.332511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:15.332511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:15.332511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:37:15.332511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:15.332511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:37:15.332511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:37:15.332511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:15.332511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:37:15.332511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:15.332511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:37:15.332511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:37:15.333018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:15.333018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:37:15.333018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:15.333018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:37:15.333018 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:37:15.333018 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:37:15.333018 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:37:15.333018 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:37:15.333611 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:37:15.333766 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:37:15.333876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:15.334311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:37:15.334311 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:15.334311 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:37:15.334311 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:37:15.334311 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:15.334311 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:37:15.334311 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:15.334311 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:37:15.335128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:15.335128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:37:15.335128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:15.335128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:37:15.335128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:37:15.335128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:37:15.335128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:37:15.335128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:15.335128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:15.335128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:15.335128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:37:15.335128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:15.335509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:15.335509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:15.335509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:37:15.335509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:15.335509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:37:15.335509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:37:15.335509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:15.335509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:37:15.335509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:15.335509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:37:15.335509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:37:15.335509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:15.335509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:37:15.335924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:15.335924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:37:15.336260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:15.336260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:37:15.336260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:37:15.336260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:37:15.336260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:37:15.336260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:37:15.336260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:15.336260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:37:15.336260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:37:15.336260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:37:15.336260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:37:15.336260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:37:15.336260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:15.336260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:15.336260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:37:15.336260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:37:15.336801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:37:15.336801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:37:15.336801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:15.336801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:15.336801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:15.336801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:37:15.336801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:15.336801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:15.336801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:15.336801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:37:15.336801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:15.336801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:37:15.336801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:37:15.336801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:15.336801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:37:15.336801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:15.336801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:37:15.336801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:37:15.336801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:15.337269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:37:15.337269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:15.337269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:37:20.074655 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:37:20.075525 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:37:20.075525 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:37:20.076528 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:37:20.103051 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:37:20.103433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:20.104284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:37:20.104284 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:20.104284 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:37:20.104284 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:37:20.104284 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:20.104284 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:37:20.104284 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:20.104284 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:37:20.105047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.029 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:20.105047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:37:20.105047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:20.105047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:37:20.105047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:37:20.105047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:37:20.105047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:37:20.105047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:20.105047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:20.105047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:20.105047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:37:20.105047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:20.105723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:20.105723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:20.105723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:37:20.105723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:20.105723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:37:20.105723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:37:20.105723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:20.105723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:37:20.105723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:20.105723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:37:20.105723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:37:20.105723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:20.105723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:37:20.106448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:20.106448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:37:20.106448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:20.106448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:37:20.106448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:37:20.106448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:37:20.106448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:37:20.106448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:37:20.106448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:20.106448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:37:20.106448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:37:20.106448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:37:20.106448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:37:20.106448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:37:20.106448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:20.107212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:20.107212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:37:20.107212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:37:20.107212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:37:20.107212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:37:20.107212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:20.107212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:20.107212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:20.107212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:37:20.107212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:20.107212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:20.107212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:20.107212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:37:20.107212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:20.107212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:37:20.107212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:37:20.107212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:20.107212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:37:20.107212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:20.108274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:37:20.108274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:37:20.108274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:20.108274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:37:20.108274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:20.108274 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:37:20.312103 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:37:20.312762 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:37:20.312923 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:37:20.314063 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:37:20.314432 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:37:20.314705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:20.315568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:37:20.315568 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:20.315568 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:37:20.315568 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:37:20.315568 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:20.315568 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:37:20.315568 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:20.315568 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:37:20.317038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:20.317038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:37:20.317038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:20.317038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:37:20.317038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:37:20.317038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:37:20.317038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:37:20.317038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:20.317038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:20.317038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:20.317038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:37:20.317038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:20.317725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:20.317725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:20.317725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:37:20.317725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:20.317725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:37:20.317725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:37:20.317725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:20.317725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:37:20.317725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:20.317725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:37:20.317725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:37:20.317725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:20.317725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:37:20.318314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:20.318314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:37:20.318675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:20.318675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:37:20.318675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:37:20.318675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:37:20.318675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:37:20.318675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:37:20.318675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:20.318675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:37:20.318675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:37:20.318675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:37:20.318675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:37:20.318675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:37:20.318675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:20.318675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:20.318675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:37:20.318675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:37:20.319621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:37:20.319621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:37:20.319621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:20.319621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:20.319621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:20.319621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:37:20.319621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:20.319621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:20.319621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:20.319621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:37:20.319621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:20.319621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:37:20.319621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:37:20.319621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:20.319621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:37:20.319621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:20.319621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:37:20.319621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:37:20.319621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:20.320482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:37:20.320482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:20.320482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:37:20.320482 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:37:20.320797 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:37:20.320954 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:37:20.321134 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:37:20.321616 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:37:20.321898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:20.322689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:37:20.322689 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:20.322689 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:37:20.322689 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:37:20.322689 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:20.322689 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:37:20.322689 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:20.322689 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:37:20.323820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:20.323820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:37:20.323820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:37:20.323820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:37:20.323820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:20.323820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:37:20.323820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:20.323820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:37:20.323820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:20.323820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:37:20.323820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:20.323820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:37:20.323820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:20.323820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:37:20.323820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:37:20.324292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:20.324292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:37:20.324292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:20.324292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:37:20.324292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:37:20.324292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:20.324292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:37:20.324292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:20.324292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:37:20.325136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:20.325136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:37:20.325136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:20.325136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:37:20.325136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:37:20.325136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:37:20.325136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:37:20.325136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:37:20.325136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:37:20.325136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:37:20.325136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:20.325136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:20.325543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:20.325543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:37:20.325543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:20.325543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:20.325543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:20.325543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:37:20.325543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:20.325543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:37:20.325543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:37:20.325543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:20.325543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:37:20.325543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:20.325543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:37:20.325543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:37:20.325929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:20.325929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:37:20.325929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:20.325929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:37:20.326719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:20.326719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:37:20.326719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:37:20.326719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:37:20.326719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:37:20.326719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:37:20.326719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:20.326719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:37:20.326719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:37:20.326719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:37:20.326719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:37:20.326719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:37:20.326719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:20.326719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:20.326719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:37:20.326719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:37:20.327167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:37:20.327167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:37:20.327167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:37:20.327167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:37:20.327167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:37:20.327167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:20.327167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:20.327167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:20.327167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:37:20.327167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:20.327167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:20.327167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:20.327167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:37:20.327167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:20.327167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:37:20.327167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:37:20.327167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:20.327167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:37:20.327721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:20.327721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:37:20.327721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:37:20.327721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:20.327721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:37:20.327721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:20.327721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:37:20.327721 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:37:20.327947 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:37:20.327947 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:37:20.328031 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:37:20.328515 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:37:20.328848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:20.329516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:37:20.329516 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:20.329516 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:37:20.329516 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:37:20.329516 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:20.329516 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:37:20.329516 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:20.329516 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:37:20.330200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:20.330200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:37:20.330200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:37:20.330200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:37:20.330200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:20.330200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:37:20.330200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:20.330200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:37:20.330200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:20.330200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:37:20.330200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:20.330200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:37:20.330200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:20.330200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:37:20.330200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:37:20.330605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:20.330605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:37:20.330605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:20.330605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:37:20.330605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:37:20.330605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:20.330605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:37:20.330605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:20.330605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:37:20.331062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:20.331062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:37:20.331062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:20.331062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:37:20.331062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:37:20.331062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:37:20.331062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:37:20.331062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:37:20.331062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:37:20.331062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:20.331062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:20.331380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:20.331380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:37:20.331380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:20.331380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:20.331380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:20.331380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:37:20.331380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:20.331380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:37:20.331380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:37:20.331380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:20.331380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:37:20.331380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:20.331380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:37:20.331380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:37:20.331767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:20.331767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:37:20.331767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:20.331767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:37:20.332076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:20.332076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:37:20.332076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:37:20.332076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:37:20.332076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:37:20.332076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:37:20.332076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:20.332076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:37:20.332076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:37:20.332076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:37:20.332076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:37:20.332076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:37:20.332076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:20.332076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:20.332076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:37:20.332076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:37:20.332076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:37:20.332574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:37:20.332574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:37:20.332574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:37:20.332574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:20.332574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:20.332574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:20.332574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:37:20.332574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:20.332574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:20.332574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:20.332574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:37:20.332574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:20.332574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:37:20.332574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:37:20.332574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:20.332574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:37:20.332574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:20.332574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:37:20.332574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:37:20.333072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:20.333072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:37:20.333072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:20.333072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:37:20.333072 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:37:20.333072 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:37:20.333072 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:37:20.333072 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:37:20.333722 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:37:20.333833 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:37:20.334035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:20.334867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:37:20.334867 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:20.334867 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:37:20.334867 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:37:20.334867 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:20.334867 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:37:20.334867 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:20.334867 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:37:20.335662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:20.335662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:37:20.335662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:20.335662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:37:20.335662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:37:20.335662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:37:20.335662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:37:20.335662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:20.335662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:20.335662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:20.335662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:37:20.335662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:20.336035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:20.336035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:20.336035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:37:20.336035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:20.336035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:37:20.336035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:37:20.336035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:20.336035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:37:20.336035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:20.336035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:37:20.336035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:37:20.336035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:20.336035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:37:20.336363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:20.336363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:37:20.336592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:20.336592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:37:20.336592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:37:20.336592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:37:20.336592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:37:20.336592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:37:20.336592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:20.336592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:37:20.336592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:37:20.336592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:37:20.336592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:37:20.336592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:37:20.336592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:20.336592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:20.336592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:37:20.336592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:37:20.337043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:37:20.337043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:37:20.337043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:20.337043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:20.337043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:20.337043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:37:20.337043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:20.337043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:20.337043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:20.337043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:37:20.337043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:20.337043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:37:20.337043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:37:20.337043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:20.337043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:37:20.337043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:20.337043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:37:20.337043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:37:20.337043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:20.337573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:37:20.337573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:20.337573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:37:25.075248 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:37:25.076146 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:37:25.076146 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:37:25.077249 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:37:25.079148 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG dbcounter [-] [105581] Writing DB stats neutron:SELECT=240 {{(pid=105581) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 14:37:25.112281 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:37:25.112766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:25.113711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:37:25.113711 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:25.113711 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:37:25.113711 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:37:25.113711 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:25.113711 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:37:25.113711 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:25.113711 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:37:25.114583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:25.114583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:37:25.114583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:25.114583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:37:25.114583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:37:25.114583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:37:25.114583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:37:25.114583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:25.114583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:25.114583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:25.114583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:37:25.114583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:25.115250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:25.115250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:25.115250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:37:25.115250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:25.115250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:37:25.115250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:37:25.115250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:25.115250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:37:25.115250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:25.115250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:37:25.115250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:37:25.115250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:25.115250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:37:25.115957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:25.115957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:37:25.115957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:25.115957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:37:25.115957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:37:25.115957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:37:25.115957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:37:25.115957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:37:25.115957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:25.115957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:37:25.115957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:37:25.115957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:37:25.115957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:37:25.115957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:37:25.115957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:25.116880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:25.116880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:37:25.116880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:37:25.116880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:37:25.116880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:37:25.116880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:25.116880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:25.116880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:25.116880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:37:25.116880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:25.116880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:25.116880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:25.116880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:37:25.116880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:25.116880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:37:25.116880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:37:25.116880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:25.116880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:37:25.116880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:25.118153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:37:25.118153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:37:25.118153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:25.118153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:37:25.118153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:25.118153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:37:25.312643 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:37:25.313279 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:37:25.313500 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:37:25.314501 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:37:25.314693 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:37:25.314789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:25.315506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:37:25.315506 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:25.315506 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:37:25.315506 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:37:25.315506 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:25.315506 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:37:25.315506 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:25.315506 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:37:25.317624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:25.317624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:37:25.317624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:25.317624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:37:25.317624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:37:25.317624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:37:25.317624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:37:25.317624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:25.317624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:25.317624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:25.317624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:37:25.317624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:25.318245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:25.318245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:25.318245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:37:25.318245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:25.318245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:37:25.318245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:37:25.318245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:25.318245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:37:25.318245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:25.318245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:37:25.318245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:37:25.318245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:25.318245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:37:25.318822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:25.318822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:37:25.319050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:25.319050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:37:25.319050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:37:25.319050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:37:25.319050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:37:25.319050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:37:25.319050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:25.319050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:37:25.319050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:37:25.319050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:37:25.319050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:37:25.319050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:37:25.319050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:25.319050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:25.319050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:37:25.319050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:37:25.319753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:37:25.319753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:37:25.319753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:25.319753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:25.319753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:25.319753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:37:25.319753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:25.319753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:25.319753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:25.319753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:37:25.319753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:25.319753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:37:25.319753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:37:25.319753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:25.319753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:37:25.319753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:25.319753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:37:25.319753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:37:25.319753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:25.320588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:37:25.320588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:25.320588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:37:25.320831 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:37:25.321071 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:37:25.321154 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:37:25.321221 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:37:25.321493 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:37:25.321635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:25.322011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:37:25.322011 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:25.322011 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:37:25.322011 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:37:25.322011 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:25.322011 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:37:25.322011 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:25.322011 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:37:25.322669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:25.322669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:37:25.322669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:37:25.322669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:37:25.322669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:25.322669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:37:25.322669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:25.322669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:37:25.322669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:25.322669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:37:25.322669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:25.322669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:37:25.322669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:25.322669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:37:25.322669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:37:25.323147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:25.323147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:37:25.323147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:25.323147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:37:25.323147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:37:25.323147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:25.323147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:37:25.323147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:25.323147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:37:25.323147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:25.323147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:37:25.323147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:25.323147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:37:25.323147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:37:25.323643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:37:25.323643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:37:25.323643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:37:25.323643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:37:25.323643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:37:25.323643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:25.323643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:25.323643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:25.323643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:37:25.323643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:25.323643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:25.323643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:25.323643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:37:25.324023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:25.324023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:37:25.324023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:37:25.324023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:25.324023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:37:25.324023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:25.324023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:37:25.324023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:37:25.324023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:25.324023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:37:25.324023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:25.324023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:37:25.324347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:25.324347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:37:25.324347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:37:25.324347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:37:25.324347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:37:25.324347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:37:25.324347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:25.324347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:37:25.324347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:37:25.324347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:37:25.324347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:37:25.324347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:37:25.324347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:25.324347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:25.324347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:37:25.324347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:37:25.324811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:37:25.324811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:37:25.324811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:37:25.324811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:37:25.324811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:37:25.324811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:25.324811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:25.324811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:25.324811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:37:25.324811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:25.324811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:25.324811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:25.324811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:37:25.324811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:25.324811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:37:25.324811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:37:25.324811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:25.324811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:37:25.325444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:25.325444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:37:25.325444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:37:25.325444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:25.325444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:37:25.325444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:25.325444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:37:25.327989 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:37:25.328235 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:37:25.328343 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:37:25.328453 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:37:25.328748 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:37:25.329004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:25.329371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:37:25.329371 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:25.329371 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:37:25.329371 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:37:25.329371 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:25.329371 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:37:25.329371 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:25.329371 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:37:25.330140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:25.330140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:37:25.330140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:37:25.330140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:37:25.330140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:25.330140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:37:25.330140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:25.330140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:37:25.330140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:25.330140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:37:25.330140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:25.330140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:37:25.330140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:25.330140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:37:25.330140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:37:25.330559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:25.330559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:37:25.330559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:25.330559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:37:25.330559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:37:25.330559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:25.330559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:37:25.330559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:25.330559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:37:25.331071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:25.331071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:37:25.331071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:25.331071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:37:25.331071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:37:25.331071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:37:25.331071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:37:25.331071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:37:25.331071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:37:25.331071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:25.331071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:25.331429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:25.331429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:37:25.331429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:25.331429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:25.331429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:25.331429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:37:25.331429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:25.331429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:37:25.331429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:37:25.331429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:25.331429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:37:25.331429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:25.331429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:37:25.331429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:37:25.331800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:25.331800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:37:25.331800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:25.331800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:37:25.332100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:25.332100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:37:25.332100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:37:25.332100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:37:25.332100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:37:25.332100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:37:25.332100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:25.332100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:37:25.332100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:37:25.332100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:37:25.332100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:37:25.332100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:37:25.332100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:25.332100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:25.332100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:37:25.332100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:37:25.332100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:37:25.332623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:37:25.332623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:37:25.332623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:37:25.332623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:25.332623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:25.332623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:25.332623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:37:25.332623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:25.332623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:25.332623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:25.332623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:37:25.332623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:25.332623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:37:25.332623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:37:25.332623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:25.332623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:37:25.332623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:25.332623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:37:25.332623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:37:25.333152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:25.333152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:37:25.333152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:25.333152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:37:25.333152 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:37:25.333385 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:37:25.333385 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:37:25.333482 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:37:25.333757 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:37:25.333917 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:37:25.334024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:25.334420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:37:25.334420 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:25.334420 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:37:25.334420 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:37:25.334420 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:25.334420 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:37:25.334420 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:25.334420 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:37:25.335208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:25.335208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:37:25.335208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:25.335208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:37:25.335208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:37:25.335208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:37:25.335208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:37:25.335208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:25.335208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:25.335208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:25.335208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:37:25.335208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:25.335579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:25.335579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:25.335579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:37:25.335579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:25.335579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:37:25.335579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:37:25.335579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:25.335579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:37:25.335579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:25.335579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:37:25.335579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:37:25.335579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:25.335579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:37:25.335922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:25.335922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:37:25.336133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:25.336133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:37:25.336133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:37:25.336133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:37:25.336133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:37:25.336133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:37:25.336133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:25.336133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:37:25.336133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:37:25.336133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:37:25.336133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:37:25.336133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:37:25.336133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:25.336133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:25.336133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:37:25.336133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:37:25.336613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:37:25.336613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:37:25.336613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:25.336613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:25.336613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:25.336613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:37:25.336613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:25.336613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:25.336613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:25.336613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:37:25.336613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:25.336613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:37:25.336613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:37:25.336613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:25.336613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:37:25.336613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:25.336613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:37:25.336613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:37:25.336613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:25.337299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:37:25.337299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:25.337299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:37:30.075907 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:37:30.076853 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:37:30.076853 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:37:30.077919 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:37:30.113767 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:37:30.114201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:30.115062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:37:30.115062 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:30.115062 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:37:30.115062 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:37:30.115062 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:30.115062 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:37:30.115062 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:30.115062 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:37:30.116065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:30.116065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:37:30.116065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:30.116065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:37:30.116065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:37:30.116065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:37:30.116065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:37:30.116065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:30.116065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:30.116065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:30.116065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:37:30.116065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:30.116922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:30.116922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:30.116922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:37:30.116922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:30.116922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:37:30.116922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:37:30.116922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:30.116922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:37:30.116922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:30.116922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:37:30.116922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:37:30.116922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:30.116922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:37:30.117681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:30.117681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:37:30.117681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:30.117681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:37:30.117681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:37:30.117681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:37:30.117681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:37:30.117681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:37:30.117681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:30.117681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:37:30.117681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:37:30.117681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:37:30.117681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:37:30.117681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:37:30.117681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:30.118727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:30.118727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:37:30.118727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:37:30.118727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:37:30.118727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:37:30.118727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:30.118727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:30.118727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:30.118727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:37:30.118727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:30.118727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:30.118727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:30.118727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:37:30.118727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:30.118727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:37:30.118727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:37:30.118727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:30.118727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:37:30.118727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:30.120129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:37:30.120129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:37:30.120129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:30.120129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:37:30.120129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:30.120129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:37:30.313196 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:37:30.313695 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:37:30.313880 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:37:30.315054 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:37:30.315420 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:37:30.316062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:30.316868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:37:30.316868 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:30.316868 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:37:30.316868 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:37:30.316868 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:30.316868 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:37:30.316868 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:30.316868 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:37:30.318302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:30.318302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:37:30.318302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:30.318302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:37:30.318302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:37:30.318302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:37:30.318302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:37:30.318302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:30.318302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:30.318302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:30.318302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:37:30.318302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:30.318971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:30.318971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:30.318971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:37:30.318971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:30.318971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:37:30.318971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:37:30.318971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:30.318971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:37:30.318971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:30.318971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:37:30.318971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:37:30.318971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:30.318971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:37:30.319608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:30.319608 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:37:30.319965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:30.319965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:37:30.319965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:37:30.319965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:37:30.319965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:37:30.319965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:37:30.319965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:30.319965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:37:30.319965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:37:30.319965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:37:30.319965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:37:30.319965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:37:30.319965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:30.319965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:30.319965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:37:30.319965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:37:30.320726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:37:30.320726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:37:30.320726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:30.320726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:30.320726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:30.320726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:37:30.320726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:30.320726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:30.320726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:30.320726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:37:30.320726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:30.320726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:37:30.320726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:37:30.320726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:30.320726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:37:30.320726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:30.320726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:37:30.320726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:37:30.320726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:30.321613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:37:30.321613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:30.321613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:37:30.321613 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:37:30.321613 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:37:30.321895 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:37:30.321895 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:37:30.322322 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:37:30.322677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:30.323379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:37:30.323379 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:30.323379 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:37:30.323379 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:37:30.323379 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:30.323379 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:37:30.323379 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:30.323379 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:37:30.324119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:30.324119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:37:30.324119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:37:30.324119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:37:30.324119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:30.324119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:37:30.324119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:30.324119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:37:30.324119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:30.324119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:37:30.324119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:30.324119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:37:30.324119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:30.324119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:37:30.324119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:37:30.324658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:30.324658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:37:30.324658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:30.324658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:37:30.324658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:37:30.324658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:30.324658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:37:30.324658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:30.324658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:37:30.324944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:30.324944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:37:30.324944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:30.324944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:37:30.324944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:37:30.324944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:37:30.324944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:37:30.324944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:37:30.324944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:37:30.324944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:37:30.324944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:30.324944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:30.325312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:30.325312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:37:30.325312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:30.325312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:30.325312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:30.325312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:37:30.325312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:30.325312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:37:30.325312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:37:30.325312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:30.325312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:37:30.325312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:30.325312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:37:30.325312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:37:30.325713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:30.325713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:37:30.325713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:30.325713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:37:30.325845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:30.325845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:37:30.325845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:37:30.325845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:37:30.325845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:37:30.325845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:37:30.325845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:30.325845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:37:30.325845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:37:30.325845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:37:30.325845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:37:30.325845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:37:30.325845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:30.325845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:30.325845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:37:30.325845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:37:30.326294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:37:30.326294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:37:30.326294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:37:30.326294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:37:30.326294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:37:30.326294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:30.326294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:30.326294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:30.326294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:37:30.326294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:30.326294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:30.326294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:30.326294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:37:30.326294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:30.326294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:37:30.326294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:37:30.326294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:30.326294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:37:30.326827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:30.326827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:37:30.326827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:37:30.326827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:30.326827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:37:30.326827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:30.326827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:37:30.328533 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:37:30.328750 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:37:30.328847 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:37:30.328973 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:37:30.329287 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:37:30.329558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:30.330037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:37:30.330037 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:30.330037 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:37:30.330037 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:37:30.330037 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:30.330037 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:37:30.330037 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:30.330037 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:37:30.330738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:30.330738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:37:30.330738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:37:30.330738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:37:30.330738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:30.330738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:37:30.330738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:30.330738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:37:30.330738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:30.330738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:37:30.330738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:30.330738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:37:30.330738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:30.330738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:37:30.330738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:37:30.331191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:30.331191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:37:30.331191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:30.331191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:37:30.331191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:37:30.331191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:30.331191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:37:30.331191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:30.331191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:37:30.331575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:30.331575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:37:30.331575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:30.331575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:37:30.331575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:37:30.331575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:37:30.331575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:37:30.331575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:37:30.331575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:37:30.331575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:30.331575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:30.331935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:30.331935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:37:30.331935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:30.331935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:30.331935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:30.331935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:37:30.331935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:30.331935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:37:30.331935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:37:30.331935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:30.331935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:37:30.331935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:30.331935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:37:30.331935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:37:30.332312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:30.332312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:37:30.332312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:30.332312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:37:30.332579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:30.332579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:37:30.332579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:37:30.332579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:37:30.332579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:37:30.332579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:37:30.332579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:30.332579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:37:30.332579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:37:30.332579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:37:30.332579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:37:30.332579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:37:30.332579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:30.332579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:30.332579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:37:30.332579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:37:30.332579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:37:30.333100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:37:30.333100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:37:30.333100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:37:30.333100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:30.333100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:30.333100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:30.333100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:37:30.333100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:30.333100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:30.333100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:30.333100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:37:30.333100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:30.333100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:37:30.333100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:37:30.333100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:30.333100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:37:30.333100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:30.333100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:37:30.333100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:37:30.333671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:30.333671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:37:30.333671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:30.333671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:37:30.333671 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:37:30.333671 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:37:30.333671 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:37:30.333671 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:37:30.334118 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:37:30.334488 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:37:30.334707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:30.335363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:37:30.335363 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:30.335363 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:37:30.335363 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:37:30.335363 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:30.335363 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:37:30.335363 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:30.335363 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:37:30.336181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:30.336181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:37:30.336181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:30.336181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:37:30.336181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:37:30.336181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:37:30.336181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:37:30.336181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:30.336181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:30.336181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:30.336181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:37:30.336181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:30.336578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:30.336578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:30.336578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:37:30.336578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:30.336578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:37:30.336578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:37:30.336578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:30.336578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:37:30.336578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:30.336578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:37:30.336578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:37:30.336578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:30.336578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:37:30.336928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:30.336928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:37:30.337197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:30.337197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:37:30.337197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:37:30.337197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:37:30.337197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:37:30.337197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:37:30.337197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:30.337197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:37:30.337197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:37:30.337197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:37:30.337197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:37:30.337197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:37:30.337197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:30.337197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:30.337197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:37:30.337197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:37:30.337799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:37:30.337799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:37:30.337799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:30.337799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:30.337799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:30.337799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:37:30.337799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:30.337799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:30.337799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:30.337799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:37:30.337799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:30.337799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:37:30.337799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:37:30.337799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:30.337799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:37:30.337799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:30.337799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:37:30.337799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:37:30.337799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:30.338331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:37:30.338331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:30.338331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:37:35.076667 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:37:35.077303 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:37:35.077502 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:37:35.078539 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:37:35.112747 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:37:35.112950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:35.113506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:37:35.113506 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:35.113506 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:37:35.113506 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:37:35.113506 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:35.113506 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:37:35.113506 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:35.113506 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:37:35.114446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:35.114446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:37:35.114446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:35.114446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:37:35.114446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:37:35.114446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:37:35.114446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:37:35.114446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:35.114446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:35.114446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:35.114446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:37:35.114446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:35.115063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:35.115063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:35.115063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:37:35.115063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:35.115063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:37:35.115063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:37:35.115063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:35.115063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:37:35.115063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:35.115063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:37:35.115063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:37:35.115063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:35.115063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:37:35.115656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:35.115656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:37:35.115656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:35.115656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:37:35.115656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:37:35.115656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:37:35.115656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:37:35.115656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:37:35.115656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:35.115656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:37:35.115656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:37:35.115656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:37:35.115656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:37:35.115656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:37:35.115656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:35.116470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:35.116470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:37:35.116470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:37:35.116470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:37:35.116470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:37:35.116470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:35.116470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:35.116470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:35.116470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:37:35.116470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:35.116470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:35.116470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:35.116470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:37:35.116470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:35.116470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:37:35.116470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:37:35.116470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:35.116470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:37:35.116470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:35.117265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:37:35.117265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:37:35.117265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:35.117265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:37:35.117265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:35.117265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:37:35.314018 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:37:35.314710 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:37:35.314710 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:37:35.315335 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:37:35.315610 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:37:35.315746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:35.316284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:37:35.316284 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:35.316284 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:37:35.316284 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:37:35.316284 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:35.316284 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:37:35.316284 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:35.316284 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:37:35.317160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:35.317160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:37:35.317160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:35.317160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:37:35.317160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:37:35.317160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:37:35.317160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:37:35.317160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:35.317160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:35.317160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:35.317160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:37:35.317160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:35.317778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:35.317778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:35.317778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:37:35.317778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:35.317778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:37:35.317778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:37:35.317778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:35.317778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:37:35.317778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:35.317778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:37:35.317778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:37:35.317778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:35.317778 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:37:35.318091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:35.318091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:37:35.318277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:35.318277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:37:35.318277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:37:35.318277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:37:35.318277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:37:35.318277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:37:35.318277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:35.318277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:37:35.318277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:37:35.318277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:37:35.318277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:37:35.318277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:37:35.318277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:35.318277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:35.318277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:37:35.318277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:37:35.318610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:37:35.318610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:37:35.318610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:35.318610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:35.318610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:35.318610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:37:35.318610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:35.318610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:35.318610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:35.318610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:37:35.318610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:35.318610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:37:35.318610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:37:35.318610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:35.318610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:37:35.318610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:35.318610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:37:35.318610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:37:35.318610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:35.318976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:37:35.318976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:35.318976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:37:35.321572 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:37:35.321798 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:37:35.321909 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:37:35.322029 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:37:35.322336 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:37:35.322553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:35.323004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:37:35.323004 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:35.323004 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:37:35.323004 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:37:35.323004 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:35.323004 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:37:35.323004 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:35.323004 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:37:35.323728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:35.323728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:37:35.323728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:37:35.323728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:37:35.323728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:35.323728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:37:35.323728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:35.323728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:37:35.323728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:35.323728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:37:35.323728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:35.323728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:37:35.323728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:35.323728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:37:35.323728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:37:35.324052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:35.324052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:37:35.324052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:35.324052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:37:35.324052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:37:35.324052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:35.324052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:37:35.324052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:35.324052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:37:35.324527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:35.324527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:37:35.324527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:35.324527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:37:35.324527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:37:35.324527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:37:35.324527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:37:35.324527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:37:35.324527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:37:35.324527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:37:35.324527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:35.324527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:35.324787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:35.324787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:37:35.324787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:35.324787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:35.324787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:35.324787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:37:35.324787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:35.324787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:37:35.324787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:37:35.324787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:35.324787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:37:35.324787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:35.324787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:37:35.324787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:37:35.325042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:35.325042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:37:35.325042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:35.325042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:37:35.325514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:35.325514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:37:35.325514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:37:35.325514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:37:35.325514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:37:35.325514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:37:35.325514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:35.325514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:37:35.325514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:37:35.325514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:37:35.325514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:37:35.325514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:37:35.325514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:35.325514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:35.325514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:37:35.325514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:37:35.325832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:37:35.325832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:37:35.325832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:37:35.325832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:37:35.325832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:37:35.325832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:35.325832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:35.325832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:35.325832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:37:35.325832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:35.325832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:35.325832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:35.325832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:37:35.325832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:35.325832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:37:35.325832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:37:35.325832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:35.325832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:37:35.326338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:35.326338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:37:35.326338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:37:35.326338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:35.326338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:37:35.326338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:35.326338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:37:35.328907 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:37:35.329105 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:37:35.330249 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:37:35.330249 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:37:35.330249 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:37:35.330249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:35.330596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:37:35.330596 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:35.330596 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:37:35.330596 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:37:35.330596 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:35.330596 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:37:35.330596 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:35.330596 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:37:35.331709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:35.331709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:37:35.331709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:37:35.331709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:37:35.331709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:35.331709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:37:35.331709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:35.331709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:37:35.331709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:35.331709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:37:35.331709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:35.331709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:37:35.331709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:35.331709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:37:35.331709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:37:35.332150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:35.332150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:37:35.332150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:35.332150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:37:35.332150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:37:35.332150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:35.332150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:37:35.332150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:35.332150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:37:35.333052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:35.333052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:37:35.333052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:35.333052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:37:35.333052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:37:35.333052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:37:35.333052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:37:35.333052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:37:35.333052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:37:35.333052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:35.333052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:35.333387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:35.333387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:37:35.333387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:35.333387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:35.333387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:35.333387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:37:35.333387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:35.333387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:37:35.333387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:37:35.333387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:35.333387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:37:35.333387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:35.333387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:37:35.333387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:37:35.333776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:35.333776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:37:35.333776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:35.333776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:37:35.335019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:35.335019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:37:35.335019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:37:35.335019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:37:35.335019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:37:35.335019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:37:35.335019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:35.335019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:37:35.335019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:37:35.335019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:37:35.335019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:37:35.335019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:37:35.335019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:35.335019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:35.335019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:37:35.335019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:37:35.335019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:37:35.336385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:37:35.336385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:37:35.336385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:37:35.336385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:35.336385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:35.336385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:35.336385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:37:35.336385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:35.336385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:35.336385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:35.336385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:37:35.336385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:35.336385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:37:35.336385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:37:35.336385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:35.336385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:37:35.336385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:35.336385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:37:35.336385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:37:35.337293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:35.337293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:37:35.337293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:35.337293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:37:35.337293 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:37:35.337293 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:37:35.337293 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:37:35.337293 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:37:35.337763 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:37:35.337763 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:37:35.337763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:35.337950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:37:35.337950 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:35.337950 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:37:35.337950 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:37:35.337950 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:35.337950 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:37:35.337950 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:35.337950 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:37:35.338721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:35.338721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:37:35.338721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:35.338721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:37:35.338721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:37:35.338721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:37:35.338721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:37:35.338721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:35.338721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:35.338721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:35.338721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:37:35.338721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:35.339359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:35.339359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:35.339359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:37:35.339359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:35.339359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:37:35.339359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:37:35.339359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:35.339359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:37:35.339359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:35.339359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:37:35.339359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:37:35.339359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:35.339359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:37:35.340001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:35.340001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:37:35.340001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:35.340001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:37:35.340001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:37:35.340001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:37:35.340001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:37:35.340001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:37:35.340001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:35.340001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:37:35.340001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:37:35.340001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:37:35.340001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:37:35.340001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:37:35.340001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:35.340777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:35.340777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:37:35.340777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:37:35.340777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:37:35.340777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:37:35.340777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:35.340777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:35.340777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:35.340777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:37:35.340777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:35.340777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:35.340777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:35.340777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:37:35.340777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:35.340777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:37:35.340777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:37:35.340777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:35.340777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:37:35.340777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:35.341657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:37:35.341657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:37:35.341657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:35.341657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:37:35.341657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:35.341657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:37:40.077372 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:37:40.078178 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:37:40.078178 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:37:40.079304 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:37:40.119249 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:37:40.119529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:40.120577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:37:40.120577 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:40.120577 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:37:40.120577 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:37:40.120577 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:40.120577 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:37:40.120577 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:40.120577 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:37:40.121314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:40.121314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:37:40.121314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:40.121314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:37:40.121314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:37:40.121314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:37:40.121314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:37:40.121314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:40.121314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:40.121314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:40.121314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:37:40.121314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:40.121753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:40.121753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:40.121753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:37:40.121753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:40.121753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:37:40.121753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:37:40.121753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:40.121753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:37:40.121753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:40.121753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:37:40.121753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:37:40.121753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:40.121753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:37:40.122138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:40.122138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:37:40.122138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:40.122138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:37:40.122138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:37:40.122138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:37:40.122138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:37:40.122138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:37:40.122138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:40.122138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:37:40.122138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:37:40.122138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:37:40.122138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:37:40.122138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:37:40.122138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:40.122625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:40.122625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:37:40.122625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:37:40.122625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:37:40.122625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:37:40.122625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:40.122625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:40.122625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:40.122625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:37:40.122625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:40.122625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:40.122625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:40.122625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:37:40.122625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:40.122625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:37:40.122625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:37:40.122625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:40.122625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:37:40.122625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:40.123199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:37:40.123199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:37:40.123199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:40.123199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:37:40.123199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:40.123199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:37:40.314681 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:37:40.315238 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:37:40.315428 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:37:40.316654 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:37:40.317023 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:37:40.317264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:40.318184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:37:40.318184 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:40.318184 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:37:40.318184 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:37:40.318184 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:40.318184 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:37:40.318184 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:40.318184 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:37:40.319818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:40.319818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:37:40.319818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:40.319818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:37:40.319818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:37:40.319818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:37:40.319818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:37:40.319818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:40.319818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:40.319818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:40.319818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:37:40.319818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:40.320390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:40.320390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:40.320390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:37:40.320390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:40.320390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:37:40.320390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:37:40.320390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:40.320390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:37:40.320390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:40.320390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:37:40.320390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:37:40.320390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:40.320390 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:37:40.320992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:40.320992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:37:40.321323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:40.321323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:37:40.321323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:37:40.321323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:37:40.321323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:37:40.321323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:37:40.321323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:40.321323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:37:40.321323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:37:40.321323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:37:40.321323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:37:40.321323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:37:40.321323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:40.321323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:40.321323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:37:40.321323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:37:40.322030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:37:40.322030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:37:40.322030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:40.322030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:40.322030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:40.322030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:37:40.322030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:40.322030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:40.322030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:40.322030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:37:40.322030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:40.322030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:37:40.322030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:37:40.322030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:40.322030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:37:40.322030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:40.322030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:37:40.322030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:37:40.322030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:40.322827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:37:40.322827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:40.322827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:37:40.322827 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:37:40.322827 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:37:40.322827 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:37:40.322827 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:37:40.323146 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:37:40.323663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:40.324060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:37:40.324060 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:40.324060 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:37:40.324060 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:37:40.324060 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:40.324060 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:37:40.324060 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:40.324060 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:37:40.324834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:40.324834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:37:40.324834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:37:40.324834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:37:40.324834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:40.324834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:37:40.324834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:40.324834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:37:40.324834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:40.324834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:37:40.324834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:40.324834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:37:40.324834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:40.324834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:37:40.324834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:37:40.325283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:40.325283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:37:40.325283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:40.325283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:37:40.325283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:37:40.325283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:40.325283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:37:40.325283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:40.325283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:37:40.325642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:40.325642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:37:40.325642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:40.325642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:37:40.325642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:37:40.325642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:37:40.325642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:37:40.325642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:37:40.325642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:37:40.325642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:37:40.325642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:40.325642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:40.325971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:40.325971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:37:40.325971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:40.325971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:40.325971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:40.325971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:37:40.325971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:40.325971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:37:40.325971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:37:40.325971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:40.325971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:37:40.325971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:40.325971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:37:40.325971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:37:40.326378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:40.326378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:37:40.326378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:40.326378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:37:40.326783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:40.326783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:37:40.326783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:37:40.326783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:37:40.326783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:37:40.326783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:37:40.326783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:40.326783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:37:40.326783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:37:40.326783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:37:40.326783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:37:40.326783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:37:40.326783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:40.326783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:40.326783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:37:40.326783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:37:40.327284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:37:40.327284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:37:40.327284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:37:40.327284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:37:40.327284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:37:40.327284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:40.327284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:40.327284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:40.327284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:37:40.327284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:40.327284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:40.327284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:40.327284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:37:40.327284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:40.327284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:37:40.327284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:37:40.327284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:40.327284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:37:40.327826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:40.327826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:37:40.327826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:37:40.327826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:40.327826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:37:40.327826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:40.327826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:37:40.329237 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:37:40.329551 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:37:40.329695 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:37:40.329827 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:37:40.330146 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:37:40.330369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:40.330917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:37:40.330917 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:40.330917 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:37:40.330917 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:37:40.330917 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:40.330917 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:37:40.330917 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:40.330917 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:37:40.331723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:40.331723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:37:40.331723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:37:40.331723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:37:40.331723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:40.331723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:37:40.331723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:40.331723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:37:40.331723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:40.331723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:37:40.331723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:40.331723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:37:40.331723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:40.331723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:37:40.331723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:37:40.332290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:40.332290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:37:40.332290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:40.332290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:37:40.332290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:37:40.332290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:40.332290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:37:40.332290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:40.332290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:37:40.332668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:40.332668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:37:40.332668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:40.332668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:37:40.332668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:37:40.332668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:37:40.332668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:37:40.332668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:37:40.332668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:37:40.332668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:40.332668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:40.333094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:40.333094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:37:40.333094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:40.333094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:40.333094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:40.333094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:37:40.333094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:40.333094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:37:40.333094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:37:40.333094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:40.333094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:37:40.333094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:40.333094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:37:40.333094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:37:40.333620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:40.333620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:37:40.333620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:40.333620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:37:40.334085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:40.334085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:37:40.334085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:37:40.334085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:37:40.334085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:37:40.334085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:37:40.334085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:40.334085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:37:40.334085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:37:40.334085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:37:40.334085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:37:40.334085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:37:40.334085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:40.334085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:40.334085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:37:40.334085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:37:40.334085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:37:40.334691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:37:40.334691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:37:40.334691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:37:40.334691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:40.334691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:40.334691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:40.334691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:37:40.334691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:40.334691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:40.334691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:40.334691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:37:40.334691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:40.334691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:37:40.334691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:37:40.334691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:40.334691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:37:40.334691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:40.334691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:37:40.334691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:37:40.335357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:40.335357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:37:40.335357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:40.335357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:37:40.335586 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:37:40.335831 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:37:40.335934 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:37:40.336093 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:37:40.336424 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:37:40.336669 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:37:40.336846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:40.337378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:37:40.337378 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:40.337378 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:37:40.337378 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:37:40.337378 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:40.337378 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:37:40.337378 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:40.337378 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:37:40.338247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:40.338247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:37:40.338247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:40.338247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:37:40.338247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:37:40.338247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:37:40.338247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:37:40.338247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:40.338247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:40.338247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:40.338247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:37:40.338247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:40.338720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:40.338720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:40.338720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:37:40.338720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:40.338720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:37:40.338720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:37:40.338720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:40.338720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:37:40.338720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:40.338720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:37:40.338720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:37:40.338720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:40.338720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:37:40.339199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:40.339199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:37:40.339486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:40.339486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:37:40.339486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:37:40.339486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:37:40.339486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:37:40.339486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:37:40.339486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:40.339486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:37:40.339486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:37:40.339486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:37:40.339486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:37:40.339486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:37:40.339486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:40.339486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:40.339486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:37:40.339486 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:37:40.340040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:37:40.340040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:37:40.340040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:40.340040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:40.340040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:40.340040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:37:40.340040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:40.340040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:40.340040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:40.340040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:37:40.340040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:40.340040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:37:40.340040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:37:40.340040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:40.340040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:37:40.340040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:40.340040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:37:40.340040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:37:40.340040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:40.340728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:37:40.340728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:40.340728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:37:45.078108 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:37:45.079157 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:37:45.079157 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:37:45.079790 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:37:45.110812 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:37:45.111201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:45.112053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:37:45.112053 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:45.112053 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:37:45.112053 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:37:45.112053 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:45.112053 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:37:45.112053 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:45.112053 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:37:45.113091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:45.113091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:37:45.113091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:45.113091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:37:45.113091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:37:45.113091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:37:45.113091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:37:45.113091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:45.113091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:45.113091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:45.113091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:37:45.113091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:45.113544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:45.113544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:45.113544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:37:45.113544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:45.113544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:37:45.113544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:37:45.113544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:45.113544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:37:45.113544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:45.113544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:37:45.113544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:37:45.113544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:45.113544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:37:45.113931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:45.113931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:37:45.113931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:45.113931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:37:45.113931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:37:45.113931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:37:45.113931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:37:45.113931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:37:45.113931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:45.113931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:37:45.113931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:37:45.113931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:37:45.113931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:37:45.113931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:37:45.113931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:45.114459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:45.114459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:37:45.114459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:37:45.114459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:37:45.114459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:37:45.114459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:45.114459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:45.114459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:45.114459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:37:45.114459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:45.114459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:45.114459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:45.114459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:37:45.114459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:45.114459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:37:45.114459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:37:45.114459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:45.114459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:37:45.114459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:45.115108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:37:45.115108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:37:45.115108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:45.115108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:37:45.115108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:45.115108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:37:45.315311 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:37:45.316071 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:37:45.316288 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:37:45.317514 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:37:45.317878 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:37:45.318144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:45.319224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:37:45.319224 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:45.319224 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:37:45.319224 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:37:45.319224 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:45.319224 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:37:45.319224 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:45.319224 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:37:45.320902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:45.320902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:37:45.320902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:45.320902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:37:45.320902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:37:45.320902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:37:45.320902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:37:45.320902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:45.320902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:45.320902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:45.320902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:37:45.320902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:45.321536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:45.321536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:45.321536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:37:45.321536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:45.321536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:37:45.321536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:37:45.321536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:45.321536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:37:45.321536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:45.321536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:37:45.321536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:37:45.321536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:45.321536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:37:45.322102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:45.322102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:37:45.322512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:45.322512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:37:45.322512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:37:45.322512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:37:45.322512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:37:45.322512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:37:45.322512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:45.322512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:37:45.322512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:37:45.322512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:37:45.322512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:37:45.322512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:37:45.322512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:45.322512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:45.322512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:37:45.322512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:37:45.323244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:37:45.323244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:37:45.323244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:45.323244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:45.323244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:45.323244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:37:45.323244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:45.323244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:45.323244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:45.323244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:37:45.323244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:45.323244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:37:45.323244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:37:45.323244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:45.323244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:37:45.323244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:45.323244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:37:45.323244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:37:45.323244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:45.324070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:37:45.324070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:45.324070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:37:45.324070 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:37:45.324070 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:37:45.324070 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:37:45.324070 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:37:45.324070 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:37:45.324579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:45.324641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:37:45.324641 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:45.324641 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:37:45.324641 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:37:45.324641 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:45.324641 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:37:45.324641 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:45.324641 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:37:45.325327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:45.325327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:37:45.325327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:37:45.325327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:37:45.325327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:45.325327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:37:45.325327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:45.325327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:37:45.325327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:45.325327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:37:45.325327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:45.325327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:37:45.325327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:45.325327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:37:45.325327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:37:45.326173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:45.326173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:37:45.326173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:45.326173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:37:45.326173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:37:45.326173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:45.326173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:37:45.326173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:45.326173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:37:45.326173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:45.326173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:37:45.326173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:45.326173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:37:45.326173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:37:45.326631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:37:45.326631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:37:45.326631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:37:45.326631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:37:45.326631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:37:45.326631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:45.326631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:45.326631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:45.326631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:37:45.326631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:45.326631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:45.326631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:45.326631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:37:45.327020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:45.327020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:37:45.327020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:37:45.327020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:45.327020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:37:45.327020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:45.327020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:37:45.327020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:37:45.327020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:45.327020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:37:45.327020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:45.327020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:37:45.327348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:45.327348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:37:45.327348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:37:45.327348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:37:45.327348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:37:45.327348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:37:45.327348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:45.327348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:37:45.327348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:37:45.327348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:37:45.327348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:37:45.327348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:37:45.327348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:45.327348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:45.327348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:37:45.327348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:37:45.327811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:37:45.327811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:37:45.327811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:37:45.327811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:37:45.327811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:37:45.327811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:45.327811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:45.327811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:45.327811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:37:45.327811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:45.327811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:45.327811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:45.327811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:37:45.327811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:45.327811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:37:45.327811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:37:45.327811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:45.327811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:37:45.328377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:45.328377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:37:45.328377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:37:45.328377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:45.328377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:37:45.328377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:45.328377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:37:45.329689 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:37:45.330000 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:37:45.330150 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:37:45.330294 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:37:45.330629 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:37:45.330851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:45.331320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:37:45.331320 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:45.331320 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:37:45.331320 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:37:45.331320 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:45.331320 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:37:45.331320 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:45.331320 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:37:45.332076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:45.332076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:37:45.332076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:37:45.332076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:37:45.332076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:45.332076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:37:45.332076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:45.332076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:37:45.332076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:45.332076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:37:45.332076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:45.332076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:37:45.332076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:45.332076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:37:45.332076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:37:45.332508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:45.332508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:37:45.332508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:45.332508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:37:45.332508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:37:45.332508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:45.332508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:37:45.332508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:45.332508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:37:45.333005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:45.333005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:37:45.333005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:45.333005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:37:45.333005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:37:45.333005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:37:45.333005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:37:45.333005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:37:45.333005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:37:45.333005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:45.333005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:45.333338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:45.333338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:37:45.333338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:45.333338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:45.333338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:45.333338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:37:45.333338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:45.333338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:37:45.333338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:37:45.333338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:45.333338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:37:45.333338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:45.333338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:37:45.333338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:37:45.333780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:45.333780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:37:45.333780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:45.333780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:37:45.334130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:45.334130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:37:45.334130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:37:45.334130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:37:45.334130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:37:45.334130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:37:45.334130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:45.334130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:37:45.334130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:37:45.334130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:37:45.334130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:37:45.334130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:37:45.334130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:45.334130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:45.334130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:37:45.334130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:37:45.334130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:37:45.334644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:37:45.334644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:37:45.334644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:37:45.334644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:45.334644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:45.334644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:45.334644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:37:45.334644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:45.334644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:45.334644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:45.334644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:37:45.334644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:45.334644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:37:45.334644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:37:45.334644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:45.334644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:37:45.334644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:45.334644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:37:45.334644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:37:45.335168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:45.335168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:37:45.335168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:45.335168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:37:45.335961 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:37:45.336228 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:37:45.336341 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:37:45.336513 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:37:45.336951 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:37:45.337316 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:37:45.337670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:45.338417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:37:45.338417 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:45.338417 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:37:45.338417 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:37:45.338417 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:45.338417 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:37:45.338417 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:45.338417 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:37:45.339239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:45.339239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:37:45.339239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:45.339239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:37:45.339239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:37:45.339239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:37:45.339239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:37:45.339239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:45.339239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:45.339239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:45.339239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:37:45.339239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:45.339731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:45.339731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:45.339731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:37:45.339731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:45.339731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:37:45.339731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:37:45.339731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:45.339731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:37:45.339731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:45.339731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:37:45.339731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:37:45.339731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:45.339731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:37:45.340089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:45.340089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:37:45.340218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:45.340218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:37:45.340218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:37:45.340218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:37:45.340218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:37:45.340218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:37:45.340218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:45.340218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:37:45.340218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:37:45.340218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:37:45.340218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:37:45.340218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:37:45.340218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:45.340218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:45.340218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:37:45.340218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:37:45.340687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:37:45.340687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:37:45.340687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:45.340687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:45.340687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:45.340687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:37:45.340687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:45.340687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:45.340687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:45.340687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:37:45.340687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:45.340687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:37:45.340687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:37:45.340687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:45.340687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:37:45.340687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:45.340687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:37:45.340687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:37:45.340687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:45.341204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:37:45.341204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:45.341204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:37:50.079121 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:37:50.079952 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:37:50.080142 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:37:50.081340 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:37:50.116848 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:37:50.116848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:50.117587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:37:50.117587 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:50.117587 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:37:50.117587 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:37:50.117587 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:50.117587 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:37:50.117587 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:50.117587 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:37:50.118340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:50.118340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:37:50.118340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:50.118340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:37:50.118340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:37:50.118340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:37:50.118340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:37:50.118340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:50.118340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:50.118340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:50.118340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:37:50.118340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:50.118789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:50.118789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:50.118789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:37:50.118789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:50.118789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:37:50.118789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:37:50.118789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:50.118789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:37:50.118789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:50.118789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:37:50.118789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:37:50.118789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:50.118789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:37:50.119151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:50.119151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:37:50.119151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:50.119151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:37:50.119151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:37:50.119151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:37:50.119151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:37:50.119151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:37:50.119151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:50.119151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:37:50.119151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:37:50.119151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:37:50.119151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:37:50.119151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:37:50.119151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:50.119599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:50.119599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:37:50.119599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:37:50.119599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:37:50.119599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:37:50.119599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:50.119599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:50.119599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:50.119599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:37:50.119599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:50.119599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:50.119599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:50.119599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:37:50.119599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:50.119599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:37:50.119599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:37:50.119599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:50.119599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:37:50.119599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:50.120074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:37:50.120074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:37:50.120074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:50.120074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:37:50.120074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:50.120074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:37:50.316066 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:37:50.316619 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:37:50.316785 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:37:50.318029 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:37:50.318436 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:37:50.318679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:50.319662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:37:50.319662 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:50.319662 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:37:50.319662 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:37:50.319662 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:50.319662 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:37:50.319662 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:50.319662 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:37:50.321094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:50.321094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:37:50.321094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:50.321094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:37:50.321094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:37:50.321094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:37:50.321094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:37:50.321094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:50.321094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:50.321094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:50.321094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:37:50.321094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:50.321906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:50.321906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:50.321906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:37:50.321906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:50.321906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:37:50.321906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:37:50.321906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:50.321906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:37:50.321906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:50.321906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:37:50.321906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:37:50.321906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:50.321906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:37:50.322458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:50.322458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:37:50.323069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:50.323069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:37:50.323069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:37:50.323069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:37:50.323069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:37:50.323069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:37:50.323069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:50.323069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:37:50.323069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:37:50.323069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:37:50.323069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:37:50.323069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:37:50.323069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:50.323069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:50.323069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:37:50.323069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:37:50.325438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:37:50.325438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:37:50.325438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:50.325438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:50.325438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:50.325438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:37:50.325438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:50.325438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:50.325438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:50.325438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:37:50.325438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:50.325438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:37:50.325438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:37:50.325438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:50.325438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:37:50.325438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:50.325438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:37:50.325438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:37:50.325438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:50.325990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:37:50.325990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:50.325990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:37:50.325990 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:37:50.325990 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:37:50.325990 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:37:50.325990 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:37:50.325990 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:37:50.326229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:50.326229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:37:50.326229 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:50.326229 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:37:50.326229 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:37:50.326229 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:50.326229 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:37:50.326229 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:50.326229 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:37:50.326229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:50.326229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:37:50.326229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:37:50.326229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:37:50.326229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:50.326229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:37:50.326229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:50.326229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:37:50.326700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:50.326700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:37:50.326700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:50.326700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:37:50.326700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:50.326700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:37:50.326700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:37:50.326700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:50.326700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:37:50.326700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:50.326700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:37:50.326700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:37:50.326700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:50.326700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:37:50.326700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:50.326700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:37:50.328516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:50.328516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:37:50.328516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:50.328516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:37:50.328516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:37:50.328516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:37:50.328516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:37:50.328516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:37:50.328516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:37:50.328516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:37:50.328516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:50.328516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:50.329371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:50.329371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:37:50.329371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:50.329371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:50.329371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:50.329371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:37:50.329371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:50.329371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:37:50.329371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:37:50.329371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:50.329371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:37:50.329371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:50.329371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:37:50.329371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:37:50.330984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:50.330984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:37:50.330984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:50.330984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:37:50.330984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:50.330984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:37:50.330984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:37:50.330984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:37:50.330984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:37:50.330984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:37:50.330984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:50.330984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:37:50.330984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:37:50.330984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:37:50.331640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:37:50.331640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:37:50.331640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:50.331640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:50.331640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:37:50.331640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:37:50.331640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:37:50.331640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:37:50.331640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:37:50.331640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:37:50.331640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:37:50.331640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:50.331640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:50.331640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:50.331640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:37:50.331640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:50.331640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:50.331640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:50.331640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:37:50.332096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:50.332096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:37:50.332096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:37:50.332096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:50.332096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:37:50.332096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:50.332096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:37:50.332096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:37:50.332096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:50.332096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:37:50.332096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:50.332096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:37:50.332096 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:37:50.332096 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:37:50.332531 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:37:50.332531 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:37:50.332531 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:37:50.332531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:50.332531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:37:50.332531 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:50.332531 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:37:50.332531 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:37:50.332531 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:50.332531 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:37:50.332531 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:50.332531 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:37:50.332983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:50.332983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:37:50.332983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:37:50.332983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:37:50.332983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:50.332983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:37:50.332983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:50.332983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:37:50.332983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:50.332983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:37:50.332983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:50.332983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:37:50.332983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:50.332983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:37:50.332983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:37:50.334561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:50.334561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:37:50.334561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:50.334561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:37:50.334561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:37:50.334561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:50.334561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:37:50.334561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:50.334561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:37:50.334561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:50.334561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:37:50.334561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:50.334561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:37:50.334561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:37:50.336313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:37:50.336313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:37:50.336313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:37:50.336313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:37:50.336313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:50.336313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:50.336313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:50.336313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:37:50.336313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:50.336313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:50.336313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:50.336313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:37:50.336313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:50.336838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:37:50.336838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:37:50.336838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:50.336838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:37:50.336838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:50.336838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:37:50.336838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:37:50.336838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:50.336838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:37:50.336838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:50.336838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:37:50.336838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:50.337079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:37:50.337079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:37:50.337079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:37:50.337079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:37:50.337079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:37:50.337079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:50.337079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:37:50.337079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:37:50.337079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:37:50.337079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:37:50.337079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:37:50.337079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:50.337079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:50.337079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:37:50.337079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:37:50.337079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:37:50.337079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:37:50.337079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:37:50.337079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:37:50.338032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:50.338032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:50.338032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:50.338032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:37:50.338032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:50.338032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:50.338032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:50.338032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:37:50.338032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:50.338032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:37:50.338032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:37:50.338032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:50.338032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:37:50.338032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:50.338032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:37:50.338032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:37:50.338032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:50.338032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:37:50.338032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:50.338032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:37:50.338600 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:37:50.338600 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:37:50.338600 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:37:50.338600 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:37:50.338600 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:37:50.338600 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:37:50.338743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:50.338743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:37:50.338743 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:50.338743 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:37:50.338743 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:37:50.338743 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:50.338743 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:37:50.338743 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:50.338743 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:37:50.339104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:50.339104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:37:50.339104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:50.339104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:37:50.339104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:37:50.339104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:37:50.339104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:37:50.339104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:50.339104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:50.339104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:50.339104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:37:50.339104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:50.339938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:50.339938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:50.339938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:37:50.339938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:50.339938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:37:50.339938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:37:50.339938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:50.339938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:37:50.339938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:50.339938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:37:50.339938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:37:50.339938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:50.339938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:37:50.340164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:50.340164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:37:50.340512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:50.340512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:37:50.340512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:37:50.340512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:37:50.340512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:37:50.340512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:37:50.340512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:50.340512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:37:50.340512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:37:50.340512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:37:50.340512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:37:50.340512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:37:50.340512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:50.340512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:50.340512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:37:50.340512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:37:50.340934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:37:50.340934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:37:50.340934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:50.340934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:50.340934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:50.340934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:37:50.340934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:50.340934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:50.340934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:50.340934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:37:50.340934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:50.340934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:37:50.340934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:37:50.340934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:50.340934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:37:50.340934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:50.340934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:37:50.340934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:37:50.340934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:50.341265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:37:50.341265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:50.341265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:37:55.080117 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:37:55.081050 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:37:55.081141 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:37:55.082443 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:37:55.115310 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:37:55.116192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:55.116686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:37:55.116686 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:55.116686 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:37:55.116686 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:37:55.116686 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:55.116686 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:37:55.116686 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:55.116686 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:37:55.117675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:55.117675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:37:55.117675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:55.117675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:37:55.117675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:37:55.117675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:37:55.117675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:37:55.117675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:55.117675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:55.117675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:55.117675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:37:55.117675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:55.118354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:55.118354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:55.118354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:37:55.118354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:55.118354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:37:55.118354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:37:55.118354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:55.118354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:37:55.118354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:55.118354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:37:55.118354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:37:55.118354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:55.118354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:37:55.119031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:55.119031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:37:55.119031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:55.119031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:37:55.119031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:37:55.119031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:37:55.119031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:37:55.119031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:37:55.119031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:55.119031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:37:55.119031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:37:55.119031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:37:55.119031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:37:55.119031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:37:55.119031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:55.119871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:55.119871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:37:55.119871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:37:55.119871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:37:55.119871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:37:55.119871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:55.119871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:55.119871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:55.119871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:37:55.119871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:55.119871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:55.119871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:55.119871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:37:55.119871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:55.119871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:37:55.119871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:37:55.119871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:55.119871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:37:55.119871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:55.120833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:37:55.120833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:37:55.120833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:55.120833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:37:55.120833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:55.120833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:37:55.316759 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:37:55.317161 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:37:55.317345 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:37:55.318603 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:37:55.318947 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:37:55.319201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:55.320216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:37:55.320216 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:55.320216 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:37:55.320216 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:37:55.320216 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:55.320216 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:37:55.320216 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:55.320216 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:37:55.321915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:55.321915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:37:55.321915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:55.321915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:37:55.321915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:37:55.321915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:37:55.321915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:37:55.321915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:55.321915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:55.321915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:55.321915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:37:55.321915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:55.322338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:55.322338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:55.322338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:37:55.322338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:55.322338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:37:55.322338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:37:55.322338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:55.322338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:37:55.322338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:55.322338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:37:55.322338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:37:55.322338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:55.322338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:37:55.322716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:55.322716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:37:55.323706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:55.323706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:37:55.323706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:37:55.323706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:37:55.323706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:37:55.323706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:37:55.323706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:55.323706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:37:55.323706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:37:55.323706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:37:55.323706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:37:55.323706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:37:55.323706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:55.323706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:55.323706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:37:55.323706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:37:55.324182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:37:55.324182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:37:55.324182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:55.324182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:55.324182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:55.324182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:37:55.324182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:55.324182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:55.324182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:55.324182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:37:55.324182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:55.324182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:37:55.324182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:37:55.324182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:55.324182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:37:55.324182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:55.324182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:37:55.324182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:37:55.324182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:55.324702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:37:55.324702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:55.324702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:37:55.324702 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:37:55.324702 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:37:55.324702 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:37:55.324702 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:37:55.326128 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:37:55.326493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:55.327281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:37:55.327281 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:55.327281 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:37:55.327281 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:37:55.327281 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:55.327281 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:37:55.327281 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:55.327281 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:37:55.328010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:55.328010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:37:55.328010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:37:55.328010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:37:55.328010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:55.328010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:37:55.328010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:55.328010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:37:55.328010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:55.328010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:37:55.328010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:55.328010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:37:55.328010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:55.328010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:37:55.328010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:37:55.328506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:55.328506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:37:55.328506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:55.328506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:37:55.328506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:37:55.328506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:55.328506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:37:55.328506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:55.328506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:37:55.328822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:55.328822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:37:55.328822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:55.328822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:37:55.328822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:37:55.328822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:37:55.328822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:37:55.328822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:37:55.328822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:37:55.328822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:37:55.328822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:55.328822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:55.329397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:55.329397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:37:55.329397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:55.329397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:55.329397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:55.329397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:37:55.329397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:55.329397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:37:55.329397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:37:55.329397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:55.329397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:37:55.329397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:55.329397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:37:55.329397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:37:55.329792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:55.329792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:37:55.329792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:55.329792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:37:55.329909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:55.329909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:37:55.329909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:37:55.329909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:37:55.329909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:37:55.329909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:37:55.329909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:55.329909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:37:55.329909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:37:55.329909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:37:55.329909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:37:55.329909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:37:55.329909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:55.329909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:55.329909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:37:55.329909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:37:55.330392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:37:55.330392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:37:55.330392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:37:55.330392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:37:55.330392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:37:55.330392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:55.330392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:55.330392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:55.330392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:37:55.330392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:55.330392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:55.330392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:55.330392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:37:55.330392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:55.330392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:37:55.330392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:37:55.330392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:55.330392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:37:55.331245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:55.331245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:37:55.331245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:37:55.331245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:55.331245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:37:55.331245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:55.331245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:37:55.331245 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:37:55.331245 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:37:55.331245 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:37:55.331593 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:37:55.331593 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:37:55.331669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:55.332146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:37:55.332146 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:55.332146 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:37:55.332146 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:37:55.332146 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:55.332146 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:37:55.332146 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:55.332146 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:37:55.332954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:55.332954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:37:55.332954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:37:55.332954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:37:55.332954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:55.332954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:37:55.332954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:55.332954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:37:55.332954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:55.332954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:37:55.332954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:55.332954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:37:55.332954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:55.332954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:37:55.332954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:37:55.333523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:55.333523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:37:55.333523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:55.333523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:37:55.333523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:37:55.333523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:55.333523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:37:55.333523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:55.333523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:37:55.333905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:55.333905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:37:55.333905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:55.333905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:37:55.333905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:37:55.333905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:37:55.333905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:37:55.333905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:37:55.333905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:37:55.333905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:55.333905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:55.334233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:55.334233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:37:55.334233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:55.334233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:55.334233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:55.334233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:37:55.334233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:55.334233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:37:55.334233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:37:55.334233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:55.334233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:37:55.334233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:55.334233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:37:55.334233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:37:55.334638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:55.334638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:37:55.334638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:55.334638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:37:55.334921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:55.334921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:37:55.334921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:37:55.334921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:37:55.334921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:37:55.334921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:37:55.334921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:55.334921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:37:55.334921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:37:55.334921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:37:55.334921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:37:55.334921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:37:55.334921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:55.334921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:55.334921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:37:55.334921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:37:55.334921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:37:55.335380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:37:55.335380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:37:55.335380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:37:55.335380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:55.335380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:55.335380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:55.335380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:37:55.335380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:55.335380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:55.335380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:55.335380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:37:55.335380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:55.335380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:37:55.335380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:37:55.335380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:55.335380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:37:55.335380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:55.335380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:37:55.335380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:37:55.335907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:55.335907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:37:55.335907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:55.335907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:37:55.336786 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:37:55.336972 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:37:55.337073 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:37:55.337211 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:37:55.337572 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:37:55.337942 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:37:55.337942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:55.338528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:37:55.338528 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:55.338528 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:37:55.338528 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:37:55.338528 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:55.338528 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:37:55.338528 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:55.338528 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:37:55.339167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:55.339167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:37:55.339167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:55.339167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:37:55.339167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:37:55.339167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:37:55.339167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:37:55.339167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:55.339167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:55.339167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:55.339167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:37:55.339167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:55.339557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:37:55.339557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:55.339557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:37:55.339557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:55.339557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:37:55.339557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:37:55.339557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:55.339557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:37:55.339557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:55.339557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:37:55.339557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:37:55.339557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:55.339557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:37:55.339896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:55.339896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:37:55.340101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:55.340101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:37:55.340101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:37:55.340101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:37:55.340101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:37:55.340101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:37:55.340101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:55.340101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:37:55.340101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:37:55.340101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:37:55.340101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:37:55.340101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:37:55.340101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:37:55.340101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:37:55.340101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:37:55.340101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:37:55.340655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:37:55.340655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:37:55.340655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:55.340655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:55.340655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:37:55.340655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:37:55.340655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:37:55.340655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:37:55.340655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:37:55.340655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:37:55.340655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:37:55.340655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:37:55.340655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:37:55.340655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:37:55.340655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:37:55.340655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:37:55.340655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:37:55.340655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:37:55.340655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:37:55.341197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:37:55.341197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:37:55.341197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:38:00.081547 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:38:00.082295 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:38:00.082555 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:38:00.083829 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:38:00.121633 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:38:00.122034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:00.122958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:38:00.122958 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:00.122958 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:38:00.122958 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:38:00.122958 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:00.122958 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:38:00.122958 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:00.122958 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:38:00.124021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:00.124021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:38:00.124021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:00.124021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:38:00.124021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:38:00.124021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:38:00.124021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:38:00.124021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:00.124021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:00.124021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:00.124021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:38:00.124021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:00.124725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:00.124725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:00.124725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:38:00.124725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:00.124725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:38:00.124725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:38:00.124725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:00.124725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:38:00.124725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:00.124725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:38:00.124725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:38:00.124725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:00.124725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:38:00.125375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:00.125375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:38:00.125375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:00.125375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:38:00.125375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:38:00.125375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:38:00.125375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:38:00.125375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:38:00.125375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:00.125375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:38:00.125375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:38:00.125375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:38:00.125375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:38:00.125375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:38:00.125375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:00.126214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:00.126214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:38:00.126214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:38:00.126214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:38:00.126214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:38:00.126214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:00.126214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:00.126214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:00.126214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:38:00.126214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:00.126214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:00.126214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:00.126214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:38:00.126214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:00.126214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:38:00.126214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:38:00.126214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:00.126214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:38:00.126214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:00.127198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:38:00.127198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:38:00.127198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:00.127198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:38:00.127198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:00.127198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:38:00.317381 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:38:00.317935 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:38:00.318036 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:38:00.319184 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:38:00.319557 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:38:00.319808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:00.321184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:38:00.321184 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:00.321184 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:38:00.321184 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:38:00.321184 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:00.321184 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:38:00.321184 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:00.321184 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:38:00.322760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:00.322760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:38:00.322760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:00.322760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:38:00.322760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:38:00.322760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:38:00.322760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:38:00.322760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:00.322760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:00.322760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:00.322760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:38:00.322760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:00.323379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:00.323379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:00.323379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:38:00.323379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:00.323379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:38:00.323379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:38:00.323379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:00.323379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:38:00.323379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:00.323379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:38:00.323379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:38:00.323379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:00.323379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:38:00.324371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:00.324371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:38:00.324371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:00.324371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:38:00.324371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:38:00.324371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:38:00.324371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:38:00.324371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:38:00.324371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:00.324371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:38:00.324371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:38:00.324371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:38:00.324371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:38:00.324371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:38:00.324371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:00.325629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:00.325629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:38:00.325629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:38:00.325629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:38:00.325629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:38:00.325629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:00.325629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:00.325629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:00.325629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:38:00.325629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:00.325629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:00.325629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:00.325629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:38:00.325629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:00.325629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:38:00.325629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:38:00.325629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:00.325629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:38:00.325629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:00.326601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:38:00.326601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:38:00.326601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:00.326601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:38:00.326601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:00.326601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:38:00.326601 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:38:00.326601 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:38:00.326601 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:38:00.326601 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:38:00.327177 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:38:00.327177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:00.327177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:38:00.327177 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:00.327177 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:38:00.327177 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:38:00.327177 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:00.327177 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:38:00.327177 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:00.327177 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:38:00.327689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:00.327689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:38:00.327689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:38:00.327689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:38:00.327689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:00.327689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:38:00.327689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:00.327689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:38:00.327689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:00.327689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:38:00.327689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:00.327689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:38:00.327689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:00.327689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:38:00.327689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:38:00.328503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:00.328503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:38:00.328503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:00.328503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:38:00.328503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:38:00.328503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:00.328503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:38:00.328503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:00.328503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:38:00.329202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:00.329202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:38:00.329202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:00.329202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:38:00.329202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:38:00.329202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:38:00.329202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:38:00.329202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:38:00.329202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:38:00.329202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:38:00.329202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:00.329202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:00.329854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:00.329854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:38:00.329854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:00.329854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:00.329854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:00.329854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:38:00.329854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:00.329854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:38:00.329854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:38:00.329854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:00.329854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:38:00.329854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:00.329854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:38:00.329854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:38:00.330521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:00.330521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:38:00.330521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:00.330521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:38:00.331088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:00.331088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:38:00.331088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:38:00.331088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:38:00.331088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:38:00.331088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:38:00.331088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:00.331088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:38:00.331088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:38:00.331088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:38:00.331088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:38:00.331088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:38:00.331088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:00.331088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:00.331088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:38:00.331088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:38:00.331777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:38:00.331777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:38:00.331777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:38:00.331777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:38:00.331777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:38:00.331777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:00.331777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:00.331777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:00.331777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:38:00.331777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:00.331777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:00.331777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:00.331777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:38:00.331777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:00.331777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:38:00.331777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:38:00.331777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:00.331777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:38:00.332393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:00.332393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:38:00.332393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:38:00.332393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:00.332393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:38:00.332393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:00.332393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:38:00.332393 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:38:00.332393 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:38:00.332393 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:38:00.332745 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:38:00.332745 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:38:00.332745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:00.333173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:38:00.333173 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:00.333173 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:38:00.333173 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:38:00.333173 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:00.333173 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:38:00.333173 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:00.333173 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:38:00.333902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:00.333902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:38:00.333902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:38:00.333902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:38:00.333902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:00.333902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:38:00.333902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:00.333902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:38:00.333902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:00.333902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:38:00.333902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:00.333902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:38:00.333902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:00.333902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:38:00.333902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:38:00.334417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:00.334417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:38:00.334417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:00.334417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:38:00.334417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:38:00.334417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:00.334417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:38:00.334417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:00.334417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:38:00.334796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:00.334796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:38:00.334796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:00.334796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:38:00.334796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:38:00.334796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:38:00.334796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:38:00.334796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:38:00.334796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:38:00.334796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:00.334796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:00.335179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:00.335179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:38:00.335179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:00.335179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:00.335179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:00.335179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:38:00.335179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:00.335179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:38:00.335179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:38:00.335179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:00.335179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:38:00.335179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:00.335179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:38:00.335179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:38:00.335579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:00.335579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:38:00.335579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:00.335579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:38:00.335841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:00.335841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:38:00.335841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:38:00.335841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:38:00.335841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:38:00.335841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:38:00.335841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:00.335841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:38:00.335841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:38:00.335841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:38:00.335841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:38:00.335841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:38:00.335841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:00.335841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:00.335841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:38:00.335841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:38:00.335841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:38:00.336348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:38:00.336348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:38:00.336348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:38:00.336348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:00.336348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:00.336348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:00.336348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:38:00.336348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:00.336348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:00.336348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:00.336348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:38:00.336348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:00.336348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:38:00.336348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:38:00.336348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:00.336348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:38:00.336348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:00.336348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:38:00.336348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:38:00.336905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:00.336905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:38:00.336905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:00.336905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:38:00.337025 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:38:00.337207 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:38:00.337302 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:38:00.337471 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:38:00.337768 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:38:00.337983 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:38:00.338127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:00.338621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:38:00.338621 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:00.338621 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:38:00.338621 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:38:00.338621 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:00.338621 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:38:00.338621 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:00.338621 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:38:00.339461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:00.339461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:38:00.339461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:00.339461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:38:00.339461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:38:00.339461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:38:00.339461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:38:00.339461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:00.339461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:00.339461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:00.339461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:38:00.339461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:00.339854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:00.339854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:00.339854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:38:00.339854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:00.339854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:38:00.339854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:38:00.339854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:00.339854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:38:00.339854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:00.339854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:38:00.339854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:38:00.339854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:00.339854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:38:00.340211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:00.340211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:38:00.340430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:00.340430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:38:00.340430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:38:00.340430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:38:00.340430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:38:00.340430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:38:00.340430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:00.340430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:38:00.340430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:38:00.340430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:38:00.340430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:38:00.340430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:38:00.340430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:00.340430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:00.340430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:38:00.340430 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:38:00.341253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:38:00.341253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:38:00.341253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:00.341253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:00.341253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:00.341253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:38:00.341253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:00.341253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:00.341253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:00.341253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:38:00.341253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:00.341253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:38:00.341253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:38:00.341253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:00.341253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:38:00.341253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:00.341253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:38:00.341253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:38:00.341253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:00.341872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:38:00.341872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:00.341872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:38:05.082549 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:38:05.083299 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:38:05.083364 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:38:05.084594 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:38:05.114008 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:38:05.114348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:05.115123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:38:05.115123 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:05.115123 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:38:05.115123 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:38:05.115123 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:05.115123 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:38:05.115123 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:05.115123 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:38:05.115815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:05.115815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:38:05.115815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:05.115815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:38:05.115815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:38:05.115815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:38:05.115815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:38:05.115815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:05.115815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:05.115815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:05.115815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:38:05.115815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:05.116483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:05.116483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:05.116483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:38:05.116483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:05.116483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:38:05.116483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:38:05.116483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:05.116483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:38:05.116483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:05.116483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:38:05.116483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:38:05.116483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:05.116483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:38:05.117029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:05.117029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:38:05.117029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:05.117029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:38:05.117029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:38:05.117029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:38:05.117029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:38:05.117029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:38:05.117029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:05.117029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:38:05.117029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:38:05.117029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:38:05.117029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:38:05.117029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:38:05.117029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:05.117772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:05.117772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:38:05.117772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:38:05.117772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:38:05.117772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:38:05.117772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:05.117772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:05.117772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:05.117772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:38:05.117772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:05.117772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:05.117772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:05.117772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:38:05.117772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:05.117772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:38:05.117772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:38:05.117772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:05.117772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:38:05.117772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:05.118795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:38:05.118795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:38:05.118795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:05.118795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:38:05.118795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:05.118795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:38:05.318127 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:38:05.318669 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:38:05.318858 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:38:05.319984 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:38:05.320326 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:38:05.320600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:05.321473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:38:05.321473 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:05.321473 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:38:05.321473 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:38:05.321473 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:05.321473 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:38:05.321473 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:05.321473 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:38:05.323000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:05.323000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:38:05.323000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:05.323000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:38:05.323000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:38:05.323000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:38:05.323000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:38:05.323000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:05.323000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:05.323000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:05.323000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:38:05.323000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:05.323643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:05.323643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:05.323643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:38:05.323643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:05.323643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:38:05.323643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:38:05.323643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:05.323643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:38:05.323643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:05.323643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:38:05.323643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:38:05.323643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:05.323643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:38:05.324252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:05.324252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:38:05.324735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:05.324735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:38:05.324735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:38:05.324735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:38:05.324735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:38:05.324735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:38:05.324735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:05.324735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:38:05.324735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:38:05.324735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:38:05.324735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:38:05.324735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:38:05.324735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:05.324735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:05.324735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:38:05.324735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:38:05.325210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:38:05.325210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:38:05.325210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:05.325210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:05.325210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:05.325210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:38:05.325210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:05.325210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:05.325210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:05.325210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:38:05.325210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:05.325210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:38:05.325210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:38:05.325210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:05.325210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:38:05.325210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:05.325210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:38:05.325210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:38:05.325210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:05.325777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:38:05.325777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:05.325777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:38:05.325777 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:38:05.325777 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:38:05.325777 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:38:05.325777 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:38:05.326117 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:38:05.326433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:05.327114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:38:05.327114 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:05.327114 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:38:05.327114 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:38:05.327114 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:05.327114 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:38:05.327114 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:05.327114 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:38:05.327828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:05.327828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:38:05.327828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:38:05.327828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:38:05.327828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:05.327828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:38:05.327828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:05.327828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:38:05.327828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:05.327828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:38:05.327828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:05.327828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:38:05.327828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:05.327828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:38:05.327828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:38:05.328372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:05.328372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:38:05.328372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:05.328372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:38:05.328372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:38:05.328372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:05.328372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:38:05.328372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:05.328372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:38:05.328796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:05.328796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:38:05.328796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:05.328796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:38:05.328796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:38:05.328796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:38:05.328796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:38:05.328796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:38:05.328796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:38:05.328796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:38:05.328796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:05.328796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:05.329143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:05.329143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:38:05.329143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:05.329143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:05.329143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:05.329143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:38:05.329143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:05.329143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:38:05.329143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:38:05.329143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:05.329143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:38:05.329143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:05.329143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:38:05.329143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:38:05.329525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:05.329525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:38:05.329525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:05.329525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:38:05.329768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:05.329768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:38:05.329768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:38:05.329768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:38:05.329768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:38:05.329768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:38:05.329768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:05.329768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:38:05.329768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:38:05.329768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:38:05.329768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:38:05.329768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:38:05.329768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:05.329768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:05.329768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:38:05.329768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:38:05.330202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:38:05.330202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:38:05.330202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:38:05.330202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:38:05.330202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:38:05.330202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:05.330202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:05.330202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:05.330202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:38:05.330202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:05.330202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:05.330202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:05.330202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:38:05.330202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:05.330202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:38:05.330202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:38:05.330202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:05.330202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:38:05.330926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:05.330926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:38:05.330926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:38:05.330926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:05.330926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:38:05.330926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:05.330926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:38:05.331626 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:38:05.331837 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:38:05.331936 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:38:05.332054 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:38:05.332517 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:38:05.332918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:05.333540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:38:05.333540 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:05.333540 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:38:05.333540 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:38:05.333540 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:05.333540 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:38:05.333540 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:05.333540 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:38:05.334224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:05.334224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:38:05.334224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:38:05.334224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:38:05.334224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:05.334224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:38:05.334224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:05.334224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:38:05.334224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:05.334224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:38:05.334224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:05.334224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:38:05.334224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:05.334224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:38:05.334224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:38:05.334681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:05.334681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:38:05.334681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:05.334681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:38:05.334681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:38:05.334681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:05.334681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:38:05.334681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:05.334681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:38:05.335129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:05.335129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:38:05.335129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:05.335129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:38:05.335129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:38:05.335129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:38:05.335129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:38:05.335129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:38:05.335129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:38:05.335129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:05.335129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:05.335479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:05.335479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:38:05.335479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:05.335479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:05.335479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:05.335479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:38:05.335479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:05.335479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:38:05.335479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:38:05.335479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:05.335479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:38:05.335479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:05.335479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:38:05.335479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:38:05.335867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:05.335867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:38:05.335867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:05.335867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:38:05.336124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:05.336124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:38:05.336124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:38:05.336124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:38:05.336124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:38:05.336124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:38:05.336124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:05.336124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:38:05.336124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:38:05.336124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:38:05.336124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:38:05.336124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:38:05.336124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:05.336124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:05.336124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:38:05.336124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:38:05.336124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:38:05.336619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:38:05.336619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:38:05.336619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:38:05.336619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:05.336619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:05.336619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:05.336619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:38:05.336619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:05.336619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:05.336619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:05.336619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:38:05.336619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:05.336619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:38:05.336619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:38:05.336619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:05.336619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:38:05.336619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:05.336619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:38:05.336619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:38:05.337142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:05.337142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:38:05.337142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:05.337142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:38:05.337259 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:38:05.337499 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:38:05.337914 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:38:05.337914 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:38:05.338242 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:38:05.338499 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:38:05.338651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:05.339114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:38:05.339114 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:05.339114 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:38:05.339114 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:38:05.339114 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:05.339114 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:38:05.339114 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:05.339114 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:38:05.339908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:05.339908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:38:05.339908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:05.339908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:38:05.339908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:38:05.339908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:38:05.339908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:38:05.339908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:05.339908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:05.339908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:05.339908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:38:05.339908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:05.340272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:05.340272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:05.340272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:38:05.340272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:05.340272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:38:05.340272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:38:05.340272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:05.340272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:38:05.340272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:05.340272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:38:05.340272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:38:05.340272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:05.340272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:38:05.340649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:05.340649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:38:05.340845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:05.340845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:38:05.340845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:38:05.340845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:38:05.340845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:38:05.340845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:38:05.340845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:05.340845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:38:05.340845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:38:05.340845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:38:05.340845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:38:05.340845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:38:05.340845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:05.340845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:05.340845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:38:05.340845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:38:05.341290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:38:05.341290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:38:05.341290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:05.341290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:05.341290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:05.341290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:38:05.341290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:05.341290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:05.341290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:05.341290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:38:05.341290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:05.341290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:38:05.341290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:38:05.341290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:05.341290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:38:05.341290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:05.341290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:38:05.341290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:38:05.341290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:05.341818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:38:05.341818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:05.341818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:38:10.083273 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:38:10.084260 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:38:10.084260 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:38:10.084827 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:38:10.119913 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:38:10.120159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:10.120749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:38:10.120749 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:10.120749 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:38:10.120749 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:38:10.120749 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:10.120749 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:38:10.120749 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:10.120749 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:38:10.121550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:10.121550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:38:10.121550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:10.121550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:38:10.121550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:38:10.121550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:38:10.121550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:38:10.121550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:10.121550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:10.121550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:10.121550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:38:10.121550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:10.122236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:10.122236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:10.122236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:38:10.122236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:10.122236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:38:10.122236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:38:10.122236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:10.122236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:38:10.122236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:10.122236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:38:10.122236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:38:10.122236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:10.122236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:38:10.123030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:10.123030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:38:10.123030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:10.123030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:38:10.123030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:38:10.123030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:38:10.123030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:38:10.123030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:38:10.123030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:10.123030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:38:10.123030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:38:10.123030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:38:10.123030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:38:10.123030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:38:10.123030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:10.123899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:10.123899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:38:10.123899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:38:10.123899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:38:10.123899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:38:10.123899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:10.123899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:10.123899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:10.123899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:38:10.123899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:10.123899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:10.123899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:10.123899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:38:10.123899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:10.123899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:38:10.123899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:38:10.123899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:10.123899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:38:10.123899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:10.125006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:38:10.125006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:38:10.125006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:10.125006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:38:10.125006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:10.125006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:38:10.318795 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:38:10.319434 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:38:10.319687 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:38:10.326128 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:38:10.327727 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:38:10.327727 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:38:10.327727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:10.329054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:38:10.329054 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:10.329054 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:38:10.329054 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:38:10.329054 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:10.329054 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:38:10.329054 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:10.329054 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:38:10.331281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.009 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:10.331281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:38:10.331281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:10.331281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:38:10.331281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:38:10.331281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:38:10.331281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:38:10.331281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:10.331281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:10.331281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:10.331281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:38:10.331281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:10.331813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:10.331813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:10.331813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:38:10.331813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:10.331813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:38:10.331813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:38:10.331813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:10.331813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:38:10.331813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:10.331813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:38:10.331813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:38:10.331813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:10.331813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:38:10.332361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:10.332361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:38:10.332361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:10.332361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:38:10.332361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:38:10.332361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:38:10.332361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:38:10.332361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:38:10.332361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:10.332361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:38:10.332361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:38:10.332361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:38:10.332361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:38:10.332361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:38:10.332361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:10.332993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:10.332993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:38:10.332993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:38:10.332993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:38:10.332993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:38:10.332993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:10.332993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:10.332993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:10.332993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:38:10.332993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:10.332993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:10.332993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:10.332993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:38:10.332993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:10.332993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:38:10.332993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:38:10.332993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:10.332993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:38:10.332993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:10.333876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:38:10.333876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:38:10.333876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:10.333876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:38:10.333876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:10.333876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:38:10.333876 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:38:10.333876 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:38:10.333876 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:38:10.333876 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:38:10.334332 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:38:10.334332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:10.335579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:38:10.335579 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:10.335579 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:38:10.335579 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:38:10.335579 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:10.335579 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:38:10.335579 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:10.335579 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:38:10.336531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:10.336531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:38:10.336531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:38:10.336531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:38:10.336531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:10.336531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:38:10.336531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:10.336531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:38:10.336531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:10.336531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:38:10.336531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:10.336531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:38:10.336531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:10.336531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:38:10.336531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:38:10.337260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:10.337260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:38:10.337260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:10.337260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:38:10.337260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:38:10.337260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:10.337260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:38:10.337260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:10.337260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:38:10.338303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:10.338303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:38:10.338303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:10.338303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:38:10.338303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:38:10.338303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:38:10.338303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:38:10.338303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:38:10.338303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:38:10.338303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:38:10.338303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:10.338303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:10.338661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:10.338661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:38:10.338661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:10.338661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:10.338661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:10.338661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:38:10.338661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:10.338661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:38:10.338661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:38:10.338661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:10.338661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:38:10.338661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:10.338661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:38:10.338661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:38:10.339056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:10.339056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:38:10.339056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:10.339056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:38:10.339056 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:38:10.339361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:10.339361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:38:10.339361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:38:10.339361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:38:10.339361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:38:10.339361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:38:10.339361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:10.339361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:38:10.339361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:38:10.339361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:38:10.339361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:38:10.339361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:38:10.339361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:10.339361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:10.339361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:38:10.339361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:38:10.339846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:38:10.339846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:38:10.339846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:38:10.339846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:38:10.339846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:38:10.339846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:10.339846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:10.339846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:10.339846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:38:10.339846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:10.339846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:10.339846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:10.339846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:38:10.339846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:10.339846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:38:10.339846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:38:10.339846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:10.339846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:38:10.340453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:10.340453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:38:10.340453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:38:10.340453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:10.340453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:38:10.340453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:10.340453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:38:10.340453 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:38:10.340453 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:38:10.340453 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:38:10.340776 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:38:10.340776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:10.340985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:38:10.340985 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:10.340985 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:38:10.340985 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:38:10.340985 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:10.340985 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:38:10.340985 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:10.340985 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:38:10.341557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:10.341557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:38:10.341557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:38:10.341557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:38:10.341557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:10.341557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:38:10.341557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:10.341557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:38:10.341557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:10.341557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:38:10.341557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:10.341557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:38:10.341557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:10.341557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:38:10.341557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:38:10.341990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:10.341990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:38:10.341990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:10.341990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:38:10.341990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:38:10.341990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:10.341990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:38:10.341990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:10.341990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:38:10.342246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:10.342246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:38:10.342246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:10.342246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:38:10.342246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:38:10.342246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:38:10.342246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:38:10.342246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:38:10.342246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:38:10.342246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:10.342246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:10.342613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:10.342613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:38:10.342613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:10.342613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:10.342613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:10.342613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:38:10.342613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:10.342613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:38:10.342613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:38:10.342613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:10.342613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:38:10.342613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:10.342613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:38:10.342613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:38:10.343028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:10.343028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:38:10.343028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:10.343028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:38:10.343028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:10.343028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:38:10.343028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:38:10.343028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:38:10.343028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:38:10.343028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:38:10.343028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:10.343028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:38:10.343028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:38:10.343028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:38:10.343490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:38:10.343490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:38:10.343490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:10.343490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:10.343490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:38:10.343490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:38:10.343490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:38:10.343490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:38:10.343490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:38:10.343490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:38:10.343490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:10.343490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:10.343490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:10.343490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:38:10.343490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:10.343490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:10.343490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:10.343490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:38:10.343490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:10.344040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:38:10.344040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:38:10.344040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:10.344040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:38:10.344040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:10.344040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:38:10.344040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:38:10.344040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:10.344040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:38:10.344040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:10.344040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:38:10.344040 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:38:10.344040 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:38:10.344548 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:38:10.344548 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:38:10.344548 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:38:10.344548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:10.344967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:38:10.344967 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:10.344967 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:38:10.344967 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:38:10.344967 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:10.344967 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:38:10.344967 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:10.344967 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:38:10.345592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:10.345592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:38:10.345592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:10.345592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:38:10.345592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:38:10.345592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:38:10.345592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:38:10.345592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:10.345592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:10.345592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:10.345592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:38:10.345592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:10.345984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:10.345984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:10.345984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:38:10.345984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:10.345984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:38:10.345984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:38:10.345984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:10.345984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:38:10.345984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:10.345984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:38:10.345984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:38:10.345984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:10.345984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:38:10.346337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:10.346337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:38:10.346400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:10.346400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:38:10.346400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:38:10.346400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:38:10.346400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:38:10.346400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:38:10.346400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:10.346400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:38:10.346400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:38:10.346400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:38:10.346400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:38:10.346400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:38:10.346400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:10.346400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:10.346400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:38:10.346400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:38:10.347013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:38:10.347013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:38:10.347013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:10.347013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:10.347013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:10.347013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:38:10.347013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:10.347013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:10.347013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:10.347013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:38:10.347013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:10.347013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:38:10.347013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:38:10.347013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:10.347013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:38:10.347013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:10.347013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:38:10.347013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:38:10.347013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:10.347867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:38:10.347867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:10.347867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:38:15.084021 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:38:15.084642 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:38:15.084834 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:38:15.086134 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:38:15.119920 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:38:15.120122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:15.120662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:38:15.120662 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:15.120662 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:38:15.120662 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:38:15.120662 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:15.120662 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:38:15.120662 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:15.120662 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:38:15.121531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:15.121531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:38:15.121531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:15.121531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:38:15.121531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:38:15.121531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:38:15.121531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:38:15.121531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:15.121531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:15.121531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:15.121531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:38:15.121531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:15.121831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:15.121831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:15.121831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:38:15.121831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:15.121831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:38:15.121831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:38:15.121831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:15.121831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:38:15.121831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:15.121831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:38:15.121831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:38:15.121831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:15.121831 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:38:15.122144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:15.122144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:38:15.122619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:15.122619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:38:15.122619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:38:15.122619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:38:15.122619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:38:15.122619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:38:15.122619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:15.122619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:38:15.122619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:38:15.122619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:38:15.122619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:38:15.122619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:38:15.122619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:15.122619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:15.122619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:38:15.122619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:38:15.122986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:38:15.122986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:38:15.122986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:15.122986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:15.122986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:15.122986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:38:15.122986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:15.122986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:15.122986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:15.122986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:38:15.122986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:15.122986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:38:15.122986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:38:15.122986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:15.122986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:38:15.122986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:15.122986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:38:15.122986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:38:15.122986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:15.123396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:38:15.123396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:15.123396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:38:15.319501 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:38:15.320898 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:38:15.320898 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:38:15.321594 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:38:15.321931 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:38:15.322123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:15.325539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:38:15.325539 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:15.325539 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:38:15.325539 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:38:15.325539 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:15.325539 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:38:15.325539 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:15.325539 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:38:15.325539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:15.325539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:38:15.325539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:15.325539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:38:15.325539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:38:15.325539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:38:15.325539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:38:15.326371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:15.326371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:15.326371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:15.326371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:38:15.326371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:15.326371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:15.326371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:15.326371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:38:15.326371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:15.326371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:38:15.326371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:38:15.326371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:15.326371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:38:15.326371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:15.327030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:38:15.327030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:38:15.327030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:15.327030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:38:15.327030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:15.327030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:38:15.327030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:15.327030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:38:15.327030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:38:15.327030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:38:15.327030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:38:15.327030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:38:15.327030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:15.327030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:38:15.327662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:38:15.327662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:38:15.327662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:38:15.327662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:38:15.327662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:15.327662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:15.327662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:38:15.327662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:38:15.327662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:38:15.327662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:38:15.327662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:15.327662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:15.327662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:15.327662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:38:15.327662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:15.327662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:15.327662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:15.327662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:38:15.327662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:15.327662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:38:15.330959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:38:15.330959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:15.330959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:38:15.330959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:15.330959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:38:15.330959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:38:15.330959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:15.330959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:38:15.330959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:15.330959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:38:15.332654 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:38:15.332900 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:38:15.333009 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:38:15.333077 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:38:15.333385 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:38:15.333557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:15.334321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:38:15.334321 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:15.334321 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:38:15.334321 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:38:15.334321 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:15.334321 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:38:15.334321 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:15.334321 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:38:15.334923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:15.334923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:38:15.334923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:38:15.334923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:38:15.334923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:15.334923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:38:15.334923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:15.334923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:38:15.334923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:15.334923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:38:15.334923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:15.334923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:38:15.334923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:15.334923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:38:15.334923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:38:15.335805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:15.335805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:38:15.335805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:15.335805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:38:15.335805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:38:15.335805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:15.335805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:38:15.335805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:15.335805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:38:15.335805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:15.335805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:38:15.335805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:15.335805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:38:15.335805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:38:15.336645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:38:15.336645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:38:15.336645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:38:15.336645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:38:15.336645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:38:15.336645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:15.336645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:15.336645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:15.336645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:38:15.336645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:15.336645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:15.336645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:15.336645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:38:15.337346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:15.337346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:38:15.337346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:38:15.337346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:15.337346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:38:15.337346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:15.337346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:38:15.337346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:38:15.337346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:15.337346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:38:15.337346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:15.337346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:38:15.338036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:15.338036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:38:15.338036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:38:15.338036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:38:15.338036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:38:15.338036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:38:15.338036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:15.338036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:38:15.338036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:38:15.338036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:38:15.338036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:38:15.338036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:38:15.338036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:15.338036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:15.338036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:38:15.338036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:38:15.339569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:38:15.339569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:38:15.339569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:38:15.339569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:38:15.339569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:38:15.339569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:15.339569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:15.339569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:15.339569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:38:15.339569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:15.339569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:15.339569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:15.339569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:38:15.339569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:15.339569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:38:15.339569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:38:15.339569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:15.339569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:38:15.340316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:15.340316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:38:15.340316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:38:15.340316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:15.340316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:38:15.340316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:15.340316 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:38:15.340316 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:38:15.340316 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:38:15.340316 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:38:15.340570 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:38:15.341056 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:38:15.341890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:15.342467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:38:15.342467 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:15.342467 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:38:15.342467 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:38:15.342467 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:15.342467 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:38:15.342467 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:15.342467 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:38:15.343664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:15.343664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:38:15.343664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:38:15.343664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:38:15.343664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:15.343664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:38:15.343664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:15.343664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:38:15.343664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:15.343664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:38:15.343664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:15.343664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:38:15.343664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:15.343664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:38:15.343664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:38:15.344008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:15.344008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:38:15.344008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:15.344008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:38:15.344008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:38:15.344008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:15.344008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:38:15.344008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:15.344008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:38:15.345213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:15.345213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:38:15.345213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:15.345213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:38:15.345213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:38:15.345213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:38:15.345213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:38:15.345213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:38:15.345213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:38:15.345213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:15.345213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:15.345509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:15.345509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:38:15.345509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:15.345509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:15.345509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:15.345509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:38:15.345509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:15.345509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:38:15.345509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:38:15.345509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:15.345509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:38:15.345509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:15.345509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:38:15.345509 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:38:15.345784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:15.345784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:38:15.345784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:15.345784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:38:15.345784 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:38:15.347205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:15.347205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:38:15.347205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:38:15.347205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:38:15.347205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:38:15.347205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:38:15.347205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:15.347205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:38:15.347205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:38:15.347205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:38:15.347205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:38:15.347205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:38:15.347205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:15.347205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:15.347205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:38:15.347205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:38:15.347205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:38:15.347595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:38:15.347595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:38:15.347595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:38:15.347595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:15.347595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:15.347595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:15.347595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:38:15.347595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:15.347595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:15.347595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:15.347595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:38:15.347595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:15.347595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:38:15.347595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:38:15.347595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:15.347595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:38:15.347595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:15.347595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:38:15.347595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:38:15.347960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:15.347960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:38:15.347960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:15.347960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:38:15.347960 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:38:15.347960 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:38:15.347960 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:38:15.348128 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:38:15.348272 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:38:15.348395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:15.348889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:38:15.348889 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:15.348889 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:38:15.348889 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:38:15.348889 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:15.348889 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:38:15.348889 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:15.348889 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:38:15.349739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:15.349739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:38:15.349739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:15.349739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:38:15.349739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:38:15.349739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:38:15.349739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:38:15.349739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:15.349739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:15.349739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:15.349739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:38:15.349739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:15.350113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:15.350113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:15.350113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:38:15.350113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:15.350113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:38:15.350113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:38:15.350113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:15.350113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:38:15.350113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:15.350113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:38:15.350113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:38:15.350113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:15.350113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:38:15.350611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:15.350611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:38:15.350722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:15.350722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:38:15.350722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:38:15.350722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:38:15.350722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:38:15.350722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:38:15.350722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:15.350722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:38:15.350722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:38:15.350722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:38:15.350722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:38:15.350722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:38:15.350722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:15.350722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:15.350722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:38:15.350722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:38:15.351178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:38:15.351178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:38:15.351178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:15.351178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:15.351178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:15.351178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:38:15.351178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:15.351178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:15.351178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:15.351178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:38:15.351178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:15.351178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:38:15.351178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:38:15.351178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:15.351178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:38:15.351178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:15.351178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:38:15.351178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:38:15.351178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:15.351694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:38:15.351694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:15.351694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:38:20.084795 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:38:20.085832 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:38:20.085832 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:38:20.086960 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:38:20.122789 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:38:20.123034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:20.123632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:38:20.123632 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:20.123632 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:38:20.123632 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:38:20.123632 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:20.123632 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:38:20.123632 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:20.123632 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:38:20.124554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:20.124554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:38:20.124554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:20.124554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:38:20.124554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:38:20.124554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:38:20.124554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:38:20.124554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:20.124554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:20.124554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:20.124554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:38:20.124554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:20.125195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:20.125195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:20.125195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:38:20.125195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:20.125195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:38:20.125195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:38:20.125195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:20.125195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:38:20.125195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:20.125195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:38:20.125195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:38:20.125195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:20.125195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:38:20.125881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:20.125881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:38:20.125881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:20.125881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:38:20.125881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:38:20.125881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:38:20.125881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:38:20.125881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:38:20.125881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:20.125881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:38:20.125881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:38:20.125881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:38:20.125881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:38:20.125881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:38:20.125881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:20.126717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:20.126717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:38:20.126717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:38:20.126717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:38:20.126717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:38:20.126717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:20.126717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:20.126717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:20.126717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:38:20.126717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:20.126717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:20.126717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:20.126717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:38:20.126717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:20.126717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:38:20.126717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:38:20.126717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:20.126717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:38:20.126717 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:20.127926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:38:20.127926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:38:20.127926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:20.127926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:38:20.127926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:20.127926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:38:20.320285 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:38:20.321064 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:38:20.321264 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:38:20.322637 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:38:20.323004 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:38:20.323335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:20.324126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:38:20.324126 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:20.324126 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:38:20.324126 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:38:20.324126 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:20.324126 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:38:20.324126 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:20.324126 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:38:20.325931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:20.325931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:38:20.325931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:20.325931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:38:20.325931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:38:20.325931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:38:20.325931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:38:20.325931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:20.325931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:20.325931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:20.325931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:38:20.325931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:20.326467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:20.326467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:20.326467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:38:20.326467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:20.326467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:38:20.326467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:38:20.326467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:20.326467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:38:20.326467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:20.326467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:38:20.326467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:38:20.326467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:20.326467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:38:20.326830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:20.326830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:38:20.327800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:20.327800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:38:20.327800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:38:20.327800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:38:20.327800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:38:20.327800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:38:20.327800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:20.327800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:38:20.327800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:38:20.327800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:38:20.327800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:38:20.327800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:38:20.327800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:20.327800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:20.327800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:38:20.327800 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:38:20.328266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:38:20.328266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:38:20.328266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:20.328266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:20.328266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:20.328266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:38:20.328266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:20.328266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:20.328266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:20.328266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:38:20.328266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:20.328266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:38:20.328266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:38:20.328266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:20.328266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:38:20.328266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:20.328266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:38:20.328266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:38:20.328266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:20.328815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:38:20.328815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:20.328815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:38:20.398632 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:38:20.398632 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:38:20.398632 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:38:20.398632 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:38:20.398632 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:38:20.398632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:20.398960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:38:20.398960 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:20.398960 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:38:20.398960 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:38:20.398960 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:20.398960 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:38:20.398960 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:20.398960 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:38:20.398960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:20.398960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:38:20.398960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:38:20.398960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:38:20.398960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:20.398960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:38:20.398960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:20.398960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:38:20.398960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:20.398960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:38:20.399507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:20.399507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:38:20.399507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:20.399507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:38:20.399507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:38:20.399507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:20.399507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:38:20.399507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:20.399507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:38:20.399507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:38:20.399507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:20.399507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:38:20.399507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:20.399507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:38:20.399507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:20.399939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:38:20.399939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:20.399939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:38:20.399939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:38:20.399939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:38:20.399939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:38:20.399939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:38:20.399939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:38:20.399939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:38:20.399939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:20.399939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:20.399939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:20.399939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:38:20.400327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:20.400327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:20.400327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:20.400327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:38:20.400327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:20.400327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:38:20.400327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:38:20.400327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:20.400327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:38:20.400327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:20.400327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:38:20.400327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:38:20.400327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:20.400327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:38:20.400724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:20.400724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:38:20.400724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:20.400724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:38:20.400724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:38:20.400724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:38:20.400724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:38:20.400724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:38:20.400724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:20.400724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:38:20.400724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:38:20.400724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:38:20.400724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:38:20.400724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:38:20.400724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:20.401160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:20.401160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:38:20.401160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:38:20.401160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:38:20.401160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:38:20.401160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:38:20.401160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:38:20.401160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:38:20.401160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:20.401160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:20.401160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:20.401160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:38:20.401160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:20.401160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:20.401160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:20.401160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:38:20.401160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:20.401160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:38:20.401160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:38:20.401705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:20.401705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:38:20.401705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:20.401705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:38:20.401705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:38:20.401705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:20.401705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:38:20.401705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:20.401705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:38:20.401705 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:38:20.401705 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:38:20.401705 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:38:20.402014 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:38:20.402014 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:38:20.402014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:20.402014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:38:20.402014 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:20.402014 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:38:20.402014 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:38:20.402014 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:20.402014 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:38:20.402014 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:20.402014 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:38:20.402014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:20.402287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:38:20.402287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:38:20.402287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:38:20.402287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:20.402287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:38:20.402287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:20.402287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:38:20.402287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:20.402287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:38:20.402287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:20.402287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:38:20.402287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:20.402287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:38:20.402287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:38:20.402287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:20.402287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:38:20.402287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:20.402665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:38:20.402665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:38:20.402665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:20.402665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:38:20.402665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:20.402665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:38:20.402665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:20.402665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:38:20.402665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:20.402665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:38:20.402665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:38:20.402665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:38:20.402665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:38:20.402972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:38:20.402972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:38:20.402972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:20.402972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:20.402972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:20.402972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:38:20.402972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:20.402972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:20.402972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:20.402972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:38:20.402972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:20.402972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:38:20.402972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:38:20.403247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:20.403247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:38:20.403247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:20.403247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:38:20.403247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:38:20.403247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:20.403247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:38:20.403247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:20.403247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:38:20.403247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:20.403247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:38:20.403247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:38:20.403531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:38:20.403531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:38:20.403531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:38:20.403531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:20.403531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:38:20.403531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:38:20.403531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:38:20.403531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:38:20.403531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:38:20.403531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:20.403531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:20.403531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:38:20.403531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:38:20.403531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:38:20.403531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:38:20.403531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:38:20.403531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:38:20.403531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:20.403531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:20.404030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:20.404030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:38:20.404030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:20.404030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:20.404030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:20.404030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:38:20.404030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:20.404030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:38:20.404030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:38:20.404030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:20.404030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:38:20.404030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:20.404030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:38:20.404030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:38:20.404030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:20.404030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:38:20.404030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:20.404030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:38:20.404367 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:38:20.404367 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:38:20.404367 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:38:20.404367 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:38:20.404367 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:38:20.404367 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:38:20.404561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:20.404561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:38:20.404561 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:20.404561 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:38:20.404561 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:38:20.404561 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:20.404561 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:38:20.404561 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:20.404561 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:38:20.404561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:20.404561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:38:20.404561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:20.404561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:38:20.404561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:38:20.404561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:38:20.404893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:38:20.404893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:20.404893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:20.404893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:20.404893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:38:20.404893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:20.404893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:20.404893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:20.404893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:38:20.404893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:20.404893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:38:20.404893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:38:20.404893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:20.404893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:38:20.405197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:20.405197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:38:20.405197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:38:20.405197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:20.405197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:38:20.405197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:20.405197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:38:20.405197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:20.405197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:38:20.405197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:38:20.405197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:38:20.405197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:38:20.405197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:38:20.405546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:20.405546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:38:20.405546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:38:20.405546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:38:20.405546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:38:20.405546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:38:20.405546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:20.405546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:20.405546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:38:20.405546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:38:20.405546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:38:20.405546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:38:20.405546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:20.405546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:20.405546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:20.405546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:38:20.405546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:20.405546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:20.405546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:20.405546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:38:20.405974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:20.405974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:38:20.405974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:38:20.405974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:20.405974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:38:20.405974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:20.405974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:38:20.405974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:38:20.405974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:20.405974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:38:20.405974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:20.405974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:38:25.085489 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:38:25.089603 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:38:25.089603 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:38:25.089603 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:38:25.090843 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG dbcounter [-] [105581] Writing DB stats neutron:SELECT=240 {{(pid=105581) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 14:38:25.124605 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:38:25.124989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:25.127319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:38:25.127319 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:25.127319 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:38:25.127319 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:38:25.127319 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:25.127319 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:38:25.127319 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:25.127319 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:38:25.127319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:25.127319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:38:25.127319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:25.127319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:38:25.127319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:38:25.127319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:38:25.127319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:38:25.127677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:25.127677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:25.127677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:25.127677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:38:25.127677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:25.127677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:25.127677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:25.127677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:38:25.127677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:25.127677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:38:25.127677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:38:25.127677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:25.127677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:38:25.127677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:25.127922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:38:25.127922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:38:25.127922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:25.127922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:38:25.127922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:25.127922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:38:25.127922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:25.127922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:38:25.127922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:38:25.127922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:38:25.127922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:38:25.127922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:38:25.127922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:25.127922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:38:25.128190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:38:25.128190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:38:25.128190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:38:25.128190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:38:25.128190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:25.128190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:25.128190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:38:25.128190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:38:25.128190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:38:25.128190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:38:25.128190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:25.128190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:25.128190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:25.128190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:38:25.128190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:25.128190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:25.128190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:25.128190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:38:25.128190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:25.128190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:38:25.128190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:38:25.128565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:25.128565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:38:25.128565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:25.128565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:38:25.128565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:38:25.128565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:25.128565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:38:25.128565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:25.128565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:38:25.321010 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:38:25.322998 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:38:25.322998 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:38:25.323753 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:38:25.324113 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:38:25.324374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:25.328452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:38:25.328452 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:25.328452 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:38:25.328452 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:38:25.328452 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:25.328452 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:38:25.328452 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:25.328452 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:38:25.328452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:25.328452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:38:25.328452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:25.328452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:38:25.328452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:38:25.328452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:38:25.328452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:38:25.330618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:25.330618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:25.330618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:25.330618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:38:25.330618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:25.330618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:25.330618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:25.330618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:38:25.330618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:25.330618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:38:25.330618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:38:25.330618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:25.330618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:38:25.330618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:25.331208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:38:25.331208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:38:25.331208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:25.331208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:38:25.331208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:25.331208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:38:25.332161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:25.332161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:38:25.332161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:38:25.332161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:38:25.332161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:38:25.332161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:38:25.332161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:25.332161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:38:25.332161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:38:25.332161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:38:25.332161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:38:25.332161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:38:25.332161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:25.332161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:25.332161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:38:25.332161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:38:25.332809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:38:25.332809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:38:25.332809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:25.332809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:25.332809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:25.332809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:38:25.332809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:25.332809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:25.332809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:25.332809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:38:25.332809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:25.332809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:38:25.332809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:38:25.332809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:25.332809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:38:25.332809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:25.332809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:38:25.332809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:38:25.332809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:25.334336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:38:25.334336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:25.334336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:38:25.334336 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:38:25.334336 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:38:25.334336 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:38:25.334336 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:38:25.334646 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:38:25.334857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:25.335573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:38:25.335573 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:25.335573 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:38:25.335573 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:38:25.335573 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:25.335573 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:38:25.335573 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:25.335573 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:38:25.337567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:25.337567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:38:25.337567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:38:25.337567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:38:25.337567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:25.337567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:38:25.337567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:25.337567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:38:25.337567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:25.337567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:38:25.337567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:25.337567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:38:25.337567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:25.337567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:38:25.337567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:38:25.338267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:25.338267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:38:25.338267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:25.338267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:38:25.338267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:38:25.338267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:25.338267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:38:25.338267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:25.338267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:38:25.338267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:25.338267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:38:25.338267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:25.338267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:38:25.338267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:38:25.338708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:38:25.338708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:38:25.338708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:38:25.338708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:38:25.338708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:38:25.338708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:25.338708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:25.338708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:25.338708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:38:25.338708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:25.338708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:25.338708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:25.338708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:38:25.339133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:25.339133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:38:25.339133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:38:25.339133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:25.339133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:38:25.339133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:25.339133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:38:25.339133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:38:25.339133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:25.339133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:38:25.339133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:25.339133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:38:25.341043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:25.341043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:38:25.341043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:38:25.341043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:38:25.341043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:38:25.341043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:38:25.341043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:25.341043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:38:25.341043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:38:25.341043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:38:25.341043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:38:25.341043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:38:25.341043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:25.341043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:25.341043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:38:25.341043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:38:25.341544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:38:25.341544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:38:25.341544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:38:25.341544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:38:25.341544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:38:25.341544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:25.341544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:25.341544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:25.341544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:38:25.341544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:25.341544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:25.341544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:25.341544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:38:25.341544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:25.341544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:38:25.341544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:38:25.341544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:25.341544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:38:25.342080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:25.342080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:38:25.342080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:38:25.342080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:25.342080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:38:25.342080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:25.342080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:38:25.342080 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:38:25.342080 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:38:25.342080 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:38:25.342376 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:38:25.343355 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:38:25.343355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:25.343577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:38:25.343577 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:25.343577 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:38:25.343577 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:38:25.343577 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:25.343577 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:38:25.343577 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:25.343577 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:38:25.344498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:25.344498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:38:25.344498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:38:25.344498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:38:25.344498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:25.344498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:38:25.344498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:25.344498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:38:25.344498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:25.344498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:38:25.344498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:25.344498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:38:25.344498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:25.344498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:38:25.344498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:38:25.344974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:25.344974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:38:25.344974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:25.344974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:38:25.344974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:38:25.344974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:25.344974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:38:25.344974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:25.344974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:38:25.345887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:25.345887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:38:25.345887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:25.345887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:38:25.345887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:38:25.345887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:38:25.345887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:38:25.345887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:38:25.345887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:38:25.345887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:25.345887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:25.346294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:25.346294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:38:25.346294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:25.346294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:25.346294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:25.346294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:38:25.346294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:25.346294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:38:25.346294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:38:25.346294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:25.346294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:38:25.346294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:25.346294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:38:25.346294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:38:25.346703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:25.346703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:38:25.346703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:25.346703 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:38:25.347121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:25.347121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:38:25.347121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:38:25.347121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:38:25.347121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:38:25.347121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:38:25.347121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:25.347121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:38:25.347121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:38:25.347121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:38:25.347121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:38:25.347121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:38:25.347121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:25.347121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:25.347121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:38:25.347121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:38:25.347121 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:38:25.347567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:38:25.347567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:38:25.347567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:38:25.347567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:25.347567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:25.347567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:25.347567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:38:25.347567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:25.347567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:25.347567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:25.347567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:38:25.347567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:25.347567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:38:25.347567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:38:25.347567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:25.347567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:38:25.347567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:25.347567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:38:25.347567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:38:25.348044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:25.348044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:38:25.348044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:25.348044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:38:25.348044 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:38:25.348184 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:38:25.348253 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:38:25.348364 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:38:25.349417 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:38:25.349417 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:38:25.349417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:25.349607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:38:25.349607 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:25.349607 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:38:25.349607 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:38:25.349607 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:25.349607 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:38:25.349607 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:25.349607 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:38:25.350357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:25.350357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:38:25.350357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:25.350357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:38:25.350357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:38:25.350357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:38:25.350357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:38:25.350357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:25.350357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:25.350357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:25.350357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:38:25.350357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:25.350745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:25.350745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:25.350745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:38:25.350745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:25.350745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:38:25.350745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:38:25.350745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:25.350745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:38:25.350745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:25.350745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:38:25.350745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:38:25.350745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:25.350745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:38:25.351078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:25.351078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:38:25.351579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:25.351579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:38:25.351579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:38:25.351579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:38:25.351579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:38:25.351579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:38:25.351579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:25.351579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:38:25.351579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:38:25.351579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:38:25.351579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:38:25.351579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:38:25.351579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:25.351579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:25.351579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:38:25.351579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:38:25.352010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:38:25.352010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:38:25.352010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:25.352010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:25.352010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:25.352010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:38:25.352010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:25.352010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:25.352010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:25.352010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:38:25.352010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:25.352010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:38:25.352010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:38:25.352010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:25.352010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:38:25.352010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:25.352010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:38:25.352010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:38:25.352010 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:25.352533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:38:25.352533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:25.352533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:38:30.086880 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:38:30.087823 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:38:30.087823 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:38:30.089345 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:38:30.126015 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:38:30.126323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:30.127138 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:38:30.127138 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:30.127138 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:38:30.127138 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:38:30.127138 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:30.127138 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:38:30.127138 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:30.127138 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:38:30.128348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:30.128348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:38:30.128348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:30.128348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:38:30.128348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:38:30.128348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:38:30.128348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:38:30.128348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:30.128348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:30.128348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:30.128348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:38:30.128348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:30.128790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:30.128790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:30.128790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:38:30.128790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:30.128790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:38:30.128790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:38:30.128790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:30.128790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:38:30.128790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:30.128790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:38:30.128790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:38:30.128790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:30.128790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:38:30.129243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:30.129243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:38:30.129754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:30.129754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:38:30.129754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:38:30.129754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:38:30.129754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:38:30.129754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:38:30.129754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:30.129754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:38:30.129754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:38:30.129754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:38:30.129754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:38:30.129754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:38:30.129754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:30.129754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:30.129754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:38:30.129754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:38:30.130284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:38:30.130284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:38:30.130284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:30.130284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:30.130284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:30.130284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:38:30.130284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:30.130284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:30.130284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:30.130284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:38:30.130284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:30.130284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:38:30.130284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:38:30.130284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:30.130284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:38:30.130284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:30.130284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:38:30.130284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:38:30.130284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:30.131015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:38:30.131015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:30.131015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:38:30.321995 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:38:30.322606 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:38:30.322801 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:38:30.323675 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:38:30.323945 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:38:30.324108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:30.324792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:38:30.324792 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:30.324792 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:38:30.324792 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:38:30.324792 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:30.324792 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:38:30.324792 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:30.324792 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:38:30.326477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:30.326477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:38:30.326477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:30.326477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:38:30.326477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:38:30.326477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:38:30.326477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:38:30.326477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:30.326477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:30.326477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:30.326477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:38:30.326477 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:30.326798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:30.326798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:30.326798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:38:30.326798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:30.326798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:38:30.326798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:38:30.326798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:30.326798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:38:30.326798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:30.326798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:38:30.326798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:38:30.326798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:30.326798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:38:30.327069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:30.327069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:38:30.331441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:30.331441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:38:30.331441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:38:30.331441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:38:30.331441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:38:30.331441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:38:30.331441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:30.331441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:38:30.331441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:38:30.331441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:38:30.331441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:38:30.331441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:38:30.331441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:30.331441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:30.331441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:38:30.331441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:38:30.331894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:38:30.331894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:38:30.331894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:30.331894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:30.331894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:30.331894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:38:30.331894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:30.331894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:30.331894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:30.331894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:38:30.331894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:30.331894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:38:30.331894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:38:30.331894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:30.331894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:38:30.331894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:30.331894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:38:30.331894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:38:30.331894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:30.332300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:38:30.332300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:30.332300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:38:30.334136 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:38:30.334459 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:38:30.334572 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:38:30.334706 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:38:30.335659 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:38:30.335659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:30.336286 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:38:30.336286 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:30.336286 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:38:30.336286 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:38:30.336286 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:30.336286 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:38:30.336286 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:30.336286 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:38:30.337100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:30.337100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:38:30.337100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:38:30.337100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:38:30.337100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:30.337100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:38:30.337100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:30.337100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:38:30.337100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:30.337100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:38:30.337100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:30.337100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:38:30.337100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:30.337100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:38:30.337100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:38:30.337536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:30.337536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:38:30.337536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:30.337536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:38:30.337536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:38:30.337536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:30.337536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:38:30.337536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:30.337536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:38:30.340592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:30.340592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:38:30.340592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:30.340592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:38:30.340592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:38:30.340592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:38:30.340592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:38:30.340592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:38:30.340592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:38:30.340592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:38:30.340592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:30.340592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:30.340942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:30.340942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:38:30.340942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:30.340942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:30.340942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:30.340942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:38:30.340942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:30.340942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:38:30.340942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:38:30.340942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:30.340942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:38:30.340942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:30.340942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:38:30.340942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:38:30.341244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:30.341244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:38:30.341244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:30.341244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:38:30.341244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:30.341244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:38:30.341244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:38:30.341244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:38:30.341244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:38:30.341244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:38:30.341244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:30.341244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:38:30.341244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:38:30.341244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:38:30.341575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:38:30.341575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:38:30.341575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:30.341575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:30.341575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:38:30.341575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:38:30.341575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:38:30.341575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:38:30.341575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:38:30.341575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:38:30.341575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:38:30.341575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:30.341575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:30.341575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:30.341575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:38:30.341575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:30.341575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:30.341575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:30.341575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:38:30.342143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:30.342143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:38:30.342143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:38:30.342143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:30.342143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:38:30.342143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:30.342143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:38:30.342143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:38:30.342143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:30.342143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:38:30.342143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:30.342143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:38:30.342143 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:38:30.342557 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:38:30.342557 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:38:30.342557 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:38:30.343256 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:38:30.343514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:30.344063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:38:30.344063 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:30.344063 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:38:30.344063 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:38:30.344063 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:30.344063 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:38:30.344063 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:30.344063 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:38:30.344795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:30.344795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:38:30.344795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:38:30.344795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:38:30.344795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:30.344795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:38:30.344795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:30.344795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:38:30.344795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:30.344795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:38:30.344795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:30.344795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:38:30.344795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:30.344795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:38:30.344795 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:38:30.345147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:30.345147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:38:30.345147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:30.345147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:38:30.345147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:38:30.345147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:30.345147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:38:30.345147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:30.345147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:38:30.345656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:30.345656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:38:30.345656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:30.345656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:38:30.345656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:38:30.345656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:38:30.345656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:38:30.345656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:38:30.345656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:38:30.345656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:30.345656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:30.345935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:30.345935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:38:30.345935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:30.345935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:30.345935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:30.345935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:38:30.345935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:30.345935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:38:30.345935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:38:30.345935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:30.345935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:38:30.345935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:30.345935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:38:30.345935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:38:30.346229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:30.346229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:38:30.346229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:30.346229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:38:30.346718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:30.346718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:38:30.346718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:38:30.346718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:38:30.346718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:38:30.346718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:38:30.346718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:30.346718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:38:30.346718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:38:30.346718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:38:30.346718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:38:30.346718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:38:30.346718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:30.346718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:30.346718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:38:30.346718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:38:30.346718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:38:30.347066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:38:30.347066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:38:30.347066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:38:30.347066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:30.347066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:30.347066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:30.347066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:38:30.347066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:30.347066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:30.347066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:30.347066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:38:30.347066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:30.347066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:38:30.347066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:38:30.347066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:30.347066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:38:30.347066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:30.347066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:38:30.347066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:38:30.347480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:30.347480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:38:30.347480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:30.347480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:38:30.348567 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:38:30.348853 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:38:30.348853 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:38:30.349011 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:38:30.349617 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:38:30.349854 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:38:30.352360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:30.352360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:38:30.352360 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:30.352360 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:38:30.352360 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:38:30.352360 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:30.352360 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:38:30.352360 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:30.352360 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:38:30.352360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:30.352360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:38:30.352360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:30.352360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:38:30.352360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:38:30.352360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:38:30.352983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:38:30.352983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:30.352983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:30.352983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:30.352983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:38:30.352983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:30.352983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:30.352983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:30.352983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:38:30.352983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:30.352983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:38:30.352983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:38:30.352983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:30.352983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:38:30.353380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:30.353380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:38:30.353380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:38:30.353380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:30.353380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:38:30.353380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:30.353380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:38:30.353380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:30.353380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:38:30.353380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:38:30.353380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:38:30.353380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:38:30.353380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:38:30.353928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:30.353928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:38:30.353928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:38:30.353928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:38:30.353928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:38:30.353928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:38:30.353928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:30.353928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:30.353928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:38:30.353928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:38:30.353928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:38:30.353928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:38:30.353928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:30.353928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:30.353928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:30.353928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:38:30.353928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:30.353928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:30.353928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:30.353928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:38:30.354553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:30.354553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:38:30.354553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:38:30.354553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:30.354553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:38:30.354553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:30.354553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:38:30.354553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:38:30.354553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:30.354553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:38:30.354553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:30.354553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:38:35.087554 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:38:35.088397 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:38:35.088397 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:38:35.089715 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:38:35.122076 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:38:35.122537 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:35.123424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:38:35.123424 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:35.123424 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:38:35.123424 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:38:35.123424 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:35.123424 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:38:35.123424 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:35.123424 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:38:35.124347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:35.124347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:38:35.124347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:35.124347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:38:35.124347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:38:35.124347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:38:35.124347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:38:35.124347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:35.124347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:35.124347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:35.124347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:38:35.124347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:35.125026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:35.125026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:35.125026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:38:35.125026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:35.125026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:38:35.125026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:38:35.125026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:35.125026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:38:35.125026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:35.125026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:38:35.125026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:38:35.125026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:35.125026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:38:35.125598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:35.125598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:38:35.125598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:35.125598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:38:35.125598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:38:35.125598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:38:35.125598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:38:35.125598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:38:35.125598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:35.125598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:38:35.125598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:38:35.125598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:38:35.125598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:38:35.125598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:38:35.125598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:35.126294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:35.126294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:38:35.126294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:38:35.126294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:38:35.126294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:38:35.126294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:35.126294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:35.126294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:35.126294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:38:35.126294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:35.126294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:35.126294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:35.126294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:38:35.126294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:35.126294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:38:35.126294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:38:35.126294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:35.126294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:38:35.126294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:35.127163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:38:35.127163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:38:35.127163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:35.127163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:38:35.127163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:35.127163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:38:35.322657 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:38:35.323314 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:38:35.323474 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:38:35.324493 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:38:35.324730 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:38:35.324888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:35.325614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:38:35.325614 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:35.325614 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:38:35.325614 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:38:35.325614 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:35.325614 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:38:35.325614 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:35.325614 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:38:35.326939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:35.326939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:38:35.326939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:35.326939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:38:35.326939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:38:35.326939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:38:35.326939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:38:35.326939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:35.326939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:35.326939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:35.326939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:38:35.326939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:35.327611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:35.327611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:35.327611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:38:35.327611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:35.327611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:38:35.327611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:38:35.327611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:35.327611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:38:35.327611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:35.327611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:38:35.327611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:38:35.327611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:35.327611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:38:35.328285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:35.328285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:38:35.328285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:35.328285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:38:35.328285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:38:35.328285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:38:35.328285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:38:35.328285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:38:35.328285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:35.328285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:38:35.328285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:38:35.328285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:38:35.328285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:38:35.328285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:38:35.328285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:35.328995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:35.328995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:38:35.328995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:38:35.328995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:38:35.328995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:38:35.328995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:35.328995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:35.328995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:35.328995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:38:35.328995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:35.328995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:35.328995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:35.328995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:38:35.328995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:35.328995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:38:35.328995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:38:35.328995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:35.328995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:38:35.328995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:35.329655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:38:35.329655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:38:35.329655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:35.329655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:38:35.329655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:35.329655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:38:35.334651 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:38:35.334924 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:38:35.335037 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:38:35.335160 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:38:35.335609 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:38:35.335839 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:35.336421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:38:35.336421 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:35.336421 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:38:35.336421 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:38:35.336421 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:35.336421 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:38:35.336421 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:35.336421 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:38:35.337691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:35.337691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:38:35.337691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:38:35.337691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:38:35.337691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:35.337691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:38:35.337691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:35.337691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:38:35.337691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:35.337691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:38:35.337691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:35.337691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:38:35.337691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:35.337691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:38:35.337691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:38:35.338136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:35.338136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:38:35.338136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:35.338136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:38:35.338136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:38:35.338136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:35.338136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:38:35.338136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:35.338136 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:38:35.338481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:35.338481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:38:35.338481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:35.338481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:38:35.338481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:38:35.338481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:38:35.338481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:38:35.338481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:38:35.338481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:38:35.338481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:38:35.338481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:35.338481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:35.338856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:35.338856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:38:35.338856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:35.338856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:35.338856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:35.338856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:38:35.338856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:35.338856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:38:35.338856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:38:35.338856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:35.338856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:38:35.338856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:35.338856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:38:35.338856 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:38:35.339218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:35.339218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:38:35.339218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:35.339218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:38:35.340619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:35.340619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:38:35.340619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:38:35.340619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:38:35.340619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:38:35.340619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:38:35.340619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:35.340619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:38:35.340619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:38:35.340619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:38:35.340619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:38:35.340619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:38:35.340619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:35.340619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:35.340619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:38:35.340619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:38:35.341095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:38:35.341095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:38:35.341095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:38:35.341095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:38:35.341095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:38:35.341095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:35.341095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:35.341095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:35.341095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:38:35.341095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:35.341095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:35.341095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:35.341095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:38:35.341095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:35.341095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:38:35.341095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:38:35.341095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:35.341095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:38:35.341600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:35.341600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:38:35.341600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:38:35.341600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:35.341600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:38:35.341600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:35.341600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:38:35.342691 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:38:35.344004 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:38:35.344126 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:38:35.344269 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:38:35.344641 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:38:35.344859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:35.345354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:38:35.345354 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:35.345354 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:38:35.345354 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:38:35.345354 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:35.345354 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:38:35.345354 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:35.345354 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:38:35.346236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:35.346236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:38:35.346236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:38:35.346236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:38:35.346236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:35.346236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:38:35.346236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:35.346236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:38:35.346236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:35.346236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:38:35.346236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:35.346236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:38:35.346236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:35.346236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:38:35.346236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:38:35.346694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:35.346694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:38:35.346694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:35.346694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:38:35.346694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:38:35.346694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:35.346694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:38:35.346694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:35.346694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:38:35.347111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:35.347111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:38:35.347111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:35.347111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:38:35.347111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:38:35.347111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:38:35.347111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:38:35.347111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:38:35.347111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:38:35.347111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:35.347111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:35.347462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:35.347462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:38:35.347462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:35.347462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:35.347462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:35.347462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:38:35.347462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:35.347462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:38:35.347462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:38:35.347462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:35.347462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:38:35.347462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:35.347462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:38:35.347462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:38:35.347882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:35.347882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:38:35.347882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:35.347882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:38:35.348676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:35.348676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:38:35.348676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:38:35.348676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:38:35.348676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:38:35.348676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:38:35.348676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:35.348676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:38:35.348676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:38:35.348676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:38:35.348676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:38:35.348676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:38:35.348676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:35.348676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:35.348676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:38:35.348676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:38:35.348676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:38:35.349151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:38:35.349151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:38:35.349151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:38:35.349151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:35.349151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:35.349151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:35.349151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:38:35.349151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:35.349151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:35.349151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:35.349151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:38:35.349151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:35.349151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:38:35.349151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:38:35.349151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:35.349151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:38:35.349151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:35.349151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:38:35.349151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:38:35.349705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:35.349705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:38:35.349705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:35.349705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:38:35.349705 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:38:35.349705 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:38:35.349705 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:38:35.349705 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:38:35.349931 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:38:35.349931 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:38:35.350024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:35.350543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:38:35.350543 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:35.350543 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:38:35.350543 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:38:35.350543 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:35.350543 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:38:35.350543 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:35.350543 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:38:35.351805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:35.351805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:38:35.351805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:35.351805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:38:35.351805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:38:35.351805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:38:35.351805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:38:35.351805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:35.351805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:35.351805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:35.351805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:38:35.351805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:35.352202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:35.352202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:35.352202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:38:35.352202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:35.352202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:38:35.352202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:38:35.352202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:35.352202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:38:35.352202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:35.352202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:38:35.352202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:38:35.352202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:35.352202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:38:35.352572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:35.352572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:38:35.352792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:35.352792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:38:35.352792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:38:35.352792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:38:35.352792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:38:35.352792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:38:35.352792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:35.352792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:38:35.352792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:38:35.352792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:38:35.352792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:38:35.352792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:38:35.352792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:35.352792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:35.352792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:38:35.352792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:38:35.353238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:38:35.353238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:38:35.353238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:35.353238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:35.353238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:35.353238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:38:35.353238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:35.353238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:35.353238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:35.353238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:38:35.353238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:35.353238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:38:35.353238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:38:35.353238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:35.353238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:38:35.353238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:35.353238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:38:35.353238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:38:35.353238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:35.353728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:38:35.353728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:35.353728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:38:40.088095 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:38:40.089151 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:38:40.089151 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:38:40.089284 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:38:40.114571 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:38:40.114773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:40.115298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:38:40.115298 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:40.115298 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:38:40.115298 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:38:40.115298 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:40.115298 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:38:40.115298 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:40.115298 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:38:40.116056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.027 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:40.116056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:38:40.116056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:40.116056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:38:40.116056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:38:40.116056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:38:40.116056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:38:40.116056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:40.116056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:40.116056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:40.116056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:38:40.116056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:40.116363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:40.116363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:40.116363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:38:40.116363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:40.116363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:38:40.116363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:38:40.116363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:40.116363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:38:40.116363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:40.116363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:38:40.116363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:38:40.116363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:40.116363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:38:40.116689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:40.116689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:38:40.117513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:40.117513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:38:40.117513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:38:40.117513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:38:40.117513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:38:40.117513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:38:40.117513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:40.117513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:38:40.117513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:38:40.117513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:38:40.117513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:38:40.117513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:38:40.117513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:40.117513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:40.117513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:38:40.117513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:38:40.118602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:38:40.118602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:38:40.118602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:40.118602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:40.118602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:40.118602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:38:40.118602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:40.118602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:40.118602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:40.118602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:38:40.118602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:40.118602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:38:40.118602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:38:40.118602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:40.118602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:38:40.118602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:40.118602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:38:40.118602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:38:40.118602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:40.119187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:38:40.119187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:40.119187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:38:40.323246 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:38:40.323802 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:38:40.323991 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:38:40.324975 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:38:40.325467 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:38:40.325467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:40.326124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:38:40.326124 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:40.326124 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:38:40.326124 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:38:40.326124 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:40.326124 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:38:40.326124 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:40.326124 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:38:40.328702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:40.328702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:38:40.328702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:40.328702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:38:40.328702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:38:40.328702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:38:40.328702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:38:40.328702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:40.328702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:40.328702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:40.328702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:38:40.328702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:40.329792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:40.329792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:40.329792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:38:40.329792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:40.329792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:38:40.329792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:38:40.329792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:40.329792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:38:40.329792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:40.329792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:38:40.329792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:38:40.329792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:40.329792 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:38:40.330353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:40.330353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:38:40.330353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:40.330353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:38:40.330353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:38:40.330353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:38:40.330353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:38:40.330353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:38:40.330353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:40.330353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:38:40.330353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:38:40.330353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:38:40.330353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:38:40.330353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:38:40.330353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:40.332820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:40.332820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:38:40.332820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:38:40.332820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:38:40.332820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:38:40.332820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:40.332820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:40.332820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:40.332820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:38:40.332820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:40.332820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:40.332820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:40.332820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:38:40.332820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:40.332820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:38:40.332820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:38:40.332820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:40.332820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:38:40.332820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:40.333699 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:38:40.333699 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:38:40.333699 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:40.333699 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:38:40.333699 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:40.333699 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:38:40.335018 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:38:40.335256 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:38:40.335378 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:38:40.335525 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:38:40.336030 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:38:40.336222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:40.336745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:38:40.336745 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:40.336745 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:38:40.336745 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:38:40.336745 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:40.336745 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:38:40.336745 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:40.336745 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:38:40.337558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:40.337558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:38:40.337558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:38:40.337558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:38:40.337558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:40.337558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:38:40.337558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:40.337558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:38:40.337558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:40.337558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:38:40.337558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:40.337558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:38:40.337558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:40.337558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:38:40.337558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:38:40.338751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:40.338751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:38:40.338751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:40.338751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:38:40.338751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:38:40.338751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:40.338751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:38:40.338751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:40.338751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:38:40.338751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:40.338751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:38:40.338751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:40.338751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:38:40.338751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:38:40.339532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:38:40.339532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:38:40.339532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:38:40.339532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:38:40.339532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:38:40.339532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:40.339532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:40.339532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:40.339532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:38:40.339532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:40.339532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:40.339532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:40.339532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:38:40.339875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:40.339875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:38:40.339875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:38:40.339875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:40.339875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:38:40.339875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:40.339875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:38:40.339875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:38:40.339875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:40.339875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:38:40.339875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:40.339875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:38:40.340178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:40.340178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:38:40.340178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:38:40.340178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:38:40.340178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:38:40.340178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:38:40.340178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:40.340178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:38:40.340178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:38:40.340178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:38:40.340178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:38:40.340178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:38:40.340178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:40.340178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:40.340178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:38:40.340178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:38:40.341841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:38:40.341841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:38:40.341841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:38:40.341841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:38:40.341841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:38:40.341841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:40.341841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:40.341841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:40.341841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:38:40.341841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:40.341841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:40.341841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:40.341841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:38:40.341841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:40.341841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:38:40.341841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:38:40.341841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:40.341841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:38:40.343003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:40.343003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:38:40.343003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:38:40.343003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:40.343003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:38:40.343003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:40.343003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:38:40.343905 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:38:40.344153 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:38:40.344273 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:38:40.344469 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:38:40.345097 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:38:40.345386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:40.345943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:38:40.345943 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:40.345943 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:38:40.345943 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:38:40.345943 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:40.345943 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:38:40.345943 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:40.345943 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:38:40.346726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:40.346726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:38:40.346726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:38:40.346726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:38:40.346726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:40.346726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:38:40.346726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:40.346726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:38:40.346726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:40.346726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:38:40.346726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:40.346726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:38:40.346726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:40.346726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:38:40.346726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:38:40.347127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:40.347127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:38:40.347127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:40.347127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:38:40.347127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:38:40.347127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:40.347127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:38:40.347127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:40.347127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:38:40.347817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:40.347817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:38:40.347817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:40.347817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:38:40.347817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:38:40.347817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:38:40.347817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:38:40.347817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:38:40.347817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:38:40.347817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:40.347817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:40.348821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:40.348821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:38:40.348821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:40.348821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:40.348821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:40.348821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:38:40.348821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:40.348821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:38:40.348821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:38:40.348821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:40.348821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:38:40.348821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:40.348821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:38:40.348821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:38:40.349271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:40.349271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:38:40.349271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:40.349271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:38:40.349271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:40.349271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:38:40.349271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:38:40.349271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:38:40.349271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:38:40.349271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:38:40.349271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:40.349271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:38:40.349271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:38:40.349271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:38:40.349637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:38:40.349637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:38:40.349637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:40.349637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:40.349637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:38:40.349637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:38:40.349637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:38:40.349637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:38:40.349637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:38:40.349637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:38:40.349637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:40.349637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:40.349637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:40.349637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:38:40.349637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:40.349637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:40.349637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:40.349637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:38:40.349637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:40.350710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:38:40.350710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:38:40.350710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:40.350710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:38:40.350710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:40.350710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:38:40.350710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:38:40.350710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:40.350710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:38:40.350710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:40.350710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:38:40.350710 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:38:40.351575 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:38:40.351575 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:38:40.351575 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:38:40.351575 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:38:40.351942 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:38:40.351942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:40.352226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:38:40.352226 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:40.352226 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:38:40.352226 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:38:40.352226 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:40.352226 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:38:40.352226 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:40.352226 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:38:40.353182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:40.353182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:38:40.353182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:40.353182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:38:40.353182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:38:40.353182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:38:40.353182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:38:40.353182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:40.353182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:40.353182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:40.353182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:38:40.353182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:40.353636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:40.353636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:40.353636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:38:40.353636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:40.353636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:38:40.353636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:38:40.353636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:40.353636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:38:40.353636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:40.353636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:38:40.353636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:38:40.353636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:40.353636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:38:40.353979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:40.353979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:38:40.354270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:40.354270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:38:40.354270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:38:40.354270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:38:40.354270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:38:40.354270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:38:40.354270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:40.354270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:38:40.354270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:38:40.354270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:38:40.354270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:38:40.354270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:38:40.354270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:40.354270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:40.354270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:38:40.354270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:38:40.355205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:38:40.355205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:38:40.355205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:40.355205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:40.355205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:40.355205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:38:40.355205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:40.355205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:40.355205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:40.355205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:38:40.355205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:40.355205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:38:40.355205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:38:40.355205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:40.355205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:38:40.355205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:40.355205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:38:40.355205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:38:40.355205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:40.355758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:38:40.355758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:40.355758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:38:45.088806 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:38:45.091731 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:38:45.091731 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:38:45.091731 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:38:45.126039 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:38:45.126310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:45.129098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:38:45.129098 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:45.129098 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:38:45.129098 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:38:45.129098 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:45.129098 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:38:45.129098 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:45.129098 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:38:45.129098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:45.129098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:38:45.129098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:45.129098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:38:45.129098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:38:45.129098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:38:45.129098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:38:45.129646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:45.129646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:45.129646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:45.129646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:38:45.129646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:45.129646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:45.129646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:45.129646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:38:45.129646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:45.129646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:38:45.129646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:38:45.129646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:45.129646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:38:45.129646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:45.130029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:38:45.130029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:38:45.130029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:45.130029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:38:45.130029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:45.130029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:38:45.130029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:45.130029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:38:45.130029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:38:45.130029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:38:45.130029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:38:45.130029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:38:45.130029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:45.130029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:38:45.130447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:38:45.130447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:38:45.130447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:38:45.130447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:38:45.130447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:45.130447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:45.130447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:38:45.130447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:38:45.130447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:38:45.130447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:38:45.130447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:45.130447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:45.130447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:45.130447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:38:45.130447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:45.130447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:45.130447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:45.130447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:38:45.130447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:45.130447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:38:45.130447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:38:45.131032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:45.131032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:38:45.131032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:45.131032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:38:45.131032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:38:45.131032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:45.131032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:38:45.131032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:45.131032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:38:45.324002 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:38:45.325119 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:38:45.325119 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:38:45.325642 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:38:45.325897 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:38:45.326092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:45.329171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:38:45.329171 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:45.329171 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:38:45.329171 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:38:45.329171 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:45.329171 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:38:45.329171 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:45.329171 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:38:45.329171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:45.329171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:38:45.329171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:45.329171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:38:45.329171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:38:45.329171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:38:45.329171 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:38:45.329805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:45.329805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:45.329805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:45.329805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:38:45.329805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:45.329805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:45.329805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:45.329805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:38:45.329805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:45.329805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:38:45.329805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:38:45.329805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:45.329805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:38:45.329805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:45.330159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:38:45.330159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:38:45.330159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:45.330159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:38:45.330159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:45.330159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:38:45.330159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:45.330159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:38:45.330159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:38:45.330159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:38:45.330159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:38:45.330159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:38:45.330159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:45.330159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:38:45.330550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:38:45.330550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:38:45.330550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:38:45.330550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:38:45.330550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:45.330550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:45.330550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:38:45.330550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:38:45.330550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:38:45.330550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:38:45.330550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:45.330550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:45.330550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:45.330550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:38:45.330550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:45.330550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:45.330550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:45.330550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:38:45.330550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:45.330550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:38:45.331084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:38:45.331084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:45.331084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:38:45.331084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:45.331084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:38:45.331084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:38:45.331084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:45.331084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:38:45.331084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:45.331084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:38:45.335577 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:38:45.336296 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:38:45.336296 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:38:45.336296 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:38:45.336592 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:38:45.336810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:45.340207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:38:45.340207 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:45.340207 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:38:45.340207 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:38:45.340207 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:45.340207 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:38:45.340207 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:45.340207 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:38:45.340207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:45.340207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:38:45.340207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:38:45.340207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:38:45.340207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:45.340207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:38:45.340207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:45.340207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:38:45.340207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:45.340207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:38:45.340878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:45.340878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:38:45.340878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:45.340878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:38:45.340878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:38:45.340878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:45.340878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:38:45.340878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:45.340878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:38:45.340878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:38:45.340878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:45.340878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:38:45.340878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:45.340878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:38:45.340878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:45.341550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:38:45.341550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:45.341550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:38:45.341550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:38:45.341550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:38:45.341550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:38:45.341550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:38:45.341550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:38:45.341550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:38:45.341550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:45.341550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:45.341550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:45.341550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:38:45.342139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:45.342139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:45.342139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:45.342139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:38:45.342139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:45.342139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:38:45.342139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:38:45.342139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:45.342139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:38:45.342139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:45.342139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:38:45.342139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:38:45.342139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:45.342139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:38:45.342544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:45.342544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:38:45.342544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:45.342544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:38:45.342544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:38:45.342544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:38:45.342544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:38:45.342544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:38:45.342544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:45.342544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:38:45.342544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:38:45.342544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:38:45.342544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:38:45.342544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:38:45.342544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:45.343041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:45.343041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:38:45.343041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:38:45.343041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:38:45.343041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:38:45.343041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:38:45.343041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:38:45.343041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:38:45.343041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:45.343041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:45.343041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:45.343041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:38:45.343041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:45.343041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:45.343041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:45.343041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:38:45.343041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:45.343041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:38:45.343041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:38:45.343639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:45.343639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:38:45.343639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:45.343639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:38:45.343639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:38:45.343639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:45.343639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:38:45.343639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:45.343639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:38:45.344357 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:38:45.344992 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:38:45.344992 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:38:45.344992 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:38:45.345378 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:38:45.345754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:45.349143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:38:45.349143 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:45.349143 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:38:45.349143 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:38:45.349143 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:45.349143 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:38:45.349143 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:45.349143 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:38:45.349143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:45.349143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:38:45.349143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:38:45.349143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:38:45.349143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:45.349143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:38:45.349143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:45.349143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:38:45.349143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:45.349143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:38:45.349694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:45.349694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:38:45.349694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:45.349694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:38:45.349694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:38:45.349694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:45.349694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:38:45.349694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:45.349694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:38:45.349694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:38:45.349694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:45.349694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:38:45.349694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:45.349694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:38:45.349694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:45.350074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:38:45.350074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:45.350074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:38:45.350074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:38:45.350074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:38:45.350074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:38:45.350074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:38:45.350074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:38:45.350074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:45.350074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:45.350074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:45.350074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:38:45.350074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:45.350334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:45.350334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:45.350334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:38:45.350334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:45.350334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:38:45.350334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:38:45.350334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:45.350334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:38:45.350334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:45.350334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:38:45.350334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:38:45.350334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:45.350334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:38:45.350590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:45.350590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:38:45.350590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:45.350590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:38:45.350590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:38:45.350590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:38:45.350590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:38:45.350590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:38:45.350590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:45.350590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:38:45.350590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:38:45.350590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:38:45.350590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:38:45.350590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:38:45.350590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:45.350911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:45.350911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:38:45.350911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:38:45.350911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:38:45.350911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:38:45.350911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:38:45.350911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:38:45.350911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:45.350911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:45.350911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:45.350911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:38:45.350911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:45.350911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:45.350911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:45.350911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:38:45.350911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:45.350911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:38:45.350911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:38:45.350911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:45.351341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:38:45.351341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:45.351341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:38:45.351341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:38:45.351341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:45.351341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:38:45.351341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:45.351341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:38:45.351341 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:38:45.351758 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:38:45.351902 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:38:45.352083 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:38:45.352481 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:38:45.352710 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:38:45.352865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:45.353459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:38:45.353459 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:45.353459 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:38:45.353459 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:38:45.353459 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:45.353459 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:38:45.353459 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:45.353459 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:38:45.354426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:45.354426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:38:45.354426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:45.354426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:38:45.354426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:38:45.354426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:38:45.354426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:38:45.354426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:45.354426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:45.354426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:45.354426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:38:45.354426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:45.355454 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:45.355454 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:45.355454 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:38:45.355454 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:45.355454 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:38:45.355454 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:38:45.355454 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:45.355454 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:38:45.355454 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:45.355454 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:38:45.355454 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:38:45.355454 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:45.355454 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:38:45.355741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:45.355741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:38:45.355741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:45.355741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:38:45.355741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:38:45.355741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:38:45.355741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:38:45.355741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:38:45.355741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:45.355741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:38:45.355741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:38:45.355741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:38:45.355741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:38:45.355741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:38:45.355741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:45.356075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:45.356075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:38:45.356075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:38:45.356075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:38:45.356075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:38:45.356075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:45.356075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:45.356075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:45.356075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:38:45.356075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:45.356075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:45.356075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:45.356075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:38:45.356075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:45.356075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:38:45.356075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:38:45.356075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:45.356075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:38:45.356075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:45.356471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:38:45.356471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:38:45.356471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:45.356471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:38:45.356471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:45.356471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:38:50.090007 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:38:50.090741 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:38:50.090741 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:38:50.091274 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:38:50.122287 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:38:50.122469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:50.123045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:38:50.123045 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:50.123045 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:38:50.123045 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:38:50.123045 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:50.123045 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:38:50.123045 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:50.123045 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:38:50.124032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:50.124032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:38:50.124032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:50.124032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:38:50.124032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:38:50.124032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:38:50.124032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:38:50.124032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:50.124032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:50.124032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:50.124032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:38:50.124032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:50.124796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:50.124796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:50.124796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:38:50.124796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:50.124796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:38:50.124796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:38:50.124796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:50.124796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:38:50.124796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:50.124796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:38:50.124796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:38:50.124796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:50.124796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:38:50.125505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:50.125505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:38:50.125505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:50.125505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:38:50.125505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:38:50.125505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:38:50.125505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:38:50.125505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:38:50.125505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:50.125505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:38:50.125505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:38:50.125505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:38:50.125505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:38:50.125505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:38:50.125505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:50.126609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:50.126609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:38:50.126609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:38:50.126609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:38:50.126609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:38:50.126609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:50.126609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:50.126609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:50.126609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:38:50.126609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:50.126609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:50.126609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:50.126609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:38:50.126609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:50.126609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:38:50.126609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:38:50.126609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:50.126609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:38:50.126609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:50.127849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:38:50.127849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:38:50.127849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:50.127849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:38:50.127849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:50.127849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:38:50.324615 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:38:50.324878 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:38:50.324962 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:38:50.325967 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:38:50.326378 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:38:50.326652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:50.327652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:38:50.327652 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:50.327652 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:38:50.327652 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:38:50.327652 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:50.327652 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:38:50.327652 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:50.327652 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:38:50.328770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:50.328770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:38:50.328770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:50.328770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:38:50.328770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:38:50.328770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:38:50.328770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:38:50.328770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:50.328770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:50.328770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:50.328770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:38:50.328770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:50.329375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:50.329375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:50.329375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:38:50.329375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:50.329375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:38:50.329375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:38:50.329375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:50.329375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:38:50.329375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:50.329375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:38:50.329375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:38:50.329375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:50.329375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:38:50.330229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:50.330229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:38:50.330229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:50.330229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:38:50.330229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:38:50.330229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:38:50.330229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:38:50.330229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:38:50.330229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:50.330229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:38:50.330229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:38:50.330229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:38:50.330229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:38:50.330229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:38:50.330229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:50.330972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:50.330972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:38:50.330972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:38:50.330972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:38:50.330972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:38:50.330972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:50.330972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:50.330972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:50.330972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:38:50.330972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:50.330972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:50.330972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:50.330972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:38:50.330972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:50.330972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:38:50.330972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:38:50.330972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:50.330972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:38:50.330972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:50.331883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:38:50.331883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:38:50.331883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:50.331883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:38:50.331883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:50.331883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:38:50.335958 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:38:50.336091 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:38:50.336664 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:38:50.336664 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:38:50.336765 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:38:50.337078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:50.337787 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:38:50.337787 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:50.337787 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:38:50.337787 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:38:50.337787 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:50.337787 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:38:50.337787 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:50.337787 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:38:50.338552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:50.338552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:38:50.338552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:38:50.338552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:38:50.338552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:50.338552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:38:50.338552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:50.338552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:38:50.338552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:50.338552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:38:50.338552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:50.338552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:38:50.338552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:50.338552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:38:50.338552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:38:50.339131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:50.339131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:38:50.339131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:50.339131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:38:50.339131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:38:50.339131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:50.339131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:38:50.339131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:50.339131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:38:50.339442 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:50.339442 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:38:50.339442 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:50.339442 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:38:50.339442 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:38:50.339442 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:38:50.339442 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:38:50.339442 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:38:50.339442 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:38:50.339442 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:38:50.339442 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:50.339442 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:50.339837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:50.339837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:38:50.339837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:50.339837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:50.339837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:50.339837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:38:50.339837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:50.339837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:38:50.339837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:38:50.339837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:50.339837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:38:50.339837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:50.339837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:38:50.339837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:38:50.340265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:50.340265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:38:50.340265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:50.340265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:38:50.340265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:50.340265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:38:50.340265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:38:50.340265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:38:50.340265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:38:50.340265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:38:50.340265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:50.340265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:38:50.340265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:38:50.340265 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:38:50.340687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:38:50.340687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:38:50.340687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:50.340687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:50.340687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:38:50.340687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:38:50.340687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:38:50.340687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:38:50.340687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:38:50.340687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:38:50.340687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:38:50.340687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:50.340687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:50.340687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:50.340687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:38:50.340687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:50.340687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:50.340687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:50.340687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:38:50.341231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:50.341231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:38:50.341231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:38:50.341231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:50.341231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:38:50.341231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:50.341231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:38:50.341231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:38:50.341231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:50.341231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:38:50.341231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:50.341231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:38:50.344730 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:38:50.344927 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:38:50.345034 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:38:50.345159 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:38:50.345616 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:38:50.345975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:50.346727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:38:50.346727 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:50.346727 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:38:50.346727 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:38:50.346727 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:50.346727 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:38:50.346727 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:50.346727 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:38:50.347420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:50.347420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:38:50.347420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:38:50.347420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:38:50.347420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:50.347420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:38:50.347420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:50.347420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:38:50.347420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:50.347420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:38:50.347420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:50.347420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:38:50.347420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:50.347420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:38:50.347420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:38:50.347851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:50.347851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:38:50.347851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:50.347851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:38:50.347851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:38:50.347851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:50.347851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:38:50.347851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:50.347851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:38:50.348252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:50.348252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:38:50.348252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:50.348252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:38:50.348252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:38:50.348252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:38:50.348252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:38:50.348252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:38:50.348252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:38:50.348252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:50.348252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:50.348612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:50.348612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:38:50.348612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:50.348612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:50.348612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:50.348612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:38:50.348612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:50.348612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:38:50.348612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:38:50.348612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:50.348612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:38:50.348612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:50.348612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:38:50.348612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:38:50.349020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:50.349020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:38:50.349020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:50.349020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:38:50.349255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:50.349255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:38:50.349255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:38:50.349255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:38:50.349255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:38:50.349255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:38:50.349255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:50.349255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:38:50.349255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:38:50.349255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:38:50.349255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:38:50.349255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:38:50.349255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:50.349255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:50.349255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:38:50.349255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:38:50.349255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:38:50.349773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:38:50.349773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:38:50.349773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:38:50.349773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:50.349773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:50.349773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:50.349773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:38:50.349773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:50.349773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:50.349773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:50.349773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:38:50.349773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:50.349773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:38:50.349773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:38:50.349773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:50.349773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:38:50.349773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:50.349773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:38:50.349773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:38:50.350346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:50.350346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:38:50.350346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:50.350346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:38:50.351788 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:38:50.351976 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:38:50.352074 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:38:50.352211 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:38:50.352652 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:38:50.353007 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:38:50.353254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:50.353958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:38:50.353958 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:50.353958 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:38:50.353958 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:38:50.353958 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:50.353958 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:38:50.353958 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:50.353958 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:38:50.354814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:50.354814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:38:50.354814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:50.354814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:38:50.354814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:38:50.354814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:38:50.354814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:38:50.354814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:50.354814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:50.354814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:50.354814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:38:50.354814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:50.355236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:50.355236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:50.355236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:38:50.355236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:50.355236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:38:50.355236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:38:50.355236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:50.355236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:38:50.355236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:50.355236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:38:50.355236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:38:50.355236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:50.355236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:38:50.355656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:50.355656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:38:50.355722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:50.355722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:38:50.355722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:38:50.355722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:38:50.355722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:38:50.355722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:38:50.355722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:50.355722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:38:50.355722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:38:50.355722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:38:50.355722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:38:50.355722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:38:50.355722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:50.355722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:50.355722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:38:50.355722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:38:50.356206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:38:50.356206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:38:50.356206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:50.356206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:50.356206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:50.356206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:38:50.356206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:50.356206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:50.356206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:50.356206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:38:50.356206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:50.356206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:38:50.356206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:38:50.356206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:50.356206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:38:50.356206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:50.356206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:38:50.356206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:38:50.356206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:50.356726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:38:50.356726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:50.356726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:38:55.090665 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:38:55.091606 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:38:55.091606 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:38:55.092500 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:38:55.127162 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:38:55.127626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:55.128526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:38:55.128526 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:55.128526 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:38:55.128526 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:38:55.128526 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:55.128526 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:38:55.128526 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:55.128526 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:38:55.129275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:55.129275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:38:55.129275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:55.129275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:38:55.129275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:38:55.129275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:38:55.129275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:38:55.129275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:55.129275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:55.129275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:55.129275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:38:55.129275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:55.129967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:55.129967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:55.129967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:38:55.129967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:55.129967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:38:55.129967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:38:55.129967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:55.129967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:38:55.129967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:55.129967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:38:55.129967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:38:55.129967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:55.129967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:38:55.130653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:55.130653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:38:55.130653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:55.130653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:38:55.130653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:38:55.130653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:38:55.130653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:38:55.130653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:38:55.130653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:55.130653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:38:55.130653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:38:55.130653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:38:55.130653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:38:55.130653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:38:55.130653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:55.131642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:55.131642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:38:55.131642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:38:55.131642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:38:55.131642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:38:55.131642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:55.131642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:55.131642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:55.131642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:38:55.131642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:55.131642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:55.131642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:55.131642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:38:55.131642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:55.131642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:38:55.131642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:38:55.131642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:55.131642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:38:55.131642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:55.132711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:38:55.132711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:38:55.132711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:55.132711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:38:55.132711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:55.132711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:38:55.325082 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:38:55.325363 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:38:55.325505 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:38:55.326256 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:38:55.326613 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:38:55.326840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:55.327745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:38:55.327745 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:55.327745 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:38:55.327745 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:38:55.327745 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:55.327745 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:38:55.327745 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:55.327745 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:38:55.328579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:55.328579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:38:55.328579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:55.328579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:38:55.328579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:38:55.328579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:38:55.328579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:38:55.328579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:55.328579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:55.328579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:55.328579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:38:55.328579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:55.328860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:55.328860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:55.328860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:38:55.328860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:55.328860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:38:55.328860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:38:55.328860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:55.328860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:38:55.328860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:55.328860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:38:55.328860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:38:55.328860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:55.328860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:38:55.329159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:55.329159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:38:55.329601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:55.329601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:38:55.329601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:38:55.329601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:38:55.329601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:38:55.329601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:38:55.329601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:55.329601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:38:55.329601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:38:55.329601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:38:55.329601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:38:55.329601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:38:55.329601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:55.329601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:55.329601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:38:55.329601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:38:55.329933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:38:55.329933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:38:55.329933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:55.329933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:55.329933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:55.329933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:38:55.329933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:55.329933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:55.329933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:55.329933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:38:55.329933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:55.329933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:38:55.329933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:38:55.329933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:55.329933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:38:55.329933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:55.329933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:38:55.329933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:38:55.329933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:55.330306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:38:55.330306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:55.330306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:38:55.336484 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:38:55.336858 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:38:55.336971 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:38:55.337093 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:38:55.337657 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:38:55.337958 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:55.338760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:38:55.338760 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:55.338760 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:38:55.338760 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:38:55.338760 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:55.338760 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:38:55.338760 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:55.338760 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:38:55.339453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:55.339453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:38:55.339453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:38:55.339453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:38:55.339453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:55.339453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:38:55.339453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:55.339453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:38:55.339453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:55.339453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:38:55.339453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:55.339453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:38:55.339453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:55.339453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:38:55.339453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:38:55.339818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:55.339818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:38:55.339818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:55.339818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:38:55.339818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:38:55.339818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:55.339818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:38:55.339818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:55.339818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:38:55.340267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:55.340267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:38:55.340267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:55.340267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:38:55.340267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:38:55.340267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:38:55.340267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:38:55.340267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:38:55.340267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:38:55.340267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:38:55.340267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:55.340267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:55.340546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:55.340546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:38:55.340546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:55.340546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:55.340546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:55.340546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:38:55.340546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:55.340546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:38:55.340546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:38:55.340546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:55.340546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:38:55.340546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:55.340546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:38:55.340546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:38:55.340836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:55.340836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:38:55.340836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:55.340836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:38:55.341254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:55.341254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:38:55.341254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:38:55.341254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:38:55.341254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:38:55.341254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:38:55.341254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:55.341254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:38:55.341254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:38:55.341254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:38:55.341254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:38:55.341254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:38:55.341254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:55.341254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:55.341254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:38:55.341254 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:38:55.341582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:38:55.341582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:38:55.341582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:38:55.341582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:38:55.341582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:38:55.341582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:55.341582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:55.341582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:55.341582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:38:55.341582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:55.341582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:55.341582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:55.341582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:38:55.341582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:55.341582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:38:55.341582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:38:55.341582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:55.341582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:38:55.341943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:55.341943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:38:55.341943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:38:55.341943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:55.341943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:38:55.341943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:55.341943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:38:55.345057 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:38:55.345356 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:38:55.345486 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:38:55.345612 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:38:55.346039 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:38:55.346380 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:55.347059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:38:55.347059 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:55.347059 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:38:55.347059 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:38:55.347059 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:55.347059 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:38:55.347059 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:55.347059 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:38:55.347832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:55.347832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:38:55.347832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:38:55.347832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:38:55.347832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:55.347832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:38:55.347832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:55.347832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:38:55.347832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:55.347832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:38:55.347832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:55.347832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:38:55.347832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:55.347832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:38:55.347832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:38:55.348151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:55.348151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:38:55.348151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:55.348151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:38:55.348151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:38:55.348151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:55.348151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:38:55.348151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:55.348151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:38:55.348686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:55.348686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:38:55.348686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:55.348686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:38:55.348686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:38:55.348686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:38:55.348686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:38:55.348686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:38:55.348686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:38:55.348686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:55.348686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:55.348929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:55.348929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:38:55.348929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:55.348929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:55.348929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:55.348929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:38:55.348929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:55.348929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:38:55.348929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:38:55.348929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:55.348929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:38:55.348929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:55.348929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:38:55.348929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:38:55.349199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:55.349199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:38:55.349199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:55.349199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:38:55.349716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:55.349716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:38:55.349716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:38:55.349716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:38:55.349716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:38:55.349716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:38:55.349716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:55.349716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:38:55.349716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:38:55.349716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:38:55.349716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:38:55.349716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:38:55.349716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:55.349716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:55.349716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:38:55.349716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:38:55.349716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:38:55.350055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:38:55.350055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:38:55.350055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:38:55.350055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:55.350055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:55.350055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:55.350055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:38:55.350055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:55.350055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:55.350055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:55.350055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:38:55.350055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:55.350055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:38:55.350055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:38:55.350055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:55.350055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:38:55.350055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:55.350055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:38:55.350055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:38:55.350433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:55.350433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:38:55.350433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:55.350433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:38:55.352082 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:38:55.352340 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:38:55.352474 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:38:55.352620 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:38:55.353023 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:38:55.353356 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:38:55.353582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:55.354206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:38:55.354206 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:55.354206 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:38:55.354206 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:38:55.354206 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:55.354206 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:38:55.354206 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:55.354206 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:38:55.355000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:55.355000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:38:55.355000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:55.355000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:38:55.355000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:38:55.355000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:38:55.355000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:38:55.355000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:55.355000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:55.355000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:55.355000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:38:55.355000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:55.355283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:38:55.355283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:55.355283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:38:55.355283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:55.355283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:38:55.355283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:38:55.355283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:55.355283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:38:55.355283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:55.355283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:38:55.355283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:38:55.355283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:55.355283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:38:55.355536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:55.355536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:38:55.356268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:55.356268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:38:55.356268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:38:55.356268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:38:55.356268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:38:55.356268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:38:55.356268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:55.356268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:38:55.356268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:38:55.356268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:38:55.356268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:38:55.356268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:38:55.356268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:38:55.356268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:38:55.356268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:38:55.356268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:38:55.356766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:38:55.356766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:38:55.356766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:55.356766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:55.356766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:38:55.356766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:38:55.356766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:38:55.356766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:38:55.356766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:38:55.356766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:38:55.356766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:38:55.356766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:38:55.356766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:38:55.356766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:38:55.356766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:38:55.356766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:38:55.356766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:38:55.356766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:38:55.356766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:38:55.357442 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:38:55.357442 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:38:55.357442 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:39:00.091305 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:39:00.092036 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:39:00.092305 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:39:00.093617 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:39:00.124378 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:00.124769 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:00.125696 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:39:00.125696 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:00.125696 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:39:00.125696 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:39:00.125696 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:00.125696 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:39:00.125696 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:00.125696 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:39:00.126567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:00.126567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:39:00.126567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:00.126567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:39:00.126567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:39:00.126567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:39:00.126567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:39:00.126567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:00.126567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:00.126567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:00.126567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:39:00.126567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:00.127106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:00.127106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:00.127106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:39:00.127106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:00.127106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:39:00.127106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:39:00.127106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:00.127106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:39:00.127106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:00.127106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:39:00.127106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:39:00.127106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:00.127106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:39:00.127660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:00.127660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:39:00.127660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:00.127660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:39:00.127660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:39:00.127660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:39:00.127660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:39:00.127660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:39:00.127660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:00.127660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:39:00.127660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:39:00.127660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:39:00.127660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:39:00.127660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:39:00.127660 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:00.128275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:00.128275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:39:00.128275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:39:00.128275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:39:00.128275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:39:00.128275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:00.128275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:00.128275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:00.128275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:39:00.128275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:00.128275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:00.128275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:00.128275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:39:00.128275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:00.128275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:39:00.128275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:39:00.128275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:00.128275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:39:00.128275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:00.129248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:39:00.129248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:39:00.129248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:00.129248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:39:00.129248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:00.129248 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:39:00.325983 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:39:00.326398 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:39:00.326511 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:39:00.327289 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:00.327601 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:00.327835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:00.328680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:39:00.328680 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:00.328680 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:39:00.328680 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:39:00.328680 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:00.328680 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:39:00.328680 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:00.328680 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:39:00.329651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:00.329651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:39:00.329651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:00.329651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:39:00.329651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:39:00.329651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:39:00.329651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:39:00.329651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:00.329651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:00.329651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:00.329651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:39:00.329651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:00.329917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:00.329917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:00.329917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:39:00.329917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:00.329917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:39:00.329917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:39:00.329917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:00.329917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:39:00.329917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:00.329917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:39:00.329917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:39:00.329917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:00.329917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:39:00.330175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:00.330175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:39:00.330723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:00.330723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:39:00.330723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:39:00.330723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:39:00.330723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:39:00.330723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:39:00.330723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:00.330723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:39:00.330723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:39:00.330723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:39:00.330723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:39:00.330723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:39:00.330723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:00.330723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:00.330723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:39:00.330723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:39:00.331451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:39:00.331451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:39:00.331451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:00.331451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:00.331451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:00.331451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:39:00.331451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:00.331451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:00.331451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:00.331451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:39:00.331451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:00.331451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:39:00.331451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:39:00.331451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:00.331451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:39:00.331451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:00.331451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:39:00.331451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:39:00.331451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:00.332229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:39:00.332229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:00.332229 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:39:00.336998 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:39:00.337275 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:39:00.337431 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:39:00.337545 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:39:00.337923 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:00.338185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:00.338863 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:39:00.338863 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:00.338863 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:39:00.338863 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:39:00.338863 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:00.338863 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:39:00.338863 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:00.338863 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:39:00.339949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:00.339949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:39:00.339949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:39:00.339949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:39:00.339949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:00.339949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:39:00.339949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:00.339949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:39:00.339949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:00.339949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:39:00.339949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:00.339949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:39:00.339949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:00.339949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:39:00.339949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:39:00.340371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:00.340371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:39:00.340371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:00.340371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:39:00.340371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:39:00.340371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:00.340371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:39:00.340371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:00.340371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:39:00.341175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:00.341175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:39:00.341175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:00.341175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:39:00.341175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:39:00.341175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:39:00.341175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:39:00.341175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:39:00.341175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:39:00.341175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:39:00.341175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:00.341175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:00.341951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:00.341951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:39:00.341951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:00.341951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:00.341951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:00.341951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:39:00.341951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:00.341951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:39:00.341951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:39:00.341951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:00.341951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:39:00.341951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:00.341951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:39:00.341951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:39:00.342533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:00.342533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:39:00.342533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:00.342533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:39:00.342911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:00.342911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:39:00.342911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:39:00.342911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:39:00.342911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:39:00.342911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:39:00.342911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:00.342911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:39:00.342911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:39:00.342911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:39:00.342911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:39:00.342911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:39:00.342911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:00.342911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:00.342911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:39:00.342911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:39:00.343323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:39:00.343323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:39:00.343323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:39:00.343323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:39:00.343323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:39:00.343323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:00.343323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:00.343323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:00.343323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:39:00.343323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:00.343323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:00.343323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:00.343323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:39:00.343323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:00.343323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:39:00.343323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:39:00.343323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:00.343323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:39:00.343779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:00.343779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:39:00.343779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:39:00.343779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:00.343779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:39:00.343779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:00.343779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:39:00.345534 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:39:00.345848 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:39:00.346013 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:39:00.346194 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:39:00.346603 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:00.346923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:00.347600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:39:00.347600 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:00.347600 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:39:00.347600 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:39:00.347600 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:00.347600 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:39:00.347600 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:00.347600 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:39:00.348305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:00.348305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:39:00.348305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:39:00.348305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:39:00.348305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:00.348305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:39:00.348305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:00.348305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:39:00.348305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:00.348305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:39:00.348305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:00.348305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:39:00.348305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:00.348305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:39:00.348305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:39:00.348731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:00.348731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:39:00.348731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:00.348731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:39:00.348731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:39:00.348731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:00.348731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:39:00.348731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:00.348731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:39:00.349178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:00.349178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:39:00.349178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:00.349178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:39:00.349178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:39:00.349178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:39:00.349178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:39:00.349178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:39:00.349178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:39:00.349178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:00.349178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:00.349518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:00.349518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:39:00.349518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:00.349518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:00.349518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:00.349518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:39:00.349518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:00.349518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:39:00.349518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:39:00.349518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:00.349518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:39:00.349518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:00.349518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:39:00.349518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:39:00.349876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:00.349876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:39:00.349876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:00.349876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:39:00.350279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:00.350279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:39:00.350279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:39:00.350279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:39:00.350279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:39:00.350279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:39:00.350279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:00.350279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:39:00.350279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:39:00.350279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:39:00.350279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:39:00.350279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:39:00.350279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:00.350279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:00.350279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:39:00.350279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:39:00.350279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:39:00.350758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:39:00.350758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:39:00.350758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:39:00.350758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:00.350758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:00.350758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:00.350758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:39:00.350758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:00.350758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:00.350758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:00.350758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:39:00.350758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:00.350758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:39:00.350758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:39:00.350758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:00.350758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:39:00.350758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:00.350758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:39:00.350758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:39:00.351205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:00.351205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:39:00.351205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:00.351205 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:39:00.352537 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:39:00.352822 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:39:00.352936 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:39:00.353084 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:39:00.353464 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:00.353773 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:00.353972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:00.354581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:39:00.354581 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:00.354581 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:39:00.354581 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:39:00.354581 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:00.354581 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:39:00.354581 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:00.354581 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:39:00.355369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:00.355369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:39:00.355369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:00.355369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:39:00.355369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:39:00.355369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:39:00.355369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:39:00.355369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:00.355369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:00.355369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:00.355369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:39:00.355369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:00.355729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:00.355729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:00.355729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:39:00.355729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:00.355729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:39:00.355729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:39:00.355729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:00.355729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:39:00.355729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:00.355729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:39:00.355729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:39:00.355729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:00.355729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:39:00.356036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:00.356036 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:39:00.356398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:00.356398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:39:00.356398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:39:00.356398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:39:00.356398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:39:00.356398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:39:00.356398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:00.356398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:39:00.356398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:39:00.356398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:39:00.356398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:39:00.356398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:39:00.356398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:00.356398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:00.356398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:39:00.356398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:39:00.357617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:39:00.357617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:39:00.357617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:00.357617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:00.357617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:00.357617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:39:00.357617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:00.357617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:00.357617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:00.357617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:39:00.357617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:00.357617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:39:00.357617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:39:00.357617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:00.357617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:39:00.357617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:00.357617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:39:00.357617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:39:00.357617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:00.358247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:39:00.358247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:00.358247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:39:05.092036 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:39:05.092992 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:39:05.092992 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:39:05.093338 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:39:05.128753 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:05.128753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:05.129252 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:39:05.129252 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:05.129252 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:39:05.129252 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:39:05.129252 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:05.129252 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:39:05.129252 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:05.129252 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:39:05.130251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:05.130251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:39:05.130251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:05.130251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:39:05.130251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:39:05.130251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:39:05.130251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:39:05.130251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:05.130251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:05.130251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:05.130251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:39:05.130251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:05.130715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:05.130715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:05.130715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:39:05.130715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:05.130715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:39:05.130715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:39:05.130715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:05.130715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:39:05.130715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:05.130715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:39:05.130715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:39:05.130715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:05.130715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:39:05.131280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:05.131280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:39:05.131364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:05.131364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:39:05.131364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:39:05.131364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:39:05.131364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:39:05.131364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:39:05.131364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:05.131364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:39:05.131364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:39:05.131364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:39:05.131364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:39:05.131364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:39:05.131364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:05.131364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:05.131364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:39:05.131364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:39:05.131920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:39:05.131920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:39:05.131920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:05.131920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:05.131920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:05.131920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:39:05.131920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:05.131920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:05.131920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:05.131920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:39:05.131920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:05.131920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:39:05.131920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:39:05.131920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:05.131920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:39:05.131920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:05.131920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:39:05.131920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:39:05.131920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:05.132567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:39:05.132567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:05.132567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:39:05.326907 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:39:05.327768 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:39:05.328023 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:39:05.329293 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:05.329690 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:05.329932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:05.330903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:39:05.330903 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:05.330903 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:39:05.330903 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:39:05.330903 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:05.330903 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:39:05.330903 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:05.330903 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:39:05.332645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:05.332645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:39:05.332645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:05.332645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:39:05.332645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:39:05.332645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:39:05.332645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:39:05.332645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:05.332645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:05.332645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:05.332645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:39:05.332645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:05.333560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:05.333560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:05.333560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:39:05.333560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:05.333560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:39:05.333560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:39:05.333560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:05.333560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:39:05.333560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:05.333560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:39:05.333560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:39:05.333560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:05.333560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:39:05.334445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:05.334445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:39:05.334990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:05.334990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:39:05.334990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:39:05.334990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:39:05.334990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:39:05.334990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:39:05.334990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:05.334990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:39:05.334990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:39:05.334990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:39:05.334990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:39:05.334990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:39:05.334990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:05.334990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:05.334990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:39:05.334990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:39:05.335826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:39:05.335826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:39:05.335826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:05.335826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:05.335826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:05.335826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:39:05.335826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:05.335826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:05.335826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:05.335826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:39:05.335826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:05.335826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:39:05.335826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:39:05.335826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:05.335826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:39:05.335826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:05.335826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:39:05.335826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:39:05.335826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:05.336341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:39:05.336341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:05.336341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:39:05.337433 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:39:05.337804 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:39:05.337949 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:39:05.338133 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:39:05.338509 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:05.338690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:05.339218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:39:05.339218 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:05.339218 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:39:05.339218 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:39:05.339218 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:05.339218 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:39:05.339218 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:05.339218 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:39:05.340219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:05.340219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:39:05.340219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:39:05.340219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:39:05.340219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:05.340219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:39:05.340219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:05.340219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:39:05.340219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:05.340219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:39:05.340219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:05.340219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:39:05.340219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:05.340219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:39:05.340219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:39:05.340990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:05.340990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:39:05.340990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:05.340990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:39:05.340990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:39:05.340990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:05.340990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:39:05.340990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:05.340990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:39:05.341807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:05.341807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:39:05.341807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:05.341807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:39:05.341807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:39:05.341807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:39:05.341807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:39:05.341807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:39:05.341807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:39:05.341807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:39:05.341807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:05.341807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:05.342500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:05.342500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:39:05.342500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:05.342500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:05.342500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:05.342500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:39:05.342500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:05.342500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:39:05.342500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:39:05.342500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:05.342500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:39:05.342500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:05.342500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:39:05.342500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:39:05.343055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:05.343055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:39:05.343055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:05.343055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:39:05.343055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:05.343055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:39:05.343055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:39:05.343055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:39:05.343055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:39:05.343055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:39:05.343055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:05.343055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:39:05.343055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:39:05.343055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:39:05.343539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:39:05.343539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:39:05.343539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:05.343539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:05.343539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:39:05.343539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:39:05.343539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:39:05.343539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:39:05.343539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:39:05.343539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:39:05.343539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:39:05.343539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:05.343539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:05.343539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:05.343539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:39:05.343539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:05.343539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:05.343539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:05.343539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:39:05.344119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:05.344119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:39:05.344119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:39:05.344119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:05.344119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:39:05.344119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:05.344119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:39:05.344119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:39:05.344119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:05.344119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:39:05.344119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:05.344119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:39:05.346128 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:39:05.346553 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:39:05.346704 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:39:05.346918 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:39:05.347399 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:05.347811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:05.348507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:39:05.348507 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:05.348507 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:39:05.348507 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:39:05.348507 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:05.348507 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:39:05.348507 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:05.348507 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:39:05.349349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:05.349349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:39:05.349349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:39:05.349349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:39:05.349349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:05.349349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:39:05.349349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:05.349349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:39:05.349349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:05.349349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:39:05.349349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:05.349349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:39:05.349349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:05.349349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:39:05.349349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:39:05.349998 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:05.349998 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:39:05.349998 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:05.349998 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:39:05.349998 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:39:05.349998 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:05.349998 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:39:05.349998 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:05.349998 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:39:05.350571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:05.350571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:39:05.350571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:05.350571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:39:05.350571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:39:05.350571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:39:05.350571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:39:05.350571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:39:05.350571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:39:05.350571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:05.350571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:05.351099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:05.351099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:39:05.351099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:05.351099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:05.351099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:05.351099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:39:05.351099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:05.351099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:39:05.351099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:39:05.351099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:05.351099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:39:05.351099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:05.351099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:39:05.351099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:39:05.351711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:05.351711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:39:05.351711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:05.351711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:39:05.352092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:05.352092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:39:05.352092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:39:05.352092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:39:05.352092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:39:05.352092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:39:05.352092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:05.352092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:39:05.352092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:39:05.352092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:39:05.352092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:39:05.352092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:39:05.352092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:05.352092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:05.352092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:39:05.352092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:39:05.352092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:39:05.352678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:39:05.352678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:39:05.352678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:39:05.352678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:05.352678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:05.352678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:05.352678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:39:05.352678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:05.352678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:05.352678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:05.352678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:39:05.352678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:05.352678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:39:05.352678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:39:05.352678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:05.352678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:39:05.352678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:05.352678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:39:05.352678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:39:05.353270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:05.353270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:39:05.353270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:05.353270 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:39:05.353270 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:39:05.353567 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:39:05.353567 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:39:05.353771 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:39:05.354210 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:05.354578 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:05.355054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:05.355779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:39:05.355779 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:05.355779 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:39:05.355779 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:39:05.355779 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:05.355779 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:39:05.355779 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:05.355779 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:39:05.356714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:05.356714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:39:05.356714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:05.356714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:39:05.356714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:39:05.356714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:39:05.356714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:39:05.356714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:05.356714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:05.356714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:05.356714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:39:05.356714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:05.357258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:05.357258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:05.357258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:39:05.357258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:05.357258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:39:05.357258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:39:05.357258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:05.357258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:39:05.357258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:05.357258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:39:05.357258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:39:05.357258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:05.357258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:39:05.357844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:05.357844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:39:05.358063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:05.358063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:39:05.358063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:39:05.358063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:39:05.358063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:39:05.358063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:39:05.358063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:05.358063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:39:05.358063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:39:05.358063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:39:05.358063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:39:05.358063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:39:05.358063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:05.358063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:05.358063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:39:05.358063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:39:05.358602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:39:05.358602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:39:05.358602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:05.358602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:05.358602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:05.358602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:39:05.358602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:05.358602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:05.358602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:05.358602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:39:05.358602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:05.358602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:39:05.358602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:39:05.358602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:05.358602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:39:05.358602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:05.358602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:39:05.358602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:39:05.358602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:05.359159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:39:05.359159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:05.359159 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:39:10.092950 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:39:10.094005 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:39:10.094005 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:39:10.094523 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:39:10.125508 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:10.125644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:10.126206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:39:10.126206 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:10.126206 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:39:10.126206 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:39:10.126206 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:10.126206 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:39:10.126206 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:10.126206 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:39:10.127362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:10.127362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:39:10.127362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:10.127362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:39:10.127362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:39:10.127362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:39:10.127362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:39:10.127362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:10.127362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:10.127362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:10.127362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:39:10.127362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:10.128305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:10.128305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:10.128305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:39:10.128305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:10.128305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:39:10.128305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:39:10.128305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:10.128305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:39:10.128305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:10.128305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:39:10.128305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:39:10.128305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:10.128305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:39:10.128974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:10.128974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:39:10.128974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:10.128974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:39:10.128974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:39:10.128974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:39:10.128974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:39:10.128974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:39:10.128974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:10.128974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:39:10.128974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:39:10.128974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:39:10.128974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:39:10.128974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:39:10.128974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:10.130513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:10.130513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:39:10.130513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:39:10.130513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:39:10.130513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:39:10.130513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:10.130513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:10.130513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:10.130513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:39:10.130513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:10.130513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:10.130513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:10.130513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:39:10.130513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:10.130513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:39:10.130513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:39:10.130513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:10.130513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:39:10.130513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:10.138288 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:39:10.138288 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:39:10.138288 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:10.138288 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:39:10.138288 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:10.138288 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:39:10.327700 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:39:10.328891 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:39:10.328891 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:39:10.330149 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:10.330498 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:10.330806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:10.331735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:39:10.331735 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:10.331735 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:39:10.331735 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:39:10.331735 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:10.331735 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:39:10.331735 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:10.331735 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:39:10.332794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:10.332794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:39:10.332794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:10.332794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:39:10.332794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:39:10.332794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:39:10.332794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:39:10.332794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:10.332794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:10.332794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:10.332794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:39:10.332794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:10.333327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:10.333327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:10.333327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:39:10.333327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:10.333327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:39:10.333327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:39:10.333327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:10.333327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:39:10.333327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:10.333327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:39:10.333327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:39:10.333327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:10.333327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:39:10.333960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:10.333960 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:39:10.334283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:10.334283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:39:10.334283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:39:10.334283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:39:10.334283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:39:10.334283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:39:10.334283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:10.334283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:39:10.334283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:39:10.334283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:39:10.334283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:39:10.334283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:39:10.334283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:10.334283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:10.334283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:39:10.334283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:39:10.336334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:39:10.336334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:39:10.336334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:10.336334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:10.336334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:10.336334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:39:10.336334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:10.336334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:10.336334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:10.336334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:39:10.336334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:10.336334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:39:10.336334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:39:10.336334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:10.336334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:39:10.336334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:10.336334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:39:10.336334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:39:10.336334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:10.337959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:39:10.337959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:10.337959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:39:10.337959 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:39:10.338552 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:39:10.338552 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:39:10.338552 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:39:10.338891 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:10.339025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:10.339550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:39:10.339550 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:10.339550 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:39:10.339550 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:39:10.339550 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:10.339550 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:39:10.339550 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:10.339550 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:39:10.340564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:10.340564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:39:10.340564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:39:10.340564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:39:10.340564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:10.340564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:39:10.340564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:10.340564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:39:10.340564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:10.340564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:39:10.340564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:10.340564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:39:10.340564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:10.340564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:39:10.340564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:39:10.341311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:10.341311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:39:10.341311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:10.341311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:39:10.341311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:39:10.341311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:10.341311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:39:10.341311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:10.341311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:39:10.341759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:10.341759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:39:10.341759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:10.341759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:39:10.341759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:39:10.341759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:39:10.341759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:39:10.341759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:39:10.341759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:39:10.341759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:39:10.341759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:10.341759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:10.342258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:10.342258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:39:10.342258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:10.342258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:10.342258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:10.342258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:39:10.342258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:10.342258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:39:10.342258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:39:10.342258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:10.342258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:39:10.342258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:10.342258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:39:10.342258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:39:10.342841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:10.342841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:39:10.342841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:10.342841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:39:10.342841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:10.342841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:39:10.342841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:39:10.342841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:39:10.342841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:39:10.342841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:39:10.342841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:10.342841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:39:10.342841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:39:10.342841 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:39:10.343662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:39:10.343662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:39:10.343662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:10.343662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:10.343662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:39:10.343662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:39:10.343662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:39:10.343662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:39:10.343662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:39:10.343662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:39:10.343662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:39:10.343662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:10.343662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:10.343662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:10.343662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:39:10.343662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:10.343662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:10.343662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:10.343662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:39:10.344566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:10.344566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:39:10.344566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:39:10.344566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:10.344566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:39:10.344566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:10.344566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:39:10.344566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:39:10.344566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:10.344566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:39:10.344566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:10.344566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:39:10.346702 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:39:10.347013 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:39:10.347147 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:39:10.347279 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:39:10.347590 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:10.347808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:10.348260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:39:10.348260 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:10.348260 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:39:10.348260 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:39:10.348260 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:10.348260 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:39:10.348260 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:10.348260 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:39:10.349004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:10.349004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:39:10.349004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:39:10.349004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:39:10.349004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:10.349004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:39:10.349004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:10.349004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:39:10.349004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:10.349004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:39:10.349004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:10.349004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:39:10.349004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:10.349004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:39:10.349004 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:39:10.349514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:10.349514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:39:10.349514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:10.349514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:39:10.349514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:39:10.349514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:10.349514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:39:10.349514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:10.349514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:39:10.349877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:10.349877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:39:10.349877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:10.349877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:39:10.349877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:39:10.349877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:39:10.349877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:39:10.349877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:39:10.349877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:39:10.349877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:10.349877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:10.350217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:10.350217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:39:10.350217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:10.350217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:10.350217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:10.350217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:39:10.350217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:10.350217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:39:10.350217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:39:10.350217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:10.350217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:39:10.350217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:10.350217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:39:10.350217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:39:10.350641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:10.350641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:39:10.350641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:10.350641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:39:10.350948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:10.350948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:39:10.350948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:39:10.350948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:39:10.350948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:39:10.350948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:39:10.350948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:10.350948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:39:10.350948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:39:10.350948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:39:10.350948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:39:10.350948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:39:10.350948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:10.350948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:10.350948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:39:10.350948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:39:10.350948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:39:10.351441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:39:10.351441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:39:10.351441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:39:10.351441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:10.351441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:10.351441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:10.351441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:39:10.351441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:10.351441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:10.351441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:10.351441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:39:10.351441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:10.351441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:39:10.351441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:39:10.351441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:10.351441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:39:10.351441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:10.351441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:39:10.351441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:39:10.352011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:10.352011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:39:10.352011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:10.352011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:39:10.353682 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:39:10.354043 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:39:10.354158 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:39:10.354305 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:39:10.354595 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:10.354805 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:10.354996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:10.355735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:39:10.355735 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:10.355735 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:39:10.355735 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:39:10.355735 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:10.355735 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:39:10.355735 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:10.355735 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:39:10.356283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:10.356283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:39:10.356283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:10.356283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:39:10.356283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:39:10.356283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:39:10.356283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:39:10.356283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:10.356283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:10.356283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:10.356283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:39:10.356283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:10.356733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:10.356733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:10.356733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:39:10.356733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:10.356733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:39:10.356733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:39:10.356733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:10.356733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:39:10.356733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:10.356733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:39:10.356733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:39:10.356733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:10.356733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:39:10.357064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:10.357064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:39:10.357494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:10.357494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:39:10.357494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:39:10.357494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:39:10.357494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:39:10.357494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:39:10.357494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:10.357494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:39:10.357494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:39:10.357494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:39:10.357494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:39:10.357494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:39:10.357494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:10.357494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:10.357494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:39:10.357494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:39:10.358052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:39:10.358052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:39:10.358052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:10.358052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:10.358052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:10.358052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:39:10.358052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:10.358052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:10.358052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:10.358052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:39:10.358052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:10.358052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:39:10.358052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:39:10.358052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:10.358052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:39:10.358052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:10.358052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:39:10.358052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:39:10.358052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:10.358590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:39:10.358590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:10.358590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:39:15.093675 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:39:15.095158 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:39:15.095350 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:39:15.096625 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:39:15.128355 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:15.128623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:15.129245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:39:15.129245 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:15.129245 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:39:15.129245 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:39:15.129245 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:15.129245 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:39:15.129245 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:15.129245 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:39:15.129965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:15.129965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:39:15.129965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:15.129965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:39:15.129965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:39:15.129965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:39:15.129965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:39:15.129965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:15.129965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:15.129965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:15.129965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:39:15.129965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:15.130785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:15.130785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:15.130785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:39:15.130785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:15.130785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:39:15.130785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:39:15.130785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:15.130785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:39:15.130785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:15.130785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:39:15.130785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:39:15.130785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:15.130785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:39:15.131525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:15.131525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:39:15.131525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:15.131525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:39:15.131525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:39:15.131525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:39:15.131525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:39:15.131525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:39:15.131525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:15.131525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:39:15.131525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:39:15.131525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:39:15.131525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:39:15.131525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:39:15.131525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:15.132234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:15.132234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:39:15.132234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:39:15.132234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:39:15.132234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:39:15.132234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:15.132234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:15.132234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:15.132234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:39:15.132234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:15.132234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:15.132234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:15.132234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:39:15.132234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:15.132234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:39:15.132234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:39:15.132234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:15.132234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:39:15.132234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:15.133200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:39:15.133200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:39:15.133200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:15.133200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:39:15.133200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:15.133200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:39:15.328446 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:39:15.329064 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:39:15.329272 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:39:15.330424 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:15.330821 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:15.331066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:15.331932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:39:15.331932 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:15.331932 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:39:15.331932 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:39:15.331932 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:15.331932 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:39:15.331932 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:15.331932 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:39:15.333451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:15.333451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:39:15.333451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:15.333451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:39:15.333451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:39:15.333451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:39:15.333451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:39:15.333451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:15.333451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:15.333451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:15.333451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:39:15.333451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:15.334132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:15.334132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:15.334132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:39:15.334132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:15.334132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:39:15.334132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:39:15.334132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:15.334132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:39:15.334132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:15.334132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:39:15.334132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:39:15.334132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:15.334132 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:39:15.334785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:15.334785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:39:15.335250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:15.335250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:39:15.335250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:39:15.335250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:39:15.335250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:39:15.335250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:39:15.335250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:15.335250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:39:15.335250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:39:15.335250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:39:15.335250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:39:15.335250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:39:15.335250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:15.335250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:15.335250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:39:15.335250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:39:15.336072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:39:15.336072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:39:15.336072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:15.336072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:15.336072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:15.336072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:39:15.336072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:15.336072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:15.336072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:15.336072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:39:15.336072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:15.336072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:39:15.336072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:39:15.336072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:15.336072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:39:15.336072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:15.336072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:39:15.336072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:39:15.336072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:15.337243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:39:15.337243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:15.337243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:39:15.338392 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:39:15.338770 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:39:15.338935 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:39:15.339110 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:39:15.339629 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:15.339949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:15.340675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:39:15.340675 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:15.340675 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:39:15.340675 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:39:15.340675 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:15.340675 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:39:15.340675 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:15.340675 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:39:15.341446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:15.341446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:39:15.341446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:39:15.341446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:39:15.341446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:15.341446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:39:15.341446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:15.341446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:39:15.341446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:15.341446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:39:15.341446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:15.341446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:39:15.341446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:15.341446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:39:15.341446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:39:15.342210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:15.342210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:39:15.342210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:15.342210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:39:15.342210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:39:15.342210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:15.342210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:39:15.342210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:15.342210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:39:15.342521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:15.342521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:39:15.342521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:15.342521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:39:15.342521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:39:15.342521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:39:15.342521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:39:15.342521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:39:15.342521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:39:15.342521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:39:15.342521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:15.342521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:15.342942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:15.342942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:39:15.342942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:15.342942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:15.342942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:15.342942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:39:15.342942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:15.342942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:39:15.342942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:39:15.342942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:15.342942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:39:15.342942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:15.342942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:39:15.342942 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:39:15.343313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:15.343313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:39:15.343313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:15.343313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:39:15.343313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:15.343313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:39:15.343313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:39:15.343313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:39:15.343313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:39:15.343313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:39:15.343313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:15.343313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:39:15.343313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:39:15.343313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:39:15.343809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:39:15.343809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:39:15.343809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:15.343809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:15.343809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:39:15.343809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:39:15.343809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:39:15.343809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:39:15.343809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:39:15.343809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:39:15.343809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:39:15.343809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:15.343809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:15.343809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:15.343809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:39:15.343809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:15.343809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:15.343809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:15.343809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:39:15.344363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:15.344363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:39:15.344363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:39:15.344363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:15.344363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:39:15.344363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:15.344363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:39:15.344363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:39:15.344363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:15.344363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:39:15.344363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:15.344363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:39:15.347071 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:39:15.347290 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:39:15.347394 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:39:15.347535 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:39:15.347958 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:15.348301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:15.348965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:39:15.348965 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:15.348965 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:39:15.348965 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:39:15.348965 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:15.348965 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:39:15.348965 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:15.348965 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:39:15.349718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:15.349718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:39:15.349718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:39:15.349718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:39:15.349718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:15.349718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:39:15.349718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:15.349718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:39:15.349718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:15.349718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:39:15.349718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:15.349718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:39:15.349718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:15.349718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:39:15.349718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:39:15.350249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:15.350249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:39:15.350249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:15.350249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:39:15.350249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:39:15.350249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:15.350249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:39:15.350249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:15.350249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:39:15.350611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:15.350611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:39:15.350611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:15.350611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:39:15.350611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:39:15.350611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:39:15.350611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:39:15.350611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:39:15.350611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:39:15.350611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:15.350611 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:15.350965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:15.350965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:39:15.350965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:15.350965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:15.350965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:15.350965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:39:15.350965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:15.350965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:39:15.350965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:39:15.350965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:15.350965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:39:15.350965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:15.350965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:39:15.350965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:39:15.351341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:15.351341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:39:15.351341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:15.351341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:39:15.351612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:15.351612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:39:15.351612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:39:15.351612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:39:15.351612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:39:15.351612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:39:15.351612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:15.351612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:39:15.351612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:39:15.351612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:39:15.351612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:39:15.351612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:39:15.351612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:15.351612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:15.351612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:39:15.351612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:39:15.351612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:39:15.352112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:39:15.352112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:39:15.352112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:39:15.352112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:15.352112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:15.352112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:15.352112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:39:15.352112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:15.352112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:15.352112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:15.352112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:39:15.352112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:15.352112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:39:15.352112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:39:15.352112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:15.352112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:39:15.352112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:15.352112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:39:15.352112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:39:15.352657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:15.352657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:39:15.352657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:15.352657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:39:15.354106 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:39:15.354329 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:39:15.354452 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:39:15.354598 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:39:15.355029 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:15.355356 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:15.355596 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:15.356216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:39:15.356216 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:15.356216 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:39:15.356216 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:39:15.356216 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:15.356216 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:39:15.356216 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:15.356216 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:39:15.357022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:15.357022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:39:15.357022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:15.357022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:39:15.357022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:39:15.357022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:39:15.357022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:39:15.357022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:15.357022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:15.357022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:15.357022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:39:15.357022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:15.357462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:15.357462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:15.357462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:39:15.357462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:15.357462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:39:15.357462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:39:15.357462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:15.357462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:39:15.357462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:15.357462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:39:15.357462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:39:15.357462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:15.357462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:39:15.357816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:15.357816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:39:15.358092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:15.358092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:39:15.358092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:39:15.358092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:39:15.358092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:39:15.358092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:39:15.358092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:15.358092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:39:15.358092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:39:15.358092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:39:15.358092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:39:15.358092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:39:15.358092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:15.358092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:15.358092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:39:15.358092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:39:15.358804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:39:15.358804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:39:15.358804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:15.358804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:15.358804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:15.358804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:39:15.358804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:15.358804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:15.358804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:15.358804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:39:15.358804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:15.358804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:39:15.358804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:39:15.358804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:15.358804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:39:15.358804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:15.358804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:39:15.358804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:39:15.358804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:15.359334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:39:15.359334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:15.359334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:39:20.095199 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:39:20.095987 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:39:20.096217 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:39:20.097492 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:39:20.132666 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:20.133539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:20.133539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:39:20.133539 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:20.133539 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:39:20.133539 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:39:20.133539 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:20.133539 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:39:20.133539 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:20.133539 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:39:20.134140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:20.134140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:39:20.134140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:20.134140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:39:20.134140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:39:20.134140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:39:20.134140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:39:20.134140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:20.134140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:20.134140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:20.134140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:39:20.134140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:20.134798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:20.134798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:20.134798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:39:20.134798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:20.134798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:39:20.134798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:39:20.134798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:20.134798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:39:20.134798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:20.134798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:39:20.134798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:39:20.134798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:20.134798 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:39:20.135279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:20.135279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:39:20.135279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:20.135279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:39:20.135279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:39:20.135279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:39:20.135279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:39:20.135279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:39:20.135279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:20.135279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:39:20.135279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:39:20.135279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:39:20.135279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:39:20.135279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:39:20.135279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:20.135673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:20.135673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:39:20.135673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:39:20.135673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:39:20.135673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:39:20.135673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:20.135673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:20.135673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:20.135673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:39:20.135673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:20.135673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:20.135673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:20.135673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:39:20.135673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:20.135673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:39:20.135673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:39:20.135673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:20.135673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:39:20.135673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:20.136117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:39:20.136117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:39:20.136117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:20.136117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:39:20.136117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:20.136117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:39:20.329004 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:39:20.329691 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:39:20.329984 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:39:20.331331 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:20.331703 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:20.331897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:20.332813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:39:20.332813 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:20.332813 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:39:20.332813 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:39:20.332813 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:20.332813 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:39:20.332813 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:20.332813 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:39:20.334485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:20.334485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:39:20.334485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:20.334485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:39:20.334485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:39:20.334485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:39:20.334485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:39:20.334485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:20.334485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:20.334485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:20.334485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:39:20.334485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:20.334988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:20.334988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:20.334988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:39:20.334988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:20.334988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:39:20.334988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:39:20.334988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:20.334988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:39:20.334988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:20.334988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:39:20.334988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:39:20.334988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:20.334988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:39:20.335579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:20.335579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:39:20.335671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:20.335671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:39:20.335671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:39:20.335671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:39:20.335671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:39:20.335671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:39:20.335671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:20.335671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:39:20.335671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:39:20.335671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:39:20.335671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:39:20.335671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:39:20.335671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:20.335671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:20.335671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:39:20.335671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:39:20.336280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:39:20.336280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:39:20.336280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:20.336280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:20.336280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:20.336280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:39:20.336280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:20.336280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:20.336280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:20.336280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:39:20.336280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:20.336280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:39:20.336280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:39:20.336280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:20.336280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:39:20.336280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:20.336280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:39:20.336280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:39:20.336280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:20.337052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:39:20.337052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:20.337052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:39:20.338783 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:39:20.339039 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:39:20.339164 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:39:20.339292 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:39:20.339782 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:20.339979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:20.340566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:39:20.340566 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:20.340566 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:39:20.340566 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:39:20.340566 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:20.340566 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:39:20.340566 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:20.340566 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:39:20.341499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:20.341499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:39:20.341499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:39:20.341499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:39:20.341499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:20.341499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:39:20.341499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:20.341499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:39:20.341499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:20.341499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:39:20.341499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:20.341499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:39:20.341499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:20.341499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:39:20.341499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:39:20.342978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:20.342978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:39:20.342978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:20.342978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:39:20.342978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:39:20.342978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:20.342978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:39:20.342978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:20.342978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:39:20.342978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:20.342978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:39:20.342978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:20.342978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:39:20.342978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:39:20.344496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:39:20.344496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:39:20.344496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:39:20.344496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:39:20.344496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:39:20.344496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:20.344496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:20.344496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:20.344496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:39:20.344496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:20.344496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:20.344496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:20.344496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:39:20.346459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:20.346459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:39:20.346459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:39:20.346459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:20.346459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:39:20.346459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:20.346459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:39:20.346459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:39:20.346459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:20.346459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:39:20.346459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:20.346459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:39:20.347716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:20.347716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:39:20.347716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:39:20.347716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:39:20.347716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:39:20.347716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:39:20.347716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:20.347716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:39:20.347716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:39:20.347716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:39:20.347716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:39:20.347716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:39:20.347716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:20.347716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:20.347716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:39:20.347716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:39:20.348317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:39:20.348317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:39:20.348317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:39:20.348317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:39:20.348317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:39:20.348317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:20.348317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:20.348317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:20.348317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:39:20.348317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:20.348317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:20.348317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:20.348317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:39:20.348317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:20.348317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:39:20.348317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:39:20.348317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:20.348317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:39:20.349078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:20.349078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:39:20.349078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:39:20.349078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:20.349078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:39:20.349078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:20.349078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:39:20.349078 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:39:20.349078 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:39:20.349078 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:39:20.349324 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:39:20.349324 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:20.349324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:20.349324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:39:20.349324 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:20.349324 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:39:20.349324 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:39:20.349324 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:20.349324 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:39:20.349324 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:20.349324 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:39:20.350169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:20.350169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:39:20.350169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:39:20.350169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:39:20.350169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:20.350169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:39:20.350169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:20.350169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:39:20.350169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:20.350169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:39:20.350169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:20.350169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:39:20.350169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:20.350169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:39:20.350169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:39:20.350741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:20.350741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:39:20.350741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:20.350741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:39:20.350741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:39:20.350741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:20.350741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:39:20.350741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:20.350741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:39:20.351527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:20.351527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:39:20.351527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:20.351527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:39:20.351527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:39:20.351527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:39:20.351527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:39:20.351527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:39:20.351527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:39:20.351527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:20.351527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:20.351756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:20.351756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:39:20.351756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:20.351756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:20.351756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:20.351756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:39:20.351756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:20.351756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:39:20.351756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:39:20.351756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:20.351756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:39:20.351756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:20.351756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:39:20.351756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:39:20.351996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:20.351996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:39:20.351996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:20.351996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:39:20.353353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:20.353353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:39:20.353353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:39:20.353353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:39:20.353353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:39:20.353353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:39:20.353353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:20.353353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:39:20.353353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:39:20.353353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:39:20.353353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:39:20.353353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:39:20.353353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:20.353353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:20.353353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:39:20.353353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:39:20.353353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:39:20.353691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:39:20.353691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:39:20.353691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:39:20.353691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:20.353691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:20.353691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:20.353691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:39:20.353691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:20.353691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:20.353691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:20.353691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:39:20.353691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:20.353691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:39:20.353691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:39:20.353691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:20.353691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:39:20.353691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:20.353691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:39:20.353691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:39:20.354018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:20.354018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:39:20.354018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:20.354018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:39:20.354528 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:39:20.354844 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:39:20.354844 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:39:20.355037 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:39:20.355451 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:20.355980 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:20.355980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:20.356523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:39:20.356523 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:20.356523 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:39:20.356523 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:39:20.356523 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:20.356523 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:39:20.356523 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:20.356523 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:39:20.357520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:20.357520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:39:20.357520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:20.357520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:39:20.357520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:39:20.357520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:39:20.357520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:39:20.357520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:20.357520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:20.357520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:20.357520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:39:20.357520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:20.358095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:20.358095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:20.358095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:39:20.358095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:20.358095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:39:20.358095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:39:20.358095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:20.358095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:39:20.358095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:20.358095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:39:20.358095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:39:20.358095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:20.358095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:39:20.358343 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:20.358343 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:39:20.358657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:20.358657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:39:20.358657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:39:20.358657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:39:20.358657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:39:20.358657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:39:20.358657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:20.358657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:39:20.358657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:39:20.358657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:39:20.358657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:39:20.358657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:39:20.358657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:20.358657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:20.358657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:39:20.358657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:39:20.359268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:39:20.359268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:39:20.359268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:20.359268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:20.359268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:20.359268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:39:20.359268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:20.359268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:20.359268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:20.359268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:39:20.359268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:20.359268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:39:20.359268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:39:20.359268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:20.359268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:39:20.359268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:20.359268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:39:20.359268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:39:20.359268 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:20.359612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:39:20.359612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:20.359612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:39:25.095876 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:39:25.096882 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:39:25.096882 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:39:25.097136 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:39:25.101099 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG dbcounter [-] [105581] Writing DB stats neutron:SELECT=241 {{(pid=105581) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 14:39:25.127233 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:25.127554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:25.128038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:39:25.128038 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:25.128038 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:39:25.128038 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:39:25.128038 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:25.128038 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:39:25.128038 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:25.128038 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:39:25.128842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:25.128842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:39:25.128842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:25.128842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:39:25.128842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:39:25.128842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:39:25.128842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:39:25.128842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:25.128842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:25.128842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:25.128842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:39:25.128842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:25.129494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:25.129494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:25.129494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:39:25.129494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:25.129494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:39:25.129494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:39:25.129494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:25.129494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:39:25.129494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:25.129494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:39:25.129494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:39:25.129494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:25.129494 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:39:25.130078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:25.130078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:39:25.130078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:25.130078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:39:25.130078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:39:25.130078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:39:25.130078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:39:25.130078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:39:25.130078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:25.130078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:39:25.130078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:39:25.130078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:39:25.130078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:39:25.130078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:39:25.130078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:25.130981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:25.130981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:39:25.130981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:39:25.130981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:39:25.130981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:39:25.130981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:25.130981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:25.130981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:25.130981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:39:25.130981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:25.130981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:25.130981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:25.130981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:39:25.130981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:25.130981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:39:25.130981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:39:25.130981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:25.130981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:39:25.130981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:25.131888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:39:25.131888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:39:25.131888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:25.131888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:39:25.131888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:25.131888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:39:25.329535 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:39:25.329961 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:39:25.330044 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:39:25.330824 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:25.331025 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:25.331142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:25.331639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:39:25.331639 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:25.331639 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:39:25.331639 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:39:25.331639 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:25.331639 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:39:25.331639 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:25.331639 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:39:25.332389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:25.332389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:39:25.332389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:25.332389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:39:25.332389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:39:25.332389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:39:25.332389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:39:25.332389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:25.332389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:25.332389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:25.332389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:39:25.332389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:25.332658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:25.332658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:25.332658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:39:25.332658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:25.332658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:39:25.332658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:39:25.332658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:25.332658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:39:25.332658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:25.332658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:39:25.332658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:39:25.332658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:25.332658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:39:25.332887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:25.332887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:39:25.333131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:25.333131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:39:25.333131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:39:25.333131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:39:25.333131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:39:25.333131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:39:25.333131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:25.333131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:39:25.333131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:39:25.333131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:39:25.333131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:39:25.333131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:39:25.333131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:25.333131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:25.333131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:39:25.333131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:39:25.333467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:39:25.333467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:39:25.333467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:25.333467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:25.333467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:25.333467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:39:25.333467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:25.333467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:25.333467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:25.333467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:39:25.333467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:25.333467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:39:25.333467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:39:25.333467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:25.333467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:39:25.333467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:25.333467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:39:25.333467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:39:25.333467 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:25.333809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:39:25.333809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:25.333809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:39:25.339125 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:39:25.339413 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:39:25.339482 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:39:25.339582 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:39:25.340016 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:25.340169 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:25.343648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:39:25.343648 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:25.343648 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:39:25.343648 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:39:25.343648 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:25.343648 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:39:25.343648 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:25.343648 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:39:25.344295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:25.344295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:39:25.344295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:39:25.344295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:39:25.344295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:25.344295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:39:25.344295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:25.344295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:39:25.344295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:25.344295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:39:25.344295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:25.344295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:39:25.344295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:25.344295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:39:25.344295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:39:25.344669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:25.344669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:39:25.344669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:25.344669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:39:25.344669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:39:25.344669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:25.344669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:39:25.344669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:25.344669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:39:25.345041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:25.345041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:39:25.345041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:25.345041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:39:25.345041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:39:25.345041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:39:25.345041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:39:25.345041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:39:25.345041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:39:25.345041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:39:25.345041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:25.345041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:25.345662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:25.345662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:39:25.345662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:25.345662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:25.345662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:25.345662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:39:25.345662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:25.345662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:39:25.345662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:39:25.345662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:25.345662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:39:25.345662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:25.345662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:39:25.345662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:39:25.345965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:25.345965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:39:25.345965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:25.345965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:39:25.345965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:25.345965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:39:25.345965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:39:25.345965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:39:25.345965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:39:25.345965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:39:25.345965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:25.345965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:39:25.345965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:39:25.345965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:39:25.346264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:39:25.346264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:39:25.346264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:25.346264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:25.346264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:39:25.346264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:39:25.346264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:39:25.346264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:39:25.346264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:39:25.346264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:39:25.346264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:39:25.346264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:25.346264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:25.346264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:25.346264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:39:25.346264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:25.346264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:25.346264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:25.346264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:39:25.346631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:25.346631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:39:25.346631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:39:25.346631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:25.346631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:39:25.346631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:25.346631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:39:25.346631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:39:25.346631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:25.346631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:39:25.346631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:25.346631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:39:25.347812 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:39:25.349105 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:39:25.349105 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:39:25.349105 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:39:25.349105 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:25.349105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:25.349261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:39:25.349261 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:25.349261 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:39:25.349261 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:39:25.349261 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:25.349261 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:39:25.349261 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:25.349261 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:39:25.349763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:25.349763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:39:25.349763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:39:25.349763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:39:25.349763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:25.349763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:39:25.349763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:25.349763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:39:25.349763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:25.349763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:39:25.349763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:25.349763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:39:25.349763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:25.349763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:39:25.349763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:39:25.350069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:25.350069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:39:25.350069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:25.350069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:39:25.350069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:39:25.350069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:25.350069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:39:25.350069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:25.350069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:39:25.350481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:25.350481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:39:25.350481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:25.350481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:39:25.350481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:39:25.350481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:39:25.350481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:39:25.350481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:39:25.350481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:39:25.350481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:25.350481 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:25.350889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:25.350889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:39:25.350889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:25.350889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:25.350889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:25.350889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:39:25.350889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:25.350889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:39:25.350889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:39:25.350889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:25.350889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:39:25.350889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:25.350889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:39:25.350889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:39:25.351382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:25.351382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:39:25.351382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:25.351382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:39:25.351382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:25.351382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:39:25.351382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:39:25.351382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:39:25.351382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:39:25.351382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:39:25.351382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:25.351382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:39:25.351382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:39:25.351382 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:39:25.351709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:39:25.351709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:39:25.351709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:25.351709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:25.351709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:39:25.351709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:39:25.351709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:39:25.351709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:39:25.351709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:39:25.351709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:39:25.351709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:25.351709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:25.351709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:25.351709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:39:25.351709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:25.351709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:25.351709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:25.351709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:39:25.351709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:25.352035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:39:25.352035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:39:25.352035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:25.352035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:39:25.352035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:25.352035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:39:25.352035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:39:25.352035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:25.352035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:39:25.352035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:25.352035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:39:25.354714 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:39:25.354965 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:39:25.355051 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:39:25.355150 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:39:25.355438 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:25.355603 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:25.355720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:25.356117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:39:25.356117 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:25.356117 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:39:25.356117 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:39:25.356117 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:25.356117 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:39:25.356117 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:25.356117 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:39:25.356729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:25.356729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:39:25.356729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:25.356729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:39:25.356729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:39:25.356729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:39:25.356729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:39:25.356729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:25.356729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:25.356729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:25.356729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:39:25.356729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:25.356980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:25.356980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:25.356980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:39:25.356980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:25.356980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:39:25.356980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:39:25.356980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:25.356980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:39:25.356980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:25.356980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:39:25.356980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:39:25.356980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:25.356980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:39:25.357206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:25.357206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:39:25.357378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:25.357378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:39:25.357378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:39:25.357378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:39:25.357378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:39:25.357378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:39:25.357378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:25.357378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:39:25.357378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:39:25.357378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:39:25.357378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:39:25.357378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:39:25.357378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:25.357378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:25.357378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:39:25.357378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:39:25.357694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:39:25.357694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:39:25.357694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:25.357694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:25.357694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:25.357694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:39:25.357694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:25.357694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:25.357694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:25.357694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:39:25.357694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:25.357694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:39:25.357694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:39:25.357694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:25.357694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:39:25.357694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:25.357694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:39:25.357694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:39:25.357694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:25.358060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:39:25.358060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:25.358060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:39:30.096692 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:39:30.097725 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:39:30.097811 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:39:30.098462 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:39:30.122917 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:30.123330 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:30.123881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:39:30.123881 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:30.123881 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:39:30.123881 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:39:30.123881 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:30.123881 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:39:30.123881 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:30.123881 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:39:30.124925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.026 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:30.124925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:39:30.124925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:30.124925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:39:30.124925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:39:30.124925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:39:30.124925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:39:30.124925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:30.124925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:30.124925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:30.124925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:39:30.124925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:30.125225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:30.125225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:30.125225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:39:30.125225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:30.125225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:39:30.125225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:39:30.125225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:30.125225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:39:30.125225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:30.125225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:39:30.125225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:39:30.125225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:30.125225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:39:30.125531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:30.125531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:39:30.125531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:30.125531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:39:30.125531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:39:30.125531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:39:30.125531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:39:30.125531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:39:30.125531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:30.125531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:39:30.125531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:39:30.125531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:39:30.125531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:39:30.125531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:39:30.125531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:30.125859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:30.125859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:39:30.125859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:39:30.125859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:39:30.125859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:39:30.125859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:30.125859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:30.125859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:30.125859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:39:30.125859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:30.125859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:30.125859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:30.125859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:39:30.125859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:30.125859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:39:30.125859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:39:30.125859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:30.125859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:39:30.125859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:30.126308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:39:30.126308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:39:30.126308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:30.126308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:39:30.126308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:30.126308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:39:30.330110 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:39:30.330688 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:39:30.330858 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:39:30.331868 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:30.332138 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:30.332310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:30.333221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:39:30.333221 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:30.333221 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:39:30.333221 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:39:30.333221 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:30.333221 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:39:30.333221 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:30.333221 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:39:30.334666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:30.334666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:39:30.334666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:30.334666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:39:30.334666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:39:30.334666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:39:30.334666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:39:30.334666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:30.334666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:30.334666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:30.334666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:39:30.334666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:30.335239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:30.335239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:30.335239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:39:30.335239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:30.335239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:39:30.335239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:39:30.335239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:30.335239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:39:30.335239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:30.335239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:39:30.335239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:39:30.335239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:30.335239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:39:30.335773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:30.335773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:39:30.337333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:30.337333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:39:30.337333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:39:30.337333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:39:30.337333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:39:30.337333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:39:30.337333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:30.337333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:39:30.337333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:39:30.337333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:39:30.337333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:39:30.337333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:39:30.337333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:30.337333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:30.337333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:39:30.337333 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:39:30.337972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:39:30.337972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:39:30.337972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:30.337972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:30.337972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:30.337972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:39:30.337972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:30.337972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:30.337972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:30.337972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:39:30.337972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:30.337972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:39:30.337972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:39:30.337972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:30.337972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:39:30.337972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:30.337972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:39:30.337972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:39:30.337972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:30.338665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:39:30.338665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:30.338665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:39:30.339791 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:39:30.339791 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:39:30.339791 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:39:30.339996 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:39:30.340386 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:30.341153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:30.342134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:39:30.342134 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:30.342134 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:39:30.342134 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:39:30.342134 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:30.342134 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:39:30.342134 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:30.342134 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:39:30.342607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:30.342607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:39:30.342607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:39:30.342607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:39:30.342607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:30.342607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:39:30.342607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:30.342607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:39:30.342607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:30.342607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:39:30.342607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:30.342607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:39:30.342607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:30.342607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:39:30.342607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:39:30.343293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:30.343293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:39:30.343293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:30.343293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:39:30.343293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:39:30.343293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:30.343293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:39:30.343293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:30.343293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:39:30.343716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:30.343716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:39:30.343716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:30.343716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:39:30.343716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:39:30.343716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:39:30.343716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:39:30.343716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:39:30.343716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:39:30.343716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:39:30.343716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:30.343716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:30.344217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:30.344217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:39:30.344217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:30.344217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:30.344217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:30.344217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:39:30.344217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:30.344217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:39:30.344217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:39:30.344217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:30.344217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:39:30.344217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:30.344217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:39:30.344217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:39:30.344753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:30.344753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:39:30.344753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:30.344753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:39:30.344753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:30.344753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:39:30.344753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:39:30.344753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:39:30.344753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:39:30.344753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:39:30.344753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:30.344753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:39:30.344753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:39:30.344753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:39:30.345287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:39:30.345287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:39:30.345287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:30.345287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:30.345287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:39:30.345287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:39:30.345287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:39:30.345287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:39:30.345287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:39:30.345287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:39:30.345287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:39:30.345287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:30.345287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:30.345287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:30.345287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:39:30.345287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:30.345287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:30.345287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:30.345287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:39:30.348688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:30.348688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:39:30.348688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:39:30.348688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:30.348688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:39:30.348688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:30.348688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:39:30.348688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:39:30.348688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:30.348688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:39:30.348688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:30.348688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:39:30.348688 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:39:30.348688 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:39:30.349206 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:39:30.349206 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:39:30.349206 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:30.349358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:30.350131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:39:30.350131 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:30.350131 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:39:30.350131 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:39:30.350131 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:30.350131 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:39:30.350131 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:30.350131 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:39:30.350864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:30.350864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:39:30.350864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:39:30.350864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:39:30.350864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:30.350864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:39:30.350864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:30.350864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:39:30.350864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:30.350864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:39:30.350864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:30.350864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:39:30.350864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:30.350864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:39:30.350864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:39:30.351237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:30.351237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:39:30.351237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:30.351237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:39:30.351237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:39:30.351237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:30.351237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:39:30.351237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:30.351237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:39:30.351710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:30.351710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:39:30.351710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:30.351710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:39:30.351710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:39:30.351710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:39:30.351710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:39:30.351710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:39:30.351710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:39:30.351710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:30.351710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:30.352053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:30.352053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:39:30.352053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:30.352053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:30.352053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:30.352053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:39:30.352053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:30.352053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:39:30.352053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:39:30.352053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:30.352053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:39:30.352053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:30.352053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:39:30.352053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:39:30.352448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:30.352448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:39:30.352448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:30.352448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:39:30.352789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:30.352789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:39:30.352789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:39:30.352789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:39:30.352789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:39:30.352789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:39:30.352789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:30.352789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:39:30.352789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:39:30.352789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:39:30.352789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:39:30.352789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:39:30.352789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:30.352789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:30.352789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:39:30.352789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:39:30.352789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:39:30.353245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:39:30.353245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:39:30.353245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:39:30.353245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:30.353245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:30.353245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:30.353245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:39:30.353245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:30.353245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:30.353245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:30.353245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:39:30.353245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:30.353245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:39:30.353245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:39:30.353245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:30.353245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:39:30.353245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:30.353245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:39:30.353245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:39:30.353951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:30.353951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:39:30.353951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:30.353951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:39:30.355109 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:39:30.355335 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:39:30.355465 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:39:30.355618 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:39:30.356880 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:30.357026 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:30.357142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:30.357644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:39:30.357644 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:30.357644 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:39:30.357644 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:39:30.357644 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:30.357644 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:39:30.357644 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:30.357644 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:39:30.358435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:30.358435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:39:30.358435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:30.358435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:39:30.358435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:39:30.358435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:39:30.358435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:39:30.358435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:30.358435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:30.358435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:30.358435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:39:30.358435 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:30.358934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:30.358934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:30.358934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:39:30.358934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:30.358934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:39:30.358934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:39:30.358934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:30.358934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:39:30.358934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:30.358934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:39:30.358934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:39:30.358934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:30.358934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:39:30.359529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:30.359529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:39:30.359529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:30.359529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:39:30.359529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:39:30.359529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:39:30.359529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:39:30.359529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:39:30.359529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:30.359529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:39:30.359529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:39:30.359529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:39:30.359529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:39:30.359529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:39:30.359529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:30.360117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:30.360117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:39:30.360117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:39:30.360117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:39:30.360117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:39:30.360117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:30.360117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:30.360117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:30.360117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:39:30.360117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:30.360117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:30.360117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:30.360117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:39:30.360117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:30.360117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:39:30.360117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:39:30.360117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:30.360117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:39:30.360117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:30.360885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:39:30.360885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:39:30.360885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:30.360885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:39:30.360885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:30.360885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:39:34.273602 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:39:34.274867 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_for_inconsistencies {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:39:34.275039 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Checking Neutron and OVN revision consistency. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:39:34.285303 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Synchronizing Neutron and OVN databases started {{(pid=105581) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:440}}
May 09 14:39:34.285449 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Number of inconsistencies found at create/update: networks=4, routers=1, router_ports=1 {{(pid=105581) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:419}}
May 09 14:39:34.285574 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Fixing resource 0b4a5724-2518-4e33-a6d3-0d92c749f303 (type: networks) at create/update {{(pid=105581) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 14:39:34.317623 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-0b4a5724-2518-4e33-a6d3-0d92c749f303, resource={'id': '0b4a5724-2518-4e33-a6d3-0d92c749f303', 'name': 'private', 'tenant_id': '5796380176014a32af8ff613b146faed', 'project_id': '5796380176014a32af8ff613b146faed', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['23eb2e2e-ec2f-4f98-bc47-7125a2f92952', 'db2636e9-0880-488d-9aa3-e100ce5eeb9f'], 'standard_attr_id': 12, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-09T14:04:20Z', 'updated_at': '2026-05-09T14:04:23Z', 'revision_number': 3, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 52278}, resource_type=networks, if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:34.317819 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-0b4a5724-2518-4e33-a6d3-0d92c749f303, col_values=(('external_ids', {'neutron:network_name': 'private', 'neutron:mtu': '1372', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:34.318212 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-f71b8cb1-fe70-438c-84f9-bbb7008a596d, external_ids_update=None, columns={'tag': 52278}, if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:34.318212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:34.318719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:39:34.318719 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:34.318719 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:39:34.318719 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:39:34.318719 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:34.318719 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:39:34.318719 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:34.318719 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:39:34.319837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Failed to fix resource 0b4a5724-2518-4e33-a6d3-0d92c749f303 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:34.319837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:39:34.319837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 14:39:34.319837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 14:39:34.319837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 14:39:34.319837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 14:39:34.319837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 09 14:39:34.319837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:39:34.319837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:34.319837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:34.319837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:34.320357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:39:34.320357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:34.320357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:34.320357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:34.320357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:39:34.320357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:34.320357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:39:34.320357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:39:34.320357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:34.320357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:39:34.320357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:34.320357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:39:34.320357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:39:34.320357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:34.320981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:39:34.320981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:34.320981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:39:34.320981 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Fixing resource a7d094cd-23f3-4e0c-9e88-24be6ee17b2c (type: networks) at create/update {{(pid=105581) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 14:39:34.346787 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-a7d094cd-23f3-4e0c-9e88-24be6ee17b2c, resource={'id': 'a7d094cd-23f3-4e0c-9e88-24be6ee17b2c', 'name': 'public', 'tenant_id': '5796380176014a32af8ff613b146faed', 'project_id': '5796380176014a32af8ff613b146faed', 'admin_state_up': True, 'mtu': 1430, 'status': 'ACTIVE', 'subnets': ['2f0821e3-c6a2-49d6-80ee-3480a17ba8d2', '644269b7-6b84-4d8a-ad28-d59a7c6b7c5e'], 'standard_attr_id': 18, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': True, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'is_default': True, 'tags': [], 'created_at': '2026-05-09T14:04:26Z', 'updated_at': '2026-05-09T14:04:34Z', 'revision_number': 3, 'provider:network_type': 'flat', 'provider:physical_network': 'public', 'provider:segmentation_id': None}, resource_type=networks, if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:34.346970 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-a7d094cd-23f3-4e0c-9e88-24be6ee17b2c, col_values=(('external_ids', {'neutron:network_name': 'public', 'neutron:mtu': '1430', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'flat', 'neutron:provnet-physical-network': 'public'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false', 'fdb_age_threshold': '300', 'broadcast-arps-to-all-routers': 'true'})), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:34.347352 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-08fea115-9b05-4667-a0d0-487c303084a5, external_ids_update=None, columns={'tag': []}, if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:34.347607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:34.348068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:39:34.348068 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:34.348068 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:39:34.348068 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:39:34.348068 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:34.348068 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:39:34.348068 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:34.348068 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:39:34.349131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Failed to fix resource a7d094cd-23f3-4e0c-9e88-24be6ee17b2c (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:34.349131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:39:34.349131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 14:39:34.349131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 14:39:34.349131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 14:39:34.349131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 14:39:34.349131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 09 14:39:34.349131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:39:34.349131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:34.349131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:34.349131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:34.349877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:39:34.349877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:34.349877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:34.349877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:34.349877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:39:34.349877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:34.349877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:39:34.349877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:39:34.349877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:34.349877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:39:34.349877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:34.349877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:39:34.349877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:39:34.349877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:34.350595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:39:34.350595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:34.350595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:39:34.350595 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Fixing resource 3e80d551-6de5-4370-806b-36d1db5f6fda (type: networks) at create/update {{(pid=105581) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 14:39:34.374135 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-3e80d551-6de5-4370-806b-36d1db5f6fda, resource={'id': '3e80d551-6de5-4370-806b-36d1db5f6fda', 'name': 'admin_net', 'tenant_id': '590c644b03ae4bed862ecf492c6b2eb2', 'project_id': '590c644b03ae4bed862ecf492c6b2eb2', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['453849e5-22a7-43ca-911e-9401ed2b0d74'], 'standard_attr_id': 42, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-09T14:07:41Z', 'updated_at': '2026-05-09T14:07:42Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 35952}, resource_type=networks, if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:34.374278 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-3e80d551-6de5-4370-806b-36d1db5f6fda, col_values=(('external_ids', {'neutron:network_name': 'admin_net', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:34.374552 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-f84050fb-ff65-46ce-b744-016e1ff15440, external_ids_update=None, columns={'tag': 35952}, if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:34.374628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:34.375431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:39:34.375431 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:34.375431 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:39:34.375431 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:39:34.375431 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:34.375431 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:39:34.375431 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:34.375431 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:39:34.375775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Failed to fix resource 3e80d551-6de5-4370-806b-36d1db5f6fda (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:34.375775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:39:34.375775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 14:39:34.375775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 14:39:34.375775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 14:39:34.375775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 14:39:34.375775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 09 14:39:34.375775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:39:34.375775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:34.375775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:34.375775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:34.376208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:39:34.376208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:34.376208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:34.376208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:34.376208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:39:34.376208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:34.376208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:39:34.376208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:39:34.376208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:34.376208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:39:34.376208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:34.376208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:39:34.376208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:39:34.376208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:34.376729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:39:34.376729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:34.376729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:39:34.376729 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Fixing resource 632a2ddb-5634-4bb3-aad0-f05d97abd45e (type: networks) at create/update {{(pid=105581) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 14:39:34.404469 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-632a2ddb-5634-4bb3-aad0-f05d97abd45e, resource={'id': '632a2ddb-5634-4bb3-aad0-f05d97abd45e', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-5urlz', 'tenant_id': '1f17f4a2a7ac4543a1253522e61993d2', 'project_id': '1f17f4a2a7ac4543a1253522e61993d2', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['32b523fe-135f-4f9e-a98b-fc0537fcdb35'], 'standard_attr_id': 52, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-09T14:08:15Z', 'updated_at': '2026-05-09T14:08:16Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 39245}, resource_type=networks, if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:34.404890 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-632a2ddb-5634-4bb3-aad0-f05d97abd45e, col_values=(('external_ids', {'neutron:network_name': 'k8s-clusterapi-cluster-magnum-system-kube-5urlz', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:34.405398 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-bca44f95-fe96-432e-84b8-1edc05e10ce0, external_ids_update=None, columns={'tag': 39245}, if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:34.405601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:34.406490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:39:34.406490 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:34.406490 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:39:34.406490 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:39:34.406490 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:34.406490 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:39:34.406490 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:34.406490 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:39:34.407601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Failed to fix resource 632a2ddb-5634-4bb3-aad0-f05d97abd45e (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:34.407601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:39:34.407601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 14:39:34.407601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 14:39:34.407601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 14:39:34.407601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 14:39:34.407601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 09 14:39:34.407601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:39:34.407601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:34.407601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:34.407601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:34.407920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:39:34.407920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:34.407920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:34.407920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:34.407920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:39:34.407920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:34.407920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:39:34.407920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:39:34.407920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:34.407920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:39:34.407920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:34.407920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:39:34.407920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:39:34.407920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:34.408318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:39:34.408318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:34.408318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:39:34.408318 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Fixing resource 09e042cf-39a9-42c8-b683-b29ffc42b049 (type: routers) at create/update {{(pid=105581) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 14:39:34.652592 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, resource={'id': '09e042cf-39a9-42c8-b683-b29ffc42b049', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-5urlz', 'project_id': '1f17f4a2a7ac4543a1253522e61993d2', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': 'a7d094cd-23f3-4e0c-9e88-24be6ee17b2c', 'external_fixed_ips': [{'subnet_id': '2f0821e3-c6a2-49d6-80ee-3480a17ba8d2', 'ip_address': '172.24.5.175'}, {'subnet_id': '644269b7-6b84-4d8a-ad28-d59a7c6b7c5e', 'ip_address': '2001:db8::d8'}], 'enable_snat': True}, 'gw_port_id': '71ded037-e812-40c0-8d8f-70c141c74323', 'description': 'Created by cluster-api-provider-openstack cluster magnum-system-kube-5urlz', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': 'a7d094cd-23f3-4e0c-9e88-24be6ee17b2c', 'external_fixed_ips': [{'ip_address': '172.24.5.175', 'subnet_id': '2f0821e3-c6a2-49d6-80ee-3480a17ba8d2'}, {'ip_address': '2001:db8::d8', 'subnet_id': '644269b7-6b84-4d8a-ad28-d59a7c6b7c5e'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2026-05-09T14:08:17Z', 'updated_at': '2026-05-09T14:08:19Z', 'revision_number': 3, 'tenant_id': '1f17f4a2a7ac4543a1253522e61993d2'}, resource_type=routers, if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:34.652838 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, columns={'external_ids': {'neutron:router_name': 'k8s-clusterapi-cluster-magnum-system-kube-5urlz', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:34.653253 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-a7d094cd-23f3-4e0c-9e88-24be6ee17b2c, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "71ded037-e812-40c0-8d8f-70c141c74323"')], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:34.653386 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-a7d094cd-23f3-4e0c-9e88-24be6ee17b2c, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "71ded037-e812-40c0-8d8f-70c141c74323"')], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:34.653760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:34.654577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:39:34.654577 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:34.654577 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:39:34.654577 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:39:34.654577 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:34.654577 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:39:34.654577 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:34.654577 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:39:34.654577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Unable to update router 09e042cf-39a9-42c8-b683-b29ffc42b049. Error: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:34.656106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Failed to fix resource 09e042cf-39a9-42c8-b683-b29ffc42b049 (type: routers): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:34.656106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:39:34.656106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 14:39:34.656106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 14:39:34.656106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 14:39:34.656106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 14:39:34.656106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1635, in update_router
May 09 14:39:34.656106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with excutils.save_and_reraise_exception():
May 09 14:39:34.656106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 271, in __exit__
May 09 14:39:34.656106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.force_reraise()
May 09 14:39:34.656106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 233, in force_reraise
May 09 14:39:34.658059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise self.value
May 09 14:39:34.658059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1569, in update_router
May 09 14:39:34.658059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:39:34.658059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:34.658059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:34.658059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:34.658059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:39:34.658059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:34.658059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:34.658059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:34.658059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:39:34.658059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:34.658059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:39:34.658059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:39:34.658693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:34.658693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:39:34.658693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:34.658693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:39:34.658693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:39:34.658693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:34.658693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:39:34.658693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:34.658693 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:39:34.658693 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Fixing resource 71ded037-e812-40c0-8d8f-70c141c74323 (type: router_ports) at create/update {{(pid=105581) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 14:39:34.757893 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-71ded037-e812-40c0-8d8f-70c141c74323, resource={'id': '71ded037-e812-40c0-8d8f-70c141c74323', 'name': '', 'network_id': 'a7d094cd-23f3-4e0c-9e88-24be6ee17b2c', 'project_id': '', 'mac_address': 'fa:16:3e:c7:86:37', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '09e042cf-39a9-42c8-b683-b29ffc42b049', 'device_owner': 'network:router_gateway', 'standard_attr_id': 57, 'fixed_ips': [{'subnet_id': '2f0821e3-c6a2-49d6-80ee-3480a17ba8d2', 'ip_address': '172.24.5.175'}, {'subnet_id': '644269b7-6b84-4d8a-ad28-d59a7c6b7c5e', 'ip_address': '2001:db8::d8'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0000173183', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'ovs_create_tap': False, 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2026-05-09T14:08:17Z', 'updated_at': '2026-05-09T14:08:19Z', 'revision_number': 5, 'tenant_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:34.758254 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-71ded037-e812-40c0-8d8f-70c141c74323, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': '2f0821e3-c6a2-49d6-80ee-3480a17ba8d2 644269b7-6b84-4d8a-ad28-d59a7c6b7c5e', 'neutron:network_name': 'neutron-a7d094cd-23f3-4e0c-9e88-24be6ee17b2c', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-09e042cf-39a9-42c8-b683-b29ffc42b049'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['2001:db8::d8/64', '172.24.5.175/24'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:34.758761 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=71ded037-e812-40c0-8d8f-70c141c74323, lrouter_port=lrp-71ded037-e812-40c0-8d8f-70c141c74323, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:34.759106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:34.759877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:39:34.759877 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:34.759877 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:39:34.759877 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:39:34.759877 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:34.759877 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:39:34.759877 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:34.759877 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:39:34.760744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Maintenance task: Failed to fix resource 71ded037-e812-40c0-8d8f-70c141c74323 (type: router_ports): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:34.760744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:39:34.760744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 14:39:34.760744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 14:39:34.760744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 14:39:34.760744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 14:39:34.760744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1976, in update_router_port
May 09 14:39:34.760744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:39:34.760744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:34.760744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:34.760744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:34.761133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:39:34.761133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:34.761133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:34.761133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:34.761133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:39:34.761133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:34.761133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:39:34.761133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:39:34.761133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:34.761133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:39:34.761133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:34.761133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:39:34.761133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:39:34.761133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:34.761553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:39:34.761553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:34.761553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:39:34.761553 np0000173183 neutron-ovn-maintenance-worker[105581]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_for_inconsistencies finished in 0.486 seconds
May 09 14:39:35.097943 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:39:35.098350 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:39:35.098554 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:39:35.099865 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:39:35.129824 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:35.130189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:35.131027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:39:35.131027 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:35.131027 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:39:35.131027 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:39:35.131027 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:35.131027 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:39:35.131027 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:35.131027 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:39:35.131997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:35.131997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:39:35.131997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:35.131997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:39:35.131997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:39:35.131997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:39:35.131997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:39:35.131997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:35.131997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:35.131997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:35.131997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:39:35.131997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:35.132607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:35.132607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:35.132607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:39:35.132607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:35.132607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:39:35.132607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:39:35.132607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:35.132607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:39:35.132607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:35.132607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:39:35.132607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:39:35.132607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:35.132607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:39:35.133131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:35.133131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:39:35.133131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:35.133131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:39:35.133131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:39:35.133131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:39:35.133131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:39:35.133131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:39:35.133131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:35.133131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:39:35.133131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:39:35.133131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:39:35.133131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:39:35.133131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:39:35.133131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:35.133753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:35.133753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:39:35.133753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:39:35.133753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:39:35.133753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:39:35.133753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:35.133753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:35.133753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:35.133753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:39:35.133753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:35.133753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:35.133753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:35.133753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:39:35.133753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:35.133753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:39:35.133753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:39:35.133753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:35.133753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:39:35.133753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:35.134571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:39:35.134571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:39:35.134571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:35.134571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:39:35.134571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:35.134571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:39:35.330559 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:39:35.331290 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:39:35.331290 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:39:35.332035 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:35.332364 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:35.332629 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:35.333503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:39:35.333503 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:35.333503 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:39:35.333503 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:39:35.333503 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:35.333503 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:39:35.333503 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:35.333503 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:39:35.334935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:35.334935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:39:35.334935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:35.334935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:39:35.334935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:39:35.334935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:39:35.334935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:39:35.334935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:35.334935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:35.334935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:35.334935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:39:35.334935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:35.335530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:35.335530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:35.335530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:39:35.335530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:35.335530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:39:35.335530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:39:35.335530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:35.335530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:39:35.335530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:35.335530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:39:35.335530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:39:35.335530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:35.335530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:39:35.336111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:35.336111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:39:35.336946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:35.336946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:39:35.336946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:39:35.336946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:39:35.336946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:39:35.336946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:39:35.336946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:35.336946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:39:35.336946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:39:35.336946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:39:35.336946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:39:35.336946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:39:35.336946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:35.336946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:35.336946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:39:35.336946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:39:35.337358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:39:35.337358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:39:35.337358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:35.337358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:35.337358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:35.337358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:39:35.337358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:35.337358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:35.337358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:35.337358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:39:35.337358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:35.337358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:39:35.337358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:39:35.337358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:35.337358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:39:35.337358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:35.337358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:39:35.337358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:39:35.337358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:35.337884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:39:35.337884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:35.337884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:39:35.339765 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:39:35.340055 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:39:35.340222 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:39:35.340397 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:39:35.340933 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:35.341224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:35.341928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:39:35.341928 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:35.341928 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:39:35.341928 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:39:35.341928 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:35.341928 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:39:35.341928 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:35.341928 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:39:35.342751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:35.342751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:39:35.342751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:39:35.342751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:39:35.342751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:35.342751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:39:35.342751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:35.342751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:39:35.342751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:35.342751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:39:35.342751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:35.342751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:39:35.342751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:35.342751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:39:35.342751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:39:35.343184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:35.343184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:39:35.343184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:35.343184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:39:35.343184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:39:35.343184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:35.343184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:39:35.343184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:35.343184 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:39:35.343615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:35.343615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:39:35.343615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:35.343615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:39:35.343615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:39:35.343615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:39:35.343615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:39:35.343615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:39:35.343615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:39:35.343615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:39:35.343615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:35.343615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:35.343940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:35.343940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:39:35.343940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:35.343940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:35.343940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:35.343940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:39:35.343940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:35.343940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:39:35.343940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:39:35.343940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:35.343940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:39:35.343940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:35.343940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:39:35.343940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:39:35.344323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:35.344323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:39:35.344323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:35.344323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:39:35.344597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:35.344597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:39:35.344597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:39:35.344597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:39:35.344597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:39:35.344597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:39:35.344597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:35.344597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:39:35.344597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:39:35.344597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:39:35.344597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:39:35.344597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:39:35.344597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:35.344597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:35.344597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:39:35.344597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:39:35.345056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:39:35.345056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:39:35.345056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:39:35.345056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:39:35.345056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:39:35.345056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:35.345056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:35.345056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:35.345056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:39:35.345056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:35.345056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:35.345056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:35.345056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:39:35.345056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:35.345056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:39:35.345056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:39:35.345056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:35.345056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:39:35.345550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:35.345550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:39:35.345550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:39:35.345550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:35.345550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:39:35.345550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:35.345550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:39:35.348426 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:39:35.348582 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:39:35.348684 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:39:35.348801 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:39:35.349312 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:35.349723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:35.350495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:39:35.350495 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:35.350495 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:39:35.350495 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:39:35.350495 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:35.350495 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:39:35.350495 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:35.350495 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:39:35.351228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:35.351228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:39:35.351228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:39:35.351228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:39:35.351228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:35.351228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:39:35.351228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:35.351228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:39:35.351228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:35.351228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:39:35.351228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:35.351228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:39:35.351228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:35.351228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:39:35.351228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:39:35.351647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:35.351647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:39:35.351647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:35.351647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:39:35.351647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:39:35.351647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:35.351647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:39:35.351647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:35.351647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:39:35.352084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:35.352084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:39:35.352084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:35.352084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:39:35.352084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:39:35.352084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:39:35.352084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:39:35.352084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:39:35.352084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:39:35.352084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:35.352084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:35.352423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:35.352423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:39:35.352423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:35.352423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:35.352423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:35.352423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:39:35.352423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:35.352423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:39:35.352423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:39:35.352423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:35.352423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:39:35.352423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:35.352423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:39:35.352423 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:39:35.352813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:35.352813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:39:35.352813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:35.352813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:39:35.353117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:35.353117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:39:35.353117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:39:35.353117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:39:35.353117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:39:35.353117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:39:35.353117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:35.353117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:39:35.353117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:39:35.353117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:39:35.353117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:39:35.353117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:39:35.353117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:35.353117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:35.353117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:39:35.353117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:39:35.353117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:39:35.353609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:39:35.353609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:39:35.353609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:39:35.353609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:35.353609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:35.353609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:35.353609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:39:35.353609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:35.353609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:35.353609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:35.353609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:39:35.353609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:35.353609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:39:35.353609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:39:35.353609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:35.353609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:39:35.353609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:35.353609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:39:35.353609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:39:35.354128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:35.354128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:39:35.354128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:35.354128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:39:35.355345 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:39:35.355557 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:39:35.355658 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:39:35.355812 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:39:35.356264 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:35.356643 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:35.356857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:35.357553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:39:35.357553 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:35.357553 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:39:35.357553 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:39:35.357553 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:35.357553 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:39:35.357553 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:35.357553 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:39:35.358369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:35.358369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:39:35.358369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:35.358369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:39:35.358369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:39:35.358369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:39:35.358369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:39:35.358369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:35.358369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:35.358369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:35.358369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:39:35.358369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:35.358749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:35.358749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:35.358749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:39:35.358749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:35.358749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:39:35.358749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:39:35.358749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:35.358749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:39:35.358749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:35.358749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:39:35.358749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:39:35.358749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:35.358749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:39:35.359111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:35.359111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:39:35.359309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:35.359309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:39:35.359309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:39:35.359309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:39:35.359309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:39:35.359309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:39:35.359309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:35.359309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:39:35.359309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:39:35.359309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:39:35.359309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:39:35.359309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:39:35.359309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:35.359309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:35.359309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:39:35.359309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:39:35.359811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:39:35.359811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:39:35.359811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:35.359811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:35.359811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:35.359811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:39:35.359811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:35.359811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:35.359811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:35.359811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:39:35.359811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:35.359811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:39:35.359811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:39:35.359811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:35.359811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:39:35.359811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:35.359811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:39:35.359811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:39:35.359811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:35.360341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:39:35.360341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:35.360341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:39:40.098647 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:39:40.099587 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:39:40.099834 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:39:40.101106 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:39:40.143287 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:40.143651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:40.144257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:39:40.144257 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:40.144257 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:39:40.144257 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:39:40.144257 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:40.144257 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:39:40.144257 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:40.144257 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:39:40.145320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:40.145320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:39:40.145320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:40.145320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:39:40.145320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:39:40.145320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:39:40.145320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:39:40.145320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:40.145320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:40.145320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:40.145320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:39:40.145320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:40.145765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:40.145765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:40.145765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:39:40.145765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:40.145765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:39:40.145765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:39:40.145765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:40.145765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:39:40.145765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:40.145765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:39:40.145765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:39:40.145765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:40.145765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:39:40.146207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:40.146207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:39:40.146631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:40.146631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:39:40.146631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:39:40.146631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:39:40.146631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:39:40.146631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:39:40.146631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:40.146631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:39:40.146631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:39:40.146631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:39:40.146631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:39:40.146631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:39:40.146631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:40.146631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:40.146631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:39:40.146631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:39:40.148308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:39:40.148308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:39:40.148308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:40.148308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:40.148308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:40.148308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:39:40.148308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:40.148308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:40.148308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:40.148308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:39:40.148308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:40.148308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:39:40.148308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:39:40.148308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:40.148308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:39:40.148308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:40.148308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:39:40.148308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:39:40.148308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:40.149469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:39:40.149469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:40.149469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:39:40.331234 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:39:40.331649 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:39:40.331834 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:39:40.333020 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:40.333366 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:40.333642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:40.334558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:39:40.334558 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:40.334558 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:39:40.334558 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:39:40.334558 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:40.334558 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:39:40.334558 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:40.334558 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:39:40.336137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:40.336137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:39:40.336137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:40.336137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:39:40.336137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:39:40.336137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:39:40.336137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:39:40.336137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:40.336137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:40.336137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:40.336137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:39:40.336137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:40.337043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:40.337043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:40.337043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:39:40.337043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:40.337043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:39:40.337043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:39:40.337043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:40.337043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:39:40.337043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:40.337043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:39:40.337043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:39:40.337043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:40.337043 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:39:40.337369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:40.337369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:39:40.337879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:40.337879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:39:40.337879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:39:40.337879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:39:40.337879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:39:40.337879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:39:40.337879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:40.337879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:39:40.337879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:39:40.337879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:39:40.337879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:39:40.337879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:39:40.337879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:40.337879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:40.337879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:39:40.337879 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:39:40.338358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:39:40.338358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:39:40.338358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:40.338358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:40.338358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:40.338358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:39:40.338358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:40.338358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:40.338358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:40.338358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:39:40.338358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:40.338358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:39:40.338358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:39:40.338358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:40.338358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:39:40.338358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:40.338358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:39:40.338358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:39:40.338358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:40.338880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:39:40.338880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:40.338880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:39:40.340474 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:39:40.340817 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:39:40.341069 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:39:40.341202 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:39:40.341829 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:40.342080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:40.342715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:39:40.342715 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:40.342715 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:39:40.342715 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:39:40.342715 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:40.342715 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:39:40.342715 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:40.342715 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:39:40.343669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:40.343669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:39:40.343669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:39:40.343669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:39:40.343669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:40.343669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:39:40.343669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:40.343669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:39:40.343669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:40.343669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:39:40.343669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:40.343669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:39:40.343669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:40.343669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:39:40.343669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:39:40.344115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:40.344115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:39:40.344115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:40.344115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:39:40.344115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:39:40.344115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:40.344115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:39:40.344115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:40.344115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:39:40.344661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:40.344661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:39:40.344661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:40.344661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:39:40.344661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:39:40.344661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:39:40.344661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:39:40.344661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:39:40.344661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:39:40.344661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:39:40.344661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:40.344661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:40.345014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:40.345014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:39:40.345014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:40.345014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:40.345014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:40.345014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:39:40.345014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:40.345014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:39:40.345014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:39:40.345014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:40.345014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:39:40.345014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:40.345014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:39:40.345014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:39:40.345737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:40.345737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:39:40.345737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:40.345737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:39:40.345870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:40.345870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:39:40.345870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:39:40.345870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:39:40.345870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:39:40.345870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:39:40.345870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:40.345870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:39:40.345870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:39:40.345870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:39:40.345870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:39:40.345870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:39:40.345870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:40.345870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:40.345870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:39:40.345870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:39:40.346257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:39:40.346257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:39:40.346257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:39:40.346257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:39:40.346257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:39:40.346257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:40.346257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:40.346257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:40.346257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:39:40.346257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:40.346257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:40.346257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:40.346257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:39:40.346257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:40.346257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:39:40.346257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:39:40.346257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:40.346257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:39:40.347654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:40.347654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:39:40.347654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:39:40.347654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:40.347654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:39:40.347654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:40.347654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:39:40.351630 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:39:40.351630 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:39:40.351630 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:39:40.351630 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:39:40.351630 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:40.351630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:40.351851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:39:40.351851 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:40.351851 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:39:40.351851 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:39:40.351851 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:40.351851 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:39:40.351851 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:40.351851 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:39:40.351851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:40.351851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:39:40.351851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:39:40.351851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:39:40.351851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:40.351851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:39:40.351851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:40.351851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:39:40.351851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:40.351851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:39:40.352348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:40.352348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:39:40.352348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:40.352348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:39:40.352348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:39:40.352348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:40.352348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:39:40.352348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:40.352348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:39:40.352348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:39:40.352348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:40.352348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:39:40.352348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:40.352348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:39:40.352348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:40.353029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:39:40.353029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:40.353029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:39:40.353029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:39:40.353029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:39:40.353029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:39:40.353029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:39:40.353029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:39:40.353029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:40.353029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:40.353029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:40.353029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:39:40.353029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:40.353346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:40.353346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:40.353346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:39:40.353346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:40.353346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:39:40.353346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:39:40.353346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:40.353346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:39:40.353346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:40.353346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:39:40.353346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:39:40.353346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:40.353346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:39:40.353688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:40.353688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:39:40.353688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:40.353688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:39:40.353688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:39:40.353688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:39:40.353688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:39:40.353688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:39:40.353688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:40.353688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:39:40.353688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:39:40.353688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:39:40.353688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:39:40.353688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:39:40.353688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:40.354063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:40.354063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:39:40.354063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:39:40.354063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:39:40.354063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:39:40.354063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:39:40.354063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:39:40.354063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:40.354063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:40.354063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:40.354063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:39:40.354063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:40.354063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:40.354063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:40.354063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:39:40.354063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:40.354063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:39:40.354063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:39:40.354063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:40.354541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:39:40.354541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:40.354541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:39:40.354541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:39:40.354541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:40.354541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:39:40.354541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:40.354541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:39:40.355566 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:39:40.355812 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:39:40.355940 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:39:40.356122 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:39:40.357615 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:40.357615 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:40.357615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:40.357774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:39:40.357774 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:40.357774 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:39:40.357774 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:39:40.357774 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:40.357774 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:39:40.357774 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:40.357774 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:39:40.358622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:40.358622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:39:40.358622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:40.358622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:39:40.358622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:39:40.358622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:39:40.358622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:39:40.358622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:40.358622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:40.358622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:40.358622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:39:40.358622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:40.358861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:40.358861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:40.358861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:39:40.358861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:40.358861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:39:40.358861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:39:40.358861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:40.358861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:39:40.358861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:40.358861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:39:40.358861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:39:40.358861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:40.358861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:39:40.359091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:40.359091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:39:40.359594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:40.359594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:39:40.359594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:39:40.359594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:39:40.359594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:39:40.359594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:39:40.359594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:40.359594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:39:40.359594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:39:40.359594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:39:40.359594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:39:40.359594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:39:40.359594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:40.359594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:40.359594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:39:40.359594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:39:40.359889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:39:40.359889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:39:40.359889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:40.359889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:40.359889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:40.359889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:39:40.359889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:40.359889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:40.359889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:40.359889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:39:40.359889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:40.359889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:39:40.359889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:39:40.359889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:40.359889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:39:40.359889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:40.359889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:39:40.359889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:39:40.359889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:40.360215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:39:40.360215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:40.360215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:39:45.099544 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:39:45.100486 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:39:45.100486 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:39:45.101602 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:39:45.137931 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:45.138344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:45.139215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:39:45.139215 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:45.139215 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:39:45.139215 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:39:45.139215 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:45.139215 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:39:45.139215 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:45.139215 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:39:45.139979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:45.139979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:39:45.139979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:45.139979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:39:45.139979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:39:45.139979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:39:45.139979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:39:45.139979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:45.139979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:45.139979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:45.139979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:39:45.139979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:45.140822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:45.140822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:45.140822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:39:45.140822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:45.140822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:39:45.140822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:39:45.140822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:45.140822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:39:45.140822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:45.140822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:39:45.140822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:39:45.140822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:45.140822 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:39:45.141367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:45.141367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:39:45.141367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:45.141367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:39:45.141367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:39:45.141367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:39:45.141367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:39:45.141367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:39:45.141367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:45.141367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:39:45.141367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:39:45.141367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:39:45.141367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:39:45.141367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:39:45.141367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:45.142434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:45.142434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:39:45.142434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:39:45.142434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:39:45.142434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:39:45.142434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:45.142434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:45.142434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:45.142434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:39:45.142434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:45.142434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:45.142434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:45.142434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:39:45.142434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:45.142434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:39:45.142434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:39:45.142434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:45.142434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:39:45.142434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:45.143234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:39:45.143234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:39:45.143234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:45.143234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:39:45.143234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:45.143234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:39:45.331877 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:39:45.332492 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:39:45.332681 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:39:45.333799 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:45.334130 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:45.334364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:45.335276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:39:45.335276 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:45.335276 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:39:45.335276 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:39:45.335276 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:45.335276 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:39:45.335276 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:45.335276 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:39:45.336819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:45.336819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:39:45.336819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:45.336819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:39:45.336819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:39:45.336819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:39:45.336819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:39:45.336819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:45.336819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:45.336819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:45.336819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:39:45.336819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:45.337381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:45.337381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:45.337381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:39:45.337381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:45.337381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:39:45.337381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:39:45.337381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:45.337381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:39:45.337381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:45.337381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:39:45.337381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:39:45.337381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:45.337381 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:39:45.337986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:45.337986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:39:45.338456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:45.338456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:39:45.338456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:39:45.338456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:39:45.338456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:39:45.338456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:39:45.338456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:45.338456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:39:45.338456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:39:45.338456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:39:45.338456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:39:45.338456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:39:45.338456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:45.338456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:45.338456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:39:45.338456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:39:45.339187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:39:45.339187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:39:45.339187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:45.339187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:45.339187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:45.339187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:39:45.339187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:45.339187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:45.339187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:45.339187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:39:45.339187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:45.339187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:39:45.339187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:39:45.339187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:45.339187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:39:45.339187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:45.339187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:39:45.339187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:39:45.339187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:45.340113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:39:45.340113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:45.340113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:39:45.341038 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:39:45.341474 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:39:45.341653 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:39:45.341835 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:39:45.342303 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:45.342641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:45.343361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:39:45.343361 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:45.343361 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:39:45.343361 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:39:45.343361 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:45.343361 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:39:45.343361 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:45.343361 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:39:45.344338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:45.344338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:39:45.344338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:39:45.344338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:39:45.344338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:45.344338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:39:45.344338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:45.344338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:39:45.344338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:45.344338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:39:45.344338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:45.344338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:39:45.344338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:45.344338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:39:45.344338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:39:45.344785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:45.344785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:39:45.344785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:45.344785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:39:45.344785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:39:45.344785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:45.344785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:39:45.344785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:45.344785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:39:45.345140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:45.345140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:39:45.345140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:45.345140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:39:45.345140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:39:45.345140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:39:45.345140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:39:45.345140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:39:45.345140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:39:45.345140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:39:45.345140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:45.345140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:45.345502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:45.345502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:39:45.345502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:45.345502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:45.345502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:45.345502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:39:45.345502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:45.345502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:39:45.345502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:39:45.345502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:45.345502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:39:45.345502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:45.345502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:39:45.345502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:39:45.345885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:45.345885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:39:45.345885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:45.345885 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:39:45.346391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:45.346391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:39:45.346391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:39:45.346391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:39:45.346391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:39:45.346391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:39:45.346391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:45.346391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:39:45.346391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:39:45.346391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:39:45.346391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:39:45.346391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:39:45.346391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:45.346391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:45.346391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:39:45.346391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:39:45.346985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:39:45.346985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:39:45.346985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:39:45.346985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:39:45.346985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:39:45.346985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:45.346985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:45.346985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:45.346985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:39:45.346985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:45.346985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:45.346985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:45.346985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:39:45.346985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:45.346985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:39:45.346985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:39:45.346985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:45.346985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:39:45.347489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:45.347489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:39:45.347489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:39:45.347489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:45.347489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:39:45.347489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:45.347489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:39:45.349319 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:39:45.349666 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:39:45.349719 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:39:45.349847 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:39:45.350278 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:45.350666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:45.351359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:39:45.351359 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:45.351359 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:39:45.351359 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:39:45.351359 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:45.351359 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:39:45.351359 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:45.351359 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:39:45.352127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:45.352127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:39:45.352127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:39:45.352127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:39:45.352127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:45.352127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:39:45.352127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:45.352127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:39:45.352127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:45.352127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:39:45.352127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:45.352127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:39:45.352127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:45.352127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:39:45.352127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:39:45.352561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:45.352561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:39:45.352561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:45.352561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:39:45.352561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:39:45.352561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:45.352561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:39:45.352561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:45.352561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:39:45.353021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:45.353021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:39:45.353021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:45.353021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:39:45.353021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:39:45.353021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:39:45.353021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:39:45.353021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:39:45.353021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:39:45.353021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:45.353021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:45.353318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:45.353318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:39:45.353318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:45.353318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:45.353318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:45.353318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:39:45.353318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:45.353318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:39:45.353318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:39:45.353318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:45.353318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:39:45.353318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:45.353318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:39:45.353318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:39:45.353720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:45.353720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:39:45.353720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:45.353720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:39:45.354071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:45.354071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:39:45.354071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:39:45.354071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:39:45.354071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:39:45.354071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:39:45.354071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:45.354071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:39:45.354071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:39:45.354071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:39:45.354071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:39:45.354071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:39:45.354071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:45.354071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:45.354071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:39:45.354071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:39:45.354071 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:39:45.354572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:39:45.354572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:39:45.354572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:39:45.354572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:45.354572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:45.354572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:45.354572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:39:45.354572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:45.354572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:45.354572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:45.354572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:39:45.354572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:45.354572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:39:45.354572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:39:45.354572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:45.354572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:39:45.354572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:45.354572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:39:45.354572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:39:45.355062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:45.355062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:39:45.355062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:45.355062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:39:45.355871 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:39:45.356106 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:39:45.356214 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:39:45.356356 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:39:45.356784 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:45.357136 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:45.357344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:45.358002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:39:45.358002 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:45.358002 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:39:45.358002 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:39:45.358002 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:45.358002 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:39:45.358002 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:45.358002 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:39:45.358832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:45.358832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:39:45.358832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:45.358832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:39:45.358832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:39:45.358832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:39:45.358832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:39:45.358832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:45.358832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:45.358832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:45.358832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:39:45.358832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:45.359172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:45.359172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:45.359172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:39:45.359172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:45.359172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:39:45.359172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:39:45.359172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:45.359172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:39:45.359172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:45.359172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:39:45.359172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:39:45.359172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:45.359172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:39:45.359557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:45.359557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:39:45.359772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:45.359772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:39:45.359772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:39:45.359772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:39:45.359772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:39:45.359772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:39:45.359772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:45.359772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:39:45.359772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:39:45.359772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:39:45.359772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:39:45.359772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:39:45.359772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:45.359772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:45.359772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:39:45.359772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:39:45.360221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:39:45.360221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:39:45.360221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:45.360221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:45.360221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:45.360221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:39:45.360221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:45.360221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:45.360221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:45.360221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:39:45.360221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:45.360221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:39:45.360221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:39:45.360221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:45.360221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:39:45.360221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:45.360221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:39:45.360221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:39:45.360221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:45.360728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:39:45.360728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:45.360728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:39:50.100207 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:39:50.101276 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:39:50.101276 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:39:50.101649 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:39:50.134193 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:50.134599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:50.135288 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:39:50.135288 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:50.135288 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:39:50.135288 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:39:50.135288 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:50.135288 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:39:50.135288 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:50.135288 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:39:50.136023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:50.136023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:39:50.136023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:50.136023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:39:50.136023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:39:50.136023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:39:50.136023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:39:50.136023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:50.136023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:50.136023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:50.136023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:39:50.136023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:50.136870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:50.136870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:50.136870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:39:50.136870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:50.136870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:39:50.136870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:39:50.136870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:50.136870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:39:50.136870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:50.136870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:39:50.136870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:39:50.136870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:50.136870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:39:50.137527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:50.137527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:39:50.137527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:50.137527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:39:50.137527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:39:50.137527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:39:50.137527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:39:50.137527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:39:50.137527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:50.137527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:39:50.137527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:39:50.137527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:39:50.137527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:39:50.137527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:39:50.137527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:50.138258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:50.138258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:39:50.138258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:39:50.138258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:39:50.138258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:39:50.138258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:50.138258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:50.138258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:50.138258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:39:50.138258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:50.138258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:50.138258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:50.138258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:39:50.138258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:50.138258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:39:50.138258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:39:50.138258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:50.138258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:39:50.138258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:50.139192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:39:50.139192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:39:50.139192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:50.139192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:39:50.139192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:50.139192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:39:50.332578 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:39:50.333162 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:39:50.333388 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:39:50.334384 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:50.334637 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:50.334799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:50.335491 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:39:50.335491 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:50.335491 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:39:50.335491 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:39:50.335491 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:50.335491 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:39:50.335491 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:50.335491 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:39:50.337073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:50.337073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:39:50.337073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:50.337073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:39:50.337073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:39:50.337073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:39:50.337073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:39:50.337073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:50.337073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:50.337073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:50.337073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:39:50.337073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:50.338312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:50.338312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:50.338312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:39:50.338312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:50.338312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:39:50.338312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:39:50.338312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:50.338312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:39:50.338312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:50.338312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:39:50.338312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:39:50.338312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:50.338312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:39:50.339583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:50.339583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:39:50.339583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:50.339583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:39:50.339583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:39:50.339583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:39:50.339583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:39:50.339583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:39:50.339583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:50.339583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:39:50.339583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:39:50.339583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:39:50.339583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:39:50.339583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:39:50.339583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:50.340976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:50.340976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:39:50.340976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:39:50.340976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:39:50.340976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:39:50.340976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:50.340976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:50.340976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:50.340976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:39:50.340976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:50.340976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:50.340976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:50.340976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:39:50.340976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:50.340976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:39:50.340976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:39:50.340976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:50.340976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:39:50.340976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:50.341768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:39:50.341768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:39:50.341768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:50.341768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:39:50.341768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:50.341768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:39:50.341768 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:39:50.341768 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:39:50.342005 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:39:50.342005 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:39:50.342542 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:50.342847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:50.343584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:39:50.343584 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:50.343584 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:39:50.343584 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:39:50.343584 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:50.343584 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:39:50.343584 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:50.343584 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:39:50.344315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:50.344315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:39:50.344315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:39:50.344315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:39:50.344315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:50.344315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:39:50.344315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:50.344315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:39:50.344315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:50.344315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:39:50.344315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:50.344315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:39:50.344315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:50.344315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:39:50.344315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:39:50.344821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:50.344821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:39:50.344821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:50.344821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:39:50.344821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:39:50.344821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:50.344821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:39:50.344821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:50.344821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:39:50.345370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:50.345370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:39:50.345370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:50.345370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:39:50.345370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:39:50.345370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:39:50.345370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:39:50.345370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:39:50.345370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:39:50.345370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:39:50.345370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:50.345370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:50.346126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:50.346126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:39:50.346126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:50.346126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:50.346126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:50.346126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:39:50.346126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:50.346126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:39:50.346126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:39:50.346126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:50.346126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:39:50.346126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:50.346126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:39:50.346126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:39:50.347328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:50.347328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:39:50.347328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:50.347328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:39:50.347328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:50.347328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:39:50.347328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:39:50.347328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:39:50.347328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:39:50.347328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:39:50.347328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:50.347328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:39:50.347328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:39:50.347328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:39:50.348538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:39:50.348538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:39:50.348538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:50.348538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:50.348538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:39:50.348538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:39:50.348538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:39:50.348538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:39:50.348538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:39:50.348538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:39:50.348538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:39:50.348538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:50.348538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:50.348538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:50.348538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:39:50.348538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:50.348538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:50.348538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:50.348538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:39:50.349069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:50.349069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:39:50.349069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:39:50.349069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:50.349069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:39:50.349069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:50.349069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:39:50.349069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:39:50.349069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:50.349069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:39:50.349069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:50.349069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:39:50.350653 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:39:50.350805 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:39:50.350907 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:39:50.351030 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:39:50.351506 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:50.351882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:50.352610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:39:50.352610 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:50.352610 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:39:50.352610 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:39:50.352610 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:50.352610 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:39:50.352610 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:50.352610 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:39:50.353426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:50.353426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:39:50.353426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:39:50.353426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:39:50.353426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:50.353426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:39:50.353426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:50.353426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:39:50.353426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:50.353426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:39:50.353426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:50.353426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:39:50.353426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:50.353426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:39:50.353426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:39:50.353799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:50.353799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:39:50.353799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:50.353799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:39:50.353799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:39:50.353799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:50.353799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:39:50.353799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:50.353799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:39:50.354366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:50.354366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:39:50.354366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:50.354366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:39:50.354366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:39:50.354366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:39:50.354366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:39:50.354366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:39:50.354366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:39:50.354366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:50.354366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:50.354704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:50.354704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:39:50.354704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:50.354704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:50.354704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:50.354704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:39:50.354704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:50.354704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:39:50.354704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:39:50.354704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:50.354704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:39:50.354704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:50.354704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:39:50.354704 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:39:50.355032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:50.355032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:39:50.355032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:50.355032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:39:50.355590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:50.355590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:39:50.355590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:39:50.355590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:39:50.355590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:39:50.355590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:39:50.355590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:50.355590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:39:50.355590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:39:50.355590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:39:50.355590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:39:50.355590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:39:50.355590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:50.355590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:50.355590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:39:50.355590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:39:50.355590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:39:50.356041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:39:50.356041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:39:50.356041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:39:50.356041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:50.356041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:50.356041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:50.356041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:39:50.356041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:50.356041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:50.356041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:50.356041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:39:50.356041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:50.356041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:39:50.356041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:39:50.356041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:50.356041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:39:50.356041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:50.356041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:39:50.356041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:39:50.356550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:50.356550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:39:50.356550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:50.356550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:39:50.356550 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:39:50.356550 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:39:50.356550 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:39:50.356757 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:39:50.357309 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:50.357672 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:50.357971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:50.358659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:39:50.358659 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:50.358659 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:39:50.358659 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:39:50.358659 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:50.358659 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:39:50.358659 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:50.358659 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:39:50.359475 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:50.359475 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:39:50.359475 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:50.359475 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:39:50.359475 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:39:50.359475 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:39:50.359475 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:39:50.359475 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:50.359475 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:50.359475 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:50.359475 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:39:50.359475 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:50.360151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:50.360151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:50.360151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:39:50.360151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:50.360151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:39:50.360151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:39:50.360151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:50.360151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:39:50.360151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:50.360151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:39:50.360151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:39:50.360151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:50.360151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:39:50.361175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:50.361175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:39:50.361175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:50.361175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:39:50.361175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:39:50.361175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:39:50.361175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:39:50.361175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:39:50.361175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:50.361175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:39:50.361175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:39:50.361175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:39:50.361175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:39:50.361175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:39:50.361175 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:50.361774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:50.361774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:39:50.361774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:39:50.361774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:39:50.361774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:39:50.361774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:50.361774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:50.361774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:50.361774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:39:50.361774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:50.361774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:50.361774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:50.361774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:39:50.361774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:50.361774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:39:50.361774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:39:50.361774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:50.361774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:39:50.361774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:50.362237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:39:50.362237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:39:50.362237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:50.362237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:39:50.362237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:50.362237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:39:55.100888 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:39:55.101770 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:39:55.101770 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:39:55.102921 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:39:55.134768 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:55.134897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:55.135374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:39:55.135374 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:55.135374 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:39:55.135374 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:39:55.135374 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:55.135374 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:39:55.135374 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:55.135374 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:39:55.136064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:55.136064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:39:55.136064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:55.136064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:39:55.136064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:39:55.136064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:39:55.136064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:39:55.136064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:55.136064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:55.136064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:55.136064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:39:55.136064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:55.136764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:55.136764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:55.136764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:39:55.136764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:55.136764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:39:55.136764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:39:55.136764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:55.136764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:39:55.136764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:55.136764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:39:55.136764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:39:55.136764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:55.136764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:39:55.137310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:55.137310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:39:55.137310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:55.137310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:39:55.137310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:39:55.137310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:39:55.137310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:39:55.137310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:39:55.137310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:55.137310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:39:55.137310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:39:55.137310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:39:55.137310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:39:55.137310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:39:55.137310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:55.138044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:55.138044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:39:55.138044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:39:55.138044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:39:55.138044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:39:55.138044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:55.138044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:55.138044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:55.138044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:39:55.138044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:55.138044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:55.138044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:55.138044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:39:55.138044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:55.138044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:39:55.138044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:39:55.138044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:55.138044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:39:55.138044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:55.138892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:39:55.138892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:39:55.138892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:55.138892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:39:55.138892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:55.138892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:39:55.333186 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:39:55.333764 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:39:55.333958 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:39:55.334905 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:55.335198 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:55.335457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:55.336280 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:39:55.336280 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:55.336280 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:39:55.336280 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:39:55.336280 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:55.336280 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:39:55.336280 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:55.336280 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:39:55.337805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:55.337805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:39:55.337805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:55.337805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:39:55.337805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:39:55.337805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:39:55.337805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:39:55.337805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:55.337805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:55.337805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:55.337805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:39:55.337805 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:55.338350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:55.338350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:55.338350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:39:55.338350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:55.338350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:39:55.338350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:39:55.338350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:55.338350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:39:55.338350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:55.338350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:39:55.338350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:39:55.338350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:55.338350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:39:55.338935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:55.338935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:39:55.339560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:55.339560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:39:55.339560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:39:55.339560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:39:55.339560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:39:55.339560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:39:55.339560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:55.339560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:39:55.339560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:39:55.339560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:39:55.339560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:39:55.339560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:39:55.339560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:55.339560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:55.339560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:39:55.339560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:39:55.340340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:39:55.340340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:39:55.340340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:55.340340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:55.340340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:55.340340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:39:55.340340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:55.340340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:55.340340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:55.340340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:39:55.340340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:55.340340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:39:55.340340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:39:55.340340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:55.340340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:39:55.340340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:55.340340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:39:55.340340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:39:55.340340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:55.340904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:39:55.340904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:55.340904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:39:55.341949 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:39:55.342226 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:39:55.342362 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:39:55.342506 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:39:55.342738 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:55.342920 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:55.343353 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:39:55.343353 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:55.343353 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:39:55.343353 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:39:55.343353 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:55.343353 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:39:55.343353 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:55.343353 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:39:55.344081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:55.344081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:39:55.344081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:39:55.344081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:39:55.344081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:55.344081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:39:55.344081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:55.344081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:39:55.344081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:55.344081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:39:55.344081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:55.344081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:39:55.344081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:55.344081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:39:55.344081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:39:55.344545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:55.344545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:39:55.344545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:55.344545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:39:55.344545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:39:55.344545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:55.344545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:39:55.344545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:55.344545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:39:55.344943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:55.344943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:39:55.344943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:55.344943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:39:55.344943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:39:55.344943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:39:55.344943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:39:55.344943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:39:55.344943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:39:55.344943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:39:55.344943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:55.344943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:55.345331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:55.345331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:39:55.345331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:55.345331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:55.345331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:55.345331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:39:55.345331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:55.345331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:39:55.345331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:39:55.345331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:55.345331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:39:55.345331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:55.345331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:39:55.345331 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:39:55.345730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:55.345730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:39:55.345730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:55.345730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:39:55.345968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:55.345968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:39:55.345968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:39:55.345968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:39:55.345968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:39:55.345968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:39:55.345968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:55.345968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:39:55.345968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:39:55.345968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:39:55.345968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:39:55.345968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:39:55.345968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:55.345968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:55.345968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:39:55.345968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:39:55.346432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:39:55.346432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:39:55.346432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:39:55.346432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:39:55.346432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:39:55.346432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:55.346432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:55.346432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:55.346432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:39:55.346432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:55.346432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:55.346432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:55.346432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:39:55.346432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:55.346432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:39:55.346432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:39:55.346432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:55.346432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:39:55.346947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:55.346947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:39:55.346947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:39:55.346947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:55.346947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:39:55.346947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:55.346947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:39:55.351048 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:39:55.351280 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:39:55.351401 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:39:55.351560 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:39:55.351789 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:55.352008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:55.352432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:39:55.352432 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:55.352432 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:39:55.352432 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:39:55.352432 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:55.352432 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:39:55.352432 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:55.352432 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:39:55.353177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:55.353177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:39:55.353177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:39:55.353177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:39:55.353177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:55.353177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:39:55.353177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:55.353177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:39:55.353177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:55.353177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:39:55.353177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:55.353177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:39:55.353177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:55.353177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:39:55.353177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:39:55.353618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:55.353618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:39:55.353618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:55.353618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:39:55.353618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:39:55.353618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:55.353618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:39:55.353618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:55.353618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:39:55.354070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:55.354070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:39:55.354070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:55.354070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:39:55.354070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:39:55.354070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:39:55.354070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:39:55.354070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:39:55.354070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:39:55.354070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:55.354070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:55.354439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:55.354439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:39:55.354439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:55.354439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:55.354439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:55.354439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:39:55.354439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:55.354439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:39:55.354439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:39:55.354439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:55.354439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:39:55.354439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:55.354439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:39:55.354439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:39:55.354874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:55.354874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:39:55.354874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:55.354874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:39:55.355126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:55.355126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:39:55.355126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:39:55.355126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:39:55.355126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:39:55.355126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:39:55.355126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:55.355126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:39:55.355126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:39:55.355126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:39:55.355126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:39:55.355126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:39:55.355126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:55.355126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:55.355126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:39:55.355126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:39:55.355126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:39:55.355642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:39:55.355642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:39:55.355642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:39:55.355642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:55.355642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:55.355642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:55.355642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:39:55.355642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:55.355642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:55.355642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:55.355642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:39:55.355642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:55.355642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:39:55.355642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:39:55.355642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:55.355642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:39:55.355642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:55.355642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:39:55.355642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:39:55.356155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:55.356155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:39:55.356155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:55.356155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:39:55.356501 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:39:55.356769 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:39:55.356875 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:39:55.357011 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:39:55.357238 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:55.357514 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:39:55.357697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:55.358108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:39:55.358108 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:55.358108 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:39:55.358108 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:39:55.358108 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:55.358108 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:39:55.358108 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:55.358108 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:39:55.358917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:55.358917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:39:55.358917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:55.358917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:39:55.358917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:39:55.358917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:39:55.358917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:39:55.358917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:55.358917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:55.358917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:55.358917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:39:55.358917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:55.359272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:39:55.359272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:55.359272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:39:55.359272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:55.359272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:39:55.359272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:39:55.359272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:55.359272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:39:55.359272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:55.359272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:39:55.359272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:39:55.359272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:55.359272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:39:55.359638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:55.359638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:39:55.359890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:55.359890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:39:55.359890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:39:55.359890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:39:55.359890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:39:55.359890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:39:55.359890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:55.359890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:39:55.359890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:39:55.359890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:39:55.359890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:39:55.359890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:39:55.359890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:39:55.359890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:39:55.359890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:39:55.359890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:39:55.360332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:39:55.360332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:39:55.360332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:55.360332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:55.360332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:39:55.360332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:39:55.360332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:39:55.360332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:39:55.360332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:39:55.360332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:39:55.360332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:39:55.360332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:39:55.360332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:39:55.360332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:39:55.360332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:39:55.360332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:39:55.360332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:39:55.360332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:39:55.360332 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:39:55.360858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:39:55.360858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:39:55.360858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:40:00.101545 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:40:00.102217 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:40:00.102455 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:40:00.103757 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:40:00.137769 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:40:00.138250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:00.139165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:40:00.139165 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:00.139165 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:40:00.139165 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:40:00.139165 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:00.139165 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:40:00.139165 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:00.139165 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:40:00.140240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:00.140240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:40:00.140240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:00.140240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:40:00.140240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:40:00.140240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:40:00.140240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:40:00.140240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:00.140240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:00.140240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:00.140240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:40:00.140240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:00.140797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:00.140797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:00.140797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:40:00.140797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:00.140797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:40:00.140797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:40:00.140797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:00.140797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:40:00.140797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:00.140797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:40:00.140797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:40:00.140797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:00.140797 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:40:00.141501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:00.141501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:40:00.141501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:00.141501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:40:00.141501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:40:00.141501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:40:00.141501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:40:00.141501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:40:00.141501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:00.141501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:40:00.141501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:40:00.141501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:40:00.141501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:40:00.141501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:40:00.141501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:00.142260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:00.142260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:40:00.142260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:40:00.142260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:40:00.142260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:40:00.142260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:00.142260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:00.142260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:00.142260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:40:00.142260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:00.142260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:00.142260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:00.142260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:40:00.142260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:00.142260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:40:00.142260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:40:00.142260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:00.142260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:40:00.142260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:00.143128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:40:00.143128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:40:00.143128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:00.143128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:40:00.143128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:00.143128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:40:00.333991 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:40:00.334517 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:40:00.334652 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:40:00.336329 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:40:00.336888 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:40:00.337163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:00.338508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:40:00.338508 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:00.338508 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:40:00.338508 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:40:00.338508 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:00.338508 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:40:00.338508 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:00.338508 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:40:00.340267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:00.340267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:40:00.340267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:00.340267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:40:00.340267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:40:00.340267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:40:00.340267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:40:00.340267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:00.340267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:00.340267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:00.340267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:40:00.340267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:00.341027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:00.341027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:00.341027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:40:00.341027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:00.341027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:40:00.341027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:40:00.341027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:00.341027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:40:00.341027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:00.341027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:40:00.341027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:40:00.341027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:00.341027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:40:00.341751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:00.341751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:40:00.342095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:00.342095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:40:00.342095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:40:00.342095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:40:00.342095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:40:00.342095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:40:00.342095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:00.342095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:40:00.342095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:40:00.342095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:40:00.342095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:40:00.342095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:40:00.342095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:00.342095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:00.342095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:40:00.342095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:40:00.343024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:40:00.343024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:40:00.343024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:00.343024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:00.343024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:00.343024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:40:00.343024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:00.343024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:00.343024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:00.343024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:40:00.343024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:00.343024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:40:00.343024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:40:00.343024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:00.343024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:40:00.343024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:00.343024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:40:00.343024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:40:00.343024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:00.344086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:40:00.344086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:00.344086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:40:00.344086 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:40:00.344762 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:40:00.344868 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:40:00.344983 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:40:00.345689 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:40:00.346014 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:00.347002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:40:00.347002 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:00.347002 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:40:00.347002 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:40:00.347002 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:00.347002 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:40:00.347002 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:00.347002 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:40:00.347836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:00.347836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:40:00.347836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:40:00.347836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:40:00.347836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:00.347836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:40:00.347836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:00.347836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:40:00.347836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:00.347836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:40:00.347836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:00.347836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:40:00.347836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:00.347836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:40:00.347836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:40:00.348309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:00.348309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:40:00.348309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:00.348309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:40:00.348309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:40:00.348309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:00.348309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:40:00.348309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:00.348309 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:40:00.348669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:00.348669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:40:00.348669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:00.348669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:40:00.348669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:40:00.348669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:40:00.348669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:40:00.348669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:40:00.348669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:40:00.348669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:40:00.348669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:00.348669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:00.349021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:00.349021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:40:00.349021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:00.349021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:00.349021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:00.349021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:40:00.349021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:00.349021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:40:00.349021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:40:00.349021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:00.349021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:40:00.349021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:00.349021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:40:00.349021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:40:00.349361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:00.349361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:40:00.349361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:00.349361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:40:00.349649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:00.349649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:40:00.349649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:40:00.349649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:40:00.349649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:40:00.349649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:40:00.349649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:00.349649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:40:00.349649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:40:00.349649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:40:00.349649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:40:00.349649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:40:00.349649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:00.349649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:00.349649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:40:00.349649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:40:00.350079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:40:00.350079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:40:00.350079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:40:00.350079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:40:00.350079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:40:00.350079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:00.350079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:00.350079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:00.350079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:40:00.350079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:00.350079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:00.350079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:00.350079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:40:00.350079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:00.350079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:40:00.350079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:40:00.350079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:00.350079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:40:00.350573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:00.350573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:40:00.350573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:40:00.350573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:00.350573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:40:00.350573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:00.350573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:40:00.351374 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:40:00.352782 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:40:00.352883 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:40:00.353009 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:40:00.353743 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:40:00.354164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:00.358161 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:40:00.358459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:40:00.358459 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:00.358459 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:40:00.358459 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:40:00.358459 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:00.358459 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:40:00.358459 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:00.358459 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:40:00.359024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:00.359024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:40:00.359024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:40:00.359024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:40:00.359024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:00.359024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:40:00.359024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:00.359024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:40:00.359024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:00.359024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:40:00.359024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:00.359024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:40:00.359024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:00.359024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:40:00.359024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:40:00.359386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:00.359386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:40:00.359386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:00.359386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:40:00.359386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:40:00.359386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:00.359386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:40:00.359386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:00.359386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:40:00.360784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:00.360784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:40:00.360784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:00.360784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:40:00.360784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:40:00.360784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:40:00.360784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:40:00.360784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:40:00.360784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:40:00.360784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:00.360784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:00.361137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:00.361137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:40:00.361137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:00.361137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:00.361137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:00.361137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:40:00.361137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:00.361137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:40:00.361137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:40:00.361137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:00.361137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:40:00.361137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:00.361137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:40:00.361137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:40:00.361529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:00.361529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:40:00.361529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:00.361529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:40:00.361529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:00.361529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:40:00.361529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:40:00.361529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:40:00.361529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:40:00.361529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:40:00.361529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:00.361529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:40:00.361529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:40:00.361529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:40:00.361883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:40:00.361883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:40:00.361883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:00.361883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:00.361883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:40:00.361883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:40:00.361883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:40:00.361883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:40:00.361883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:40:00.361883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:40:00.361883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:00.361883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:00.361883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:00.361883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:40:00.361883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:00.361883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:00.361883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:00.361883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:40:00.361883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:00.362335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:40:00.362335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:40:00.362335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:00.362335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:40:00.362335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:00.362335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:40:00.362335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:40:00.362335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:00.362335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:40:00.362335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:00.362335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:40:00.362335 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:40:00.362335 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:40:00.362693 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:40:00.362693 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:40:00.362766 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:40:00.362948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:00.363776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:40:00.363776 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:00.363776 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:40:00.363776 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:40:00.363776 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:00.363776 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:40:00.363776 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:00.363776 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:40:00.364490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:00.364490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:40:00.364490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:00.364490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:40:00.364490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:40:00.364490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:40:00.364490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:40:00.364490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:00.364490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:00.364490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:00.364490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:40:00.364490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:00.364819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:00.364819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:00.364819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:40:00.364819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:00.364819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:40:00.364819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:40:00.364819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:00.364819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:40:00.364819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:00.364819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:40:00.364819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:40:00.364819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:00.364819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:40:00.365146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:00.365146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:40:00.365489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:00.365489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:40:00.365489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:40:00.365489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:40:00.365489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:40:00.365489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:40:00.365489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:00.365489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:40:00.365489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:40:00.365489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:40:00.365489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:40:00.365489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:40:00.365489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:00.365489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:00.365489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:40:00.365489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:40:00.365882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:40:00.365882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:40:00.365882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:00.365882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:00.365882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:00.365882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:40:00.365882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:00.365882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:00.365882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:00.365882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:40:00.365882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:00.365882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:40:00.365882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:40:00.365882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:00.365882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:40:00.365882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:00.365882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:40:00.365882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:40:00.365882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:00.366458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:40:00.366458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:00.366458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:40:05.102124 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:40:05.103081 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:40:05.103081 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:40:05.104021 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:40:05.139269 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:40:05.139706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:05.140219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:40:05.140219 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:05.140219 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:40:05.140219 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:40:05.140219 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:05.140219 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:40:05.140219 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:05.140219 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:40:05.141120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:05.141120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:40:05.141120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:05.141120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:40:05.141120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:40:05.141120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:40:05.141120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:40:05.141120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:05.141120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:05.141120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:05.141120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:40:05.141120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:05.141594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:05.141594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:05.141594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:40:05.141594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:05.141594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:40:05.141594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:40:05.141594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:05.141594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:40:05.141594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:05.141594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:40:05.141594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:40:05.141594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:05.141594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:40:05.141941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:05.141941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:40:05.141941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:05.141941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:40:05.141941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:40:05.141941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:40:05.141941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:40:05.141941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:40:05.141941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:05.141941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:40:05.141941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:40:05.141941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:40:05.141941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:40:05.141941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:40:05.141941 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:05.142356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:05.142356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:40:05.142356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:40:05.142356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:40:05.142356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:40:05.142356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:05.142356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:05.142356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:05.142356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:40:05.142356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:05.142356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:05.142356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:05.142356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:40:05.142356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:05.142356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:40:05.142356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:40:05.142356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:05.142356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:40:05.142356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:05.142876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:40:05.142876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:40:05.142876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:05.142876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:40:05.142876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:05.142876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:40:05.334675 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:40:05.335391 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:40:05.335655 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:40:05.336841 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:40:05.337175 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:40:05.337558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:05.338338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:40:05.338338 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:05.338338 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:40:05.338338 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:40:05.338338 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:05.338338 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:40:05.338338 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:05.338338 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:40:05.340195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:05.340195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:40:05.340195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:05.340195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:40:05.340195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:40:05.340195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:40:05.340195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:40:05.340195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:05.340195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:05.340195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:05.340195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:40:05.340195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:05.340689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:05.340689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:05.340689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:40:05.340689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:05.340689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:40:05.340689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:40:05.340689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:05.340689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:40:05.340689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:05.340689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:40:05.340689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:40:05.340689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:05.340689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:40:05.341115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:05.341115 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:40:05.342034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:05.342034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:40:05.342034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:40:05.342034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:40:05.342034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:40:05.342034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:40:05.342034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:05.342034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:40:05.342034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:40:05.342034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:40:05.342034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:40:05.342034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:40:05.342034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:05.342034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:05.342034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:40:05.342034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:40:05.342673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:40:05.342673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:40:05.342673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:05.342673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:05.342673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:05.342673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:40:05.342673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:05.342673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:05.342673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:05.342673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:40:05.342673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:05.342673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:40:05.342673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:40:05.342673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:05.342673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:40:05.342673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:05.342673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:40:05.342673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:40:05.342673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:05.344389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:40:05.344389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:05.344389 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:40:05.344852 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:40:05.345291 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:40:05.345599 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:40:05.345711 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:40:05.346352 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:40:05.346788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:05.347579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:40:05.347579 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:05.347579 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:40:05.347579 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:40:05.347579 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:05.347579 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:40:05.347579 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:05.347579 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:40:05.348313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:05.348313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:40:05.348313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:40:05.348313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:40:05.348313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:05.348313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:40:05.348313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:05.348313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:40:05.348313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:05.348313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:40:05.348313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:05.348313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:40:05.348313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:05.348313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:40:05.348313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:40:05.350520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:05.350520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:40:05.350520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:05.350520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:40:05.350520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:40:05.350520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:05.350520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:40:05.350520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:05.350520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:40:05.350520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:05.350520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:40:05.350520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:05.350520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:40:05.350520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:40:05.351300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:40:05.351300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:40:05.351300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:40:05.351300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:40:05.351300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:40:05.351300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:05.351300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:05.351300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:05.351300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:40:05.351300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:05.351300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:05.351300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:05.351300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:40:05.351775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:05.351775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:40:05.351775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:40:05.351775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:05.351775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:40:05.351775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:05.351775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:40:05.351775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:40:05.351775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:05.351775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:40:05.351775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:05.351775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:40:05.352063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:05.352063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:40:05.352063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:40:05.352063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:40:05.352063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:40:05.352063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:40:05.352063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:05.352063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:40:05.352063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:40:05.352063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:40:05.352063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:40:05.352063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:40:05.352063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:05.352063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:05.352063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:40:05.352063 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:40:05.352479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:40:05.352479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:40:05.352479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:40:05.352479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:40:05.352479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:40:05.352479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:05.352479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:05.352479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:05.352479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:40:05.352479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:05.352479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:05.352479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:05.352479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:40:05.352479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:05.352479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:40:05.352479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:40:05.352479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:05.352479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:40:05.352895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:05.352895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:40:05.352895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:40:05.352895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:05.352895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:40:05.352895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:05.352895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:40:05.353584 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:40:05.353846 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:40:05.353952 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:40:05.354111 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:40:05.354829 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:40:05.355170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:05.355896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:40:05.355896 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:05.355896 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:40:05.355896 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:40:05.355896 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:05.355896 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:40:05.355896 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:05.355896 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:40:05.356670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:05.356670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:40:05.356670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:40:05.356670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:40:05.356670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:05.356670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:40:05.356670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:05.356670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:40:05.356670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:05.356670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:40:05.356670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:05.356670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:40:05.356670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:05.356670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:40:05.356670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:40:05.356949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:05.356949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:40:05.356949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:05.356949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:40:05.356949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:40:05.356949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:05.356949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:40:05.356949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:05.356949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:40:05.357538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:05.357538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:40:05.357538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:05.357538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:40:05.357538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:40:05.357538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:40:05.357538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:40:05.357538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:40:05.357538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:40:05.357538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:05.357538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:05.357836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:05.357836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:40:05.357836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:05.357836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:05.357836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:05.357836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:40:05.357836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:05.357836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:40:05.357836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:40:05.357836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:05.357836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:40:05.357836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:05.357836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:40:05.357836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:40:05.358087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:05.358087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:40:05.358087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:05.358087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:40:05.358602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:05.358602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:40:05.358602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:40:05.358602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:40:05.358602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:40:05.358602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:40:05.358602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:05.358602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:40:05.358602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:40:05.358602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:40:05.358602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:40:05.358602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:40:05.358602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:05.358602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:05.358602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:40:05.358602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:40:05.358602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:40:05.359112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:40:05.359112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:40:05.359112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:40:05.359112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:05.359112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:05.359112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:05.359112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:40:05.359112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:05.359112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:05.359112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:05.359112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:40:05.359112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:05.359112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:40:05.359112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:40:05.359112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:05.359112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:40:05.359112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:05.359112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:40:05.359112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:40:05.359712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:05.359712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:40:05.359712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:05.359712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:40:05.361865 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:40:05.362123 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:40:05.362246 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:40:05.362396 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:40:05.362704 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:40:05.362916 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:40:05.363051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:05.363565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:40:05.363565 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:05.363565 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:40:05.363565 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:40:05.363565 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:05.363565 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:40:05.363565 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:05.363565 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:40:05.364372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:05.364372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:40:05.364372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:05.364372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:40:05.364372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:40:05.364372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:40:05.364372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:40:05.364372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:05.364372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:05.364372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:05.364372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:40:05.364372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:05.365020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:05.365020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:05.365020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:40:05.365020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:05.365020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:40:05.365020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:40:05.365020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:05.365020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:40:05.365020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:05.365020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:40:05.365020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:40:05.365020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:05.365020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:40:05.365361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:05.365361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:40:05.365361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:05.365361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:40:05.365361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:40:05.365361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:40:05.365361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:40:05.365361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:40:05.365361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:05.365361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:40:05.365361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:40:05.365361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:40:05.365361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:40:05.365361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:40:05.365361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:05.365883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:05.365883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:40:05.365883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:40:05.365883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:40:05.365883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:40:05.365883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:05.365883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:05.365883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:05.365883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:40:05.365883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:05.365883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:05.365883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:05.365883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:40:05.365883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:05.365883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:40:05.365883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:40:05.365883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:05.365883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:40:05.365883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:05.366524 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:40:05.366524 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:40:05.366524 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:05.366524 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:40:05.366524 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:05.366524 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:40:10.102882 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:40:10.104615 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:40:10.104780 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:40:10.105899 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:40:10.147534 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:40:10.147962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:10.148924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:40:10.148924 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:10.148924 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:40:10.148924 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:40:10.148924 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:10.148924 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:40:10.148924 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:10.148924 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:40:10.150191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:10.150191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:40:10.150191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:10.150191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:40:10.150191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:40:10.150191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:40:10.150191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:40:10.150191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:10.150191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:10.150191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:10.150191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:40:10.150191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:10.150697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:10.150697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:10.150697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:40:10.150697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:10.150697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:40:10.150697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:40:10.150697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:10.150697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:40:10.150697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:10.150697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:40:10.150697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:40:10.150697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:10.150697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:40:10.151103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:10.151103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:40:10.151103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:10.151103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:40:10.151103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:40:10.151103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:40:10.151103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:40:10.151103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:40:10.151103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:10.151103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:40:10.151103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:40:10.151103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:40:10.151103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:40:10.151103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:40:10.151103 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:10.151653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:10.151653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:40:10.151653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:40:10.151653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:40:10.151653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:40:10.151653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:10.151653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:10.151653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:10.151653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:40:10.151653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:10.151653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:10.151653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:10.151653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:40:10.151653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:10.151653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:40:10.151653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:40:10.151653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:10.151653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:40:10.151653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:10.152312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:40:10.152312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:40:10.152312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:10.152312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:40:10.152312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:10.152312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:40:10.335656 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:40:10.336692 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:40:10.337038 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:40:10.338386 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:40:10.338872 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:40:10.339269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:10.340347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:40:10.340347 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:10.340347 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:40:10.340347 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:40:10.340347 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:10.340347 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:40:10.340347 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:10.340347 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:40:10.341989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:10.341989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:40:10.341989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:10.341989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:40:10.341989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:40:10.341989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:40:10.341989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:40:10.341989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:10.341989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:10.341989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:10.341989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:40:10.341989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:10.343021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:10.343021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:10.343021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:40:10.343021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:10.343021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:40:10.343021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:40:10.343021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:10.343021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:40:10.343021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:10.343021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:40:10.343021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:40:10.343021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:10.343021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:40:10.343815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:10.343815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:40:10.343815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:10.343815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:40:10.343815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:40:10.343815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:40:10.343815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:40:10.343815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:40:10.343815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:10.343815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:40:10.343815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:40:10.343815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:40:10.343815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:40:10.343815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:40:10.343815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:10.344694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:10.344694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:40:10.344694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:40:10.344694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:40:10.344694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:40:10.344694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:10.344694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:10.344694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:10.344694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:40:10.344694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:10.344694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:10.344694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:10.344694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:40:10.344694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:10.344694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:40:10.344694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:40:10.344694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:10.344694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:40:10.344694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:10.345921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:40:10.345921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:40:10.345921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:10.345921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:40:10.345921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:10.345921 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:40:10.345921 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:40:10.345921 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:40:10.345921 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:40:10.345921 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:40:10.347050 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:40:10.347050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:10.347581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:40:10.347581 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:10.347581 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:40:10.347581 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:40:10.347581 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:10.347581 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:40:10.347581 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:10.347581 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:40:10.348354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:10.348354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:40:10.348354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:40:10.348354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:40:10.348354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:10.348354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:40:10.348354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:10.348354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:40:10.348354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:10.348354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:40:10.348354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:10.348354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:40:10.348354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:10.348354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:40:10.348354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:40:10.351436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:10.351436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:40:10.351436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:10.351436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:40:10.351436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:40:10.351436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:10.351436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:40:10.351436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:10.351436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:40:10.351436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:10.351436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:40:10.351436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:10.351436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:40:10.351436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:40:10.351846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:40:10.351846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:40:10.351846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:40:10.351846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:40:10.351846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:40:10.351846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:10.351846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:10.351846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:10.351846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:40:10.351846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:10.351846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:10.351846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:10.351846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:40:10.352217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:10.352217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:40:10.352217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:40:10.352217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:10.352217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:40:10.352217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:10.352217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:40:10.352217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:40:10.352217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:10.352217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:40:10.352217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:10.352217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:40:10.352522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:10.352522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:40:10.352522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:40:10.352522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:40:10.352522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:40:10.352522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:40:10.352522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:10.352522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:40:10.352522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:40:10.352522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:40:10.352522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:40:10.352522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:40:10.352522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:10.352522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:10.352522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:40:10.352522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:40:10.352974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:40:10.352974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:40:10.352974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:40:10.352974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:40:10.352974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:40:10.352974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:10.352974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:10.352974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:10.352974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:40:10.352974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:10.352974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:10.352974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:10.352974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:40:10.352974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:10.352974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:40:10.352974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:40:10.352974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:10.352974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:40:10.353474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:10.353474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:40:10.353474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:40:10.353474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:10.353474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:40:10.353474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:10.353474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:40:10.354877 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:40:10.355650 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:40:10.355782 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:40:10.355944 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:40:10.356662 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:40:10.356933 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:10.357619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:40:10.357619 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:10.357619 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:40:10.357619 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:40:10.357619 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:10.357619 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:40:10.357619 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:10.357619 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:40:10.358750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:10.358750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:40:10.358750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:40:10.358750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:40:10.358750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:10.358750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:40:10.358750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:10.358750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:40:10.358750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:10.358750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:40:10.358750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:10.358750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:40:10.358750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:10.358750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:40:10.358750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:40:10.359372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:10.359372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:40:10.359372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:10.359372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:40:10.359372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:40:10.359372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:10.359372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:40:10.359372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:10.359372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:40:10.359810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:10.359810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:40:10.359810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:10.359810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:40:10.359810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:40:10.359810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:40:10.359810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:40:10.359810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:40:10.359810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:40:10.359810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:10.359810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:10.360439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:10.360439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:40:10.360439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:10.360439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:10.360439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:10.360439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:40:10.360439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:10.360439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:40:10.360439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:40:10.360439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:10.360439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:40:10.360439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:10.360439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:40:10.360439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:40:10.360989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:10.360989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:40:10.360989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:10.360989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:40:10.361428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:10.361428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:40:10.361428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:40:10.361428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:40:10.361428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:40:10.361428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:40:10.361428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:10.361428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:40:10.361428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:40:10.361428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:40:10.361428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:40:10.361428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:40:10.361428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:10.361428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:10.361428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:40:10.361428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:40:10.361428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:40:10.362143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:40:10.362143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:40:10.362143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:40:10.362143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:10.362143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:10.362143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:10.362143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:40:10.362143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:10.362143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:10.362143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:10.362143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:40:10.362143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:10.362143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:40:10.362143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:40:10.362143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:10.362143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:40:10.362143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:10.362143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:40:10.362143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:40:10.362917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:10.362917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:40:10.362917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:10.362917 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:40:10.362917 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:40:10.362917 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:40:10.362917 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:40:10.362917 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:40:10.363339 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:40:10.363713 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:40:10.363929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:10.364512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:40:10.364512 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:10.364512 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:40:10.364512 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:40:10.364512 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:10.364512 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:40:10.364512 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:10.364512 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:40:10.365421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:10.365421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:40:10.365421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:10.365421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:40:10.365421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:40:10.365421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:40:10.365421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:40:10.365421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:10.365421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:10.365421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:10.365421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:40:10.365421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:10.365907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:10.365907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:10.365907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:40:10.365907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:10.365907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:40:10.365907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:40:10.365907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:10.365907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:40:10.365907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:10.365907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:40:10.365907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:40:10.365907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:10.365907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:40:10.366363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:10.366363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:40:10.366513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:10.366513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:40:10.366513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:40:10.366513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:40:10.366513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:40:10.366513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:40:10.366513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:10.366513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:40:10.366513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:40:10.366513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:40:10.366513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:40:10.366513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:40:10.366513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:10.366513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:10.366513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:40:10.366513 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:40:10.367081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:40:10.367081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:40:10.367081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:10.367081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:10.367081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:10.367081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:40:10.367081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:10.367081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:10.367081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:10.367081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:40:10.367081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:10.367081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:40:10.367081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:40:10.367081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:10.367081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:40:10.367081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:10.367081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:40:10.367081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:40:10.367081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:10.367697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:40:10.367697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:10.367697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:40:15.104654 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:40:15.105551 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:40:15.105551 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:40:15.106814 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:40:15.139929 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:40:15.140790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:15.141588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:40:15.141588 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:15.141588 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:40:15.141588 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:40:15.141588 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:15.141588 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:40:15.141588 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:15.141588 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:40:15.142707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:15.142707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:40:15.142707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:15.142707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:40:15.142707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:40:15.142707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:40:15.142707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:40:15.142707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:15.142707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:15.142707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:15.142707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:40:15.142707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:15.143366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:15.143366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:15.143366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:40:15.143366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:15.143366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:40:15.143366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:40:15.143366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:15.143366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:40:15.143366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:15.143366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:40:15.143366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:40:15.143366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:15.143366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:40:15.144026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:15.144026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:40:15.144026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:15.144026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:40:15.144026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:40:15.144026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:40:15.144026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:40:15.144026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:40:15.144026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:15.144026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:40:15.144026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:40:15.144026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:40:15.144026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:40:15.144026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:40:15.144026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:15.144888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:15.144888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:40:15.144888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:40:15.144888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:40:15.144888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:40:15.144888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:15.144888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:15.144888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:15.144888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:40:15.144888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:15.144888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:15.144888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:15.144888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:40:15.144888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:15.144888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:40:15.144888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:40:15.144888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:15.144888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:40:15.144888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:15.145823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:40:15.145823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:40:15.145823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:15.145823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:40:15.145823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:15.145823 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:40:15.336425 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:40:15.336961 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:40:15.337179 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:40:15.338228 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:40:15.338476 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:40:15.338638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:15.339312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:40:15.339312 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:15.339312 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:40:15.339312 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:40:15.339312 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:15.339312 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:40:15.339312 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:15.339312 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:40:15.340307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:15.340307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:40:15.340307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:15.340307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:40:15.340307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:40:15.340307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:40:15.340307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:40:15.340307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:15.340307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:15.340307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:15.340307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:40:15.340307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:15.340974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:15.340974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:15.340974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:40:15.340974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:15.340974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:40:15.340974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:40:15.340974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:15.340974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:40:15.340974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:15.340974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:40:15.340974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:40:15.340974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:15.340974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:40:15.341555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:15.341555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:40:15.341555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:15.341555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:40:15.341555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:40:15.341555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:40:15.341555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:40:15.341555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:40:15.341555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:15.341555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:40:15.341555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:40:15.341555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:40:15.341555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:40:15.341555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:40:15.341555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:15.342370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:15.342370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:40:15.342370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:40:15.342370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:40:15.342370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:40:15.342370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:15.342370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:15.342370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:15.342370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:40:15.342370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:15.342370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:15.342370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:15.342370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:40:15.342370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:15.342370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:40:15.342370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:40:15.342370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:15.342370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:40:15.342370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:15.343232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:40:15.343232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:40:15.343232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:15.343232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:40:15.343232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:15.343232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:40:15.345718 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:40:15.345984 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:40:15.346160 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:40:15.346378 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:40:15.346778 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:40:15.347047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:15.347909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:40:15.347909 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:15.347909 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:40:15.347909 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:40:15.347909 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:15.347909 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:40:15.347909 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:15.347909 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:40:15.349110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:15.349110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:40:15.349110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:40:15.349110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:40:15.349110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:15.349110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:40:15.349110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:15.349110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:40:15.349110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:15.349110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:40:15.349110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:15.349110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:40:15.349110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:15.349110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:40:15.349110 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:40:15.349650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:15.349650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:40:15.349650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:15.349650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:40:15.349650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:40:15.349650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:15.349650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:40:15.349650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:15.349650 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:40:15.350420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:15.350420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:40:15.350420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:15.350420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:40:15.350420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:40:15.350420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:40:15.350420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:40:15.350420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:40:15.350420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:40:15.350420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:40:15.350420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:15.350420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:15.350827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:15.350827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:40:15.350827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:15.350827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:15.350827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:15.350827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:40:15.350827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:15.350827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:40:15.350827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:40:15.350827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:15.350827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:40:15.350827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:15.350827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:40:15.350827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:40:15.351221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:15.351221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:40:15.351221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:15.351221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:40:15.352058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:15.352058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:40:15.352058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:40:15.352058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:40:15.352058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:40:15.352058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:40:15.352058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:15.352058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:40:15.352058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:40:15.352058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:40:15.352058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:40:15.352058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:40:15.352058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:15.352058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:15.352058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:40:15.352058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:40:15.352616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:40:15.352616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:40:15.352616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:40:15.352616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:40:15.352616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:40:15.352616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:15.352616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:15.352616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:15.352616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:40:15.352616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:15.352616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:15.352616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:15.352616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:40:15.352616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:15.352616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:40:15.352616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:40:15.352616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:15.352616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:40:15.353150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:15.353150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:40:15.353150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:40:15.353150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:15.353150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:40:15.353150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:15.353150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:40:15.355666 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:40:15.355961 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:40:15.356070 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:40:15.356222 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:40:15.356713 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:40:15.356953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:15.358511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:40:15.358511 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:15.358511 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:40:15.358511 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:40:15.358511 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:15.358511 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:40:15.358511 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:15.358511 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:40:15.359217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:15.359217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:40:15.359217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:40:15.359217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:40:15.359217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:15.359217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:40:15.359217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:15.359217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:40:15.359217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:15.359217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:40:15.359217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:15.359217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:40:15.359217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:15.359217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:40:15.359217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:40:15.359678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:15.359678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:40:15.359678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:15.359678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:40:15.359678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:40:15.359678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:15.359678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:40:15.359678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:15.359678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:40:15.360082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:15.360082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:40:15.360082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:15.360082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:40:15.360082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:40:15.360082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:40:15.360082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:40:15.360082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:40:15.360082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:40:15.360082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:15.360082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:15.360492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:15.360492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:40:15.360492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:15.360492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:15.360492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:15.360492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:40:15.360492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:15.360492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:40:15.360492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:40:15.360492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:15.360492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:40:15.360492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:15.360492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:40:15.360492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:40:15.360869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:15.360869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:40:15.360869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:15.360869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:40:15.361146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:15.361146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:40:15.361146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:40:15.361146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:40:15.361146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:40:15.361146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:40:15.361146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:15.361146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:40:15.361146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:40:15.361146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:40:15.361146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:40:15.361146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:40:15.361146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:15.361146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:15.361146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:40:15.361146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:40:15.361146 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:40:15.361752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:40:15.361752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:40:15.361752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:40:15.361752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:15.361752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:15.361752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:15.361752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:40:15.361752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:15.361752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:15.361752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:15.361752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:40:15.361752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:15.361752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:40:15.361752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:40:15.361752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:15.361752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:40:15.361752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:15.361752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:40:15.361752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:40:15.362328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:15.362328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:40:15.362328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:15.362328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:40:15.362673 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:40:15.362983 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:40:15.363107 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:40:15.363252 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:40:15.363632 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:40:15.363853 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:40:15.364002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:15.364535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:40:15.364535 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:15.364535 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:40:15.364535 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:40:15.364535 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:15.364535 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:40:15.364535 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:15.364535 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:40:15.365351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:15.365351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:40:15.365351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:15.365351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:40:15.365351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:40:15.365351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:40:15.365351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:40:15.365351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:15.365351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:15.365351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:15.365351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:40:15.365351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:15.365824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:15.365824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:15.365824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:40:15.365824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:15.365824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:40:15.365824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:40:15.365824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:15.365824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:40:15.365824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:15.365824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:40:15.365824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:40:15.365824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:15.365824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:40:15.366193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:15.366193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:40:15.366318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:15.366318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:40:15.366318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:40:15.366318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:40:15.366318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:40:15.366318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:40:15.366318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:15.366318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:40:15.366318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:40:15.366318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:40:15.366318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:40:15.366318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:40:15.366318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:15.366318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:15.366318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:40:15.366318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:40:15.366808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:40:15.366808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:40:15.366808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:15.366808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:15.366808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:15.366808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:40:15.366808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:15.366808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:15.366808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:15.366808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:40:15.366808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:15.366808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:40:15.366808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:40:15.366808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:15.366808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:40:15.366808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:15.366808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:40:15.366808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:40:15.366808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:15.367392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:40:15.367392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:15.367392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:40:20.105217 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:40:20.106047 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:40:20.106118 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:40:20.107471 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:40:20.141136 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:40:20.141606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:20.142469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:40:20.142469 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:20.142469 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:40:20.142469 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:40:20.142469 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:20.142469 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:40:20.142469 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:20.142469 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:40:20.143480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:20.143480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:40:20.143480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:20.143480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:40:20.143480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:40:20.143480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:40:20.143480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:40:20.143480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:20.143480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:20.143480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:20.143480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:40:20.143480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:20.143817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:20.143817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:20.143817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:40:20.143817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:20.143817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:40:20.143817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:40:20.143817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:20.143817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:40:20.143817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:20.143817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:40:20.143817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:40:20.143817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:20.143817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:40:20.144165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:20.144165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:40:20.144551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:20.144551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:40:20.144551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:40:20.144551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:40:20.144551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:40:20.144551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:40:20.144551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:20.144551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:40:20.144551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:40:20.144551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:40:20.144551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:40:20.144551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:40:20.144551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:20.144551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:20.144551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:40:20.144551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:40:20.144973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:40:20.144973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:40:20.144973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:20.144973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:20.144973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:20.144973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:40:20.144973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:20.144973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:20.144973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:20.144973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:40:20.144973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:20.144973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:40:20.144973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:40:20.144973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:20.144973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:40:20.144973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:20.144973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:40:20.144973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:40:20.144973 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:20.145528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:40:20.145528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:20.145528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:40:20.337061 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:40:20.337740 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:40:20.337933 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:40:20.339092 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:40:20.339431 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:40:20.339723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:20.340626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:40:20.340626 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:20.340626 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:40:20.340626 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:40:20.340626 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:20.340626 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:40:20.340626 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:20.340626 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:40:20.342214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:20.342214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:40:20.342214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:20.342214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:40:20.342214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:40:20.342214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:40:20.342214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:40:20.342214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:20.342214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:20.342214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:20.342214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:40:20.342214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:20.342923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:20.342923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:20.342923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:40:20.342923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:20.342923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:40:20.342923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:40:20.342923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:20.342923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:40:20.342923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:20.342923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:40:20.342923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:40:20.342923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:20.342923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:40:20.343489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:20.343489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:40:20.343489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:20.343489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:40:20.343489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:40:20.343489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:40:20.343489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:40:20.343489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:40:20.343489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:20.343489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:40:20.343489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:40:20.343489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:40:20.343489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:40:20.343489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:40:20.343489 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:20.344160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:20.344160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:40:20.344160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:40:20.344160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:40:20.344160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:40:20.344160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:20.344160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:20.344160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:20.344160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:40:20.344160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:20.344160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:20.344160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:20.344160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:40:20.344160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:20.344160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:40:20.344160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:40:20.344160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:20.344160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:40:20.344160 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:20.344983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:40:20.344983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:40:20.344983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:20.344983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:40:20.344983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:20.344983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:40:20.346035 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:40:20.346272 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:40:20.346376 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:40:20.346520 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:40:20.347247 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:40:20.347351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:20.348002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:40:20.348002 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:20.348002 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:40:20.348002 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:40:20.348002 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:20.348002 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:40:20.348002 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:20.348002 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:40:20.348807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:20.348807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:40:20.348807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:40:20.348807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:40:20.348807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:20.348807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:40:20.348807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:20.348807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:40:20.348807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:20.348807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:40:20.348807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:20.348807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:40:20.348807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:20.348807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:40:20.348807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:40:20.351775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:20.351775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:40:20.351775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:20.351775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:40:20.351775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:40:20.351775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:20.351775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:40:20.351775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:20.351775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:40:20.351775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:20.351775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:40:20.351775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:20.351775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:40:20.351775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:40:20.352172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:40:20.352172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:40:20.352172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:40:20.352172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:40:20.352172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:40:20.352172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:20.352172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:20.352172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:20.352172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:40:20.352172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:20.352172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:20.352172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:20.352172 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:40:20.352779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:20.352779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:40:20.352779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:40:20.352779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:20.352779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:40:20.352779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:20.352779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:40:20.352779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:40:20.352779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:20.352779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:40:20.352779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:20.352779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:40:20.353093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:20.353093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:40:20.353093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:40:20.353093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:40:20.353093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:40:20.353093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:40:20.353093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:20.353093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:40:20.353093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:40:20.353093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:40:20.353093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:40:20.353093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:40:20.353093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:20.353093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:20.353093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:40:20.353093 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:40:20.353573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:40:20.353573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:40:20.353573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:40:20.353573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:40:20.353573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:40:20.353573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:20.353573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:20.353573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:20.353573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:40:20.353573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:20.353573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:20.353573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:20.353573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:40:20.353573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:20.353573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:40:20.353573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:40:20.353573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:20.353573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:40:20.354271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:20.354271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:40:20.354271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:40:20.354271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:20.354271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:40:20.354271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:20.354271 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:40:20.356121 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:40:20.356343 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:40:20.356575 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:40:20.356575 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:40:20.356928 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:40:20.357153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:20.357733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:40:20.357733 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:20.357733 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:40:20.357733 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:40:20.357733 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:20.357733 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:40:20.357733 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:20.357733 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:40:20.358415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:20.358415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:40:20.358415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:40:20.358415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:40:20.358415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:20.358415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:40:20.358415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:20.358415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:40:20.358415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:20.358415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:40:20.358415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:20.358415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:40:20.358415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:20.358415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:40:20.358415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:40:20.358840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:20.358840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:40:20.358840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:20.358840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:40:20.358840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:40:20.358840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:20.358840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:40:20.358840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:20.358840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:40:20.359266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:20.359266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:40:20.359266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:20.359266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:40:20.359266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:40:20.359266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:40:20.359266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:40:20.359266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:40:20.359266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:40:20.359266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:20.359266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:20.359626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:20.359626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:40:20.359626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:20.359626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:20.359626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:20.359626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:40:20.359626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:20.359626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:40:20.359626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:40:20.359626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:20.359626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:40:20.359626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:20.359626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:40:20.359626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:40:20.360007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:20.360007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:40:20.360007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:20.360007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:40:20.360393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:20.360393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:40:20.360393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:40:20.360393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:40:20.360393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:40:20.360393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:40:20.360393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:20.360393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:40:20.360393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:40:20.360393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:40:20.360393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:40:20.360393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:40:20.360393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:20.360393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:20.360393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:40:20.360393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:40:20.360393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:40:20.360918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:40:20.360918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:40:20.360918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:40:20.360918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:20.360918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:20.360918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:20.360918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:40:20.360918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:20.360918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:20.360918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:20.360918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:40:20.360918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:20.360918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:40:20.360918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:40:20.360918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:20.360918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:40:20.360918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:20.360918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:40:20.360918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:40:20.361470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:20.361470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:40:20.361470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:20.361470 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:40:20.363076 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:40:20.363360 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:40:20.363474 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:40:20.363621 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:40:20.363972 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:40:20.364254 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:40:20.364507 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:20.365166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:40:20.365166 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:20.365166 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:40:20.365166 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:40:20.365166 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:20.365166 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:40:20.365166 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:20.365166 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:40:20.365993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:20.365993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:40:20.365993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:20.365993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:40:20.365993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:40:20.365993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:40:20.365993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:40:20.365993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:20.365993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:20.365993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:20.365993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:40:20.365993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:20.366348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:20.366348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:20.366348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:40:20.366348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:20.366348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:40:20.366348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:40:20.366348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:20.366348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:40:20.366348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:20.366348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:40:20.366348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:40:20.366348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:20.366348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:40:20.366715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:20.366715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:40:20.366936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:20.366936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:40:20.366936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:40:20.366936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:40:20.366936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:40:20.366936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:40:20.366936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:20.366936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:40:20.366936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:40:20.366936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:40:20.366936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:40:20.366936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:40:20.366936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:20.366936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:20.366936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:40:20.366936 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:40:20.367355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:40:20.367355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:40:20.367355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:20.367355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:20.367355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:20.367355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:40:20.367355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:20.367355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:20.367355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:20.367355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:40:20.367355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:20.367355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:40:20.367355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:40:20.367355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:20.367355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:40:20.367355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:20.367355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:40:20.367355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:40:20.367355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:20.367881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:40:20.367881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:20.367881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:40:25.105921 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:40:25.107108 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:40:25.107108 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:40:25.107802 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:40:25.109983 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG dbcounter [-] [105581] Writing DB stats neutron:SELECT=433 {{(pid=105581) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 14:40:25.140247 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:40:25.140247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:25.140691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:40:25.140691 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:25.140691 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:40:25.140691 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:40:25.140691 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:25.140691 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:40:25.140691 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:25.140691 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:40:25.141346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:25.141346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:40:25.141346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:25.141346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:40:25.141346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:40:25.141346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:40:25.141346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:40:25.141346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:25.141346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:25.141346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:25.141346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:40:25.141346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:25.141895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:25.141895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:25.141895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:40:25.141895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:25.141895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:40:25.141895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:40:25.141895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:25.141895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:40:25.141895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:25.141895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:40:25.141895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:40:25.141895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:25.141895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:40:25.142181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:25.142181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:40:25.142181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:25.142181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:40:25.142181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:40:25.142181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:40:25.142181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:40:25.142181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:40:25.142181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:25.142181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:40:25.142181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:40:25.142181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:40:25.142181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:40:25.142181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:40:25.142181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:25.142524 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:25.142524 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:40:25.142524 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:40:25.142524 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:40:25.142524 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:40:25.142524 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:25.142524 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:25.142524 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:25.142524 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:40:25.142524 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:25.142524 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:25.142524 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:25.142524 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:40:25.142524 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:25.142524 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:40:25.142524 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:40:25.142524 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:25.142524 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:40:25.142524 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:25.142928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:40:25.142928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:40:25.142928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:25.142928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:40:25.142928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:25.142928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:40:25.337953 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:40:25.338590 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:40:25.338696 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:40:25.339449 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:40:25.339650 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:40:25.339836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:25.340506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:40:25.340506 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:25.340506 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:40:25.340506 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:40:25.340506 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:25.340506 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:40:25.340506 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:25.340506 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:40:25.341997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:25.341997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:40:25.341997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:25.341997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:40:25.341997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:40:25.341997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:40:25.341997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:40:25.341997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:25.341997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:25.341997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:25.341997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:40:25.341997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:25.342364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:25.342364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:25.342364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:40:25.342364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:25.342364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:40:25.342364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:40:25.342364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:25.342364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:40:25.342364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:25.342364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:40:25.342364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:40:25.342364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:25.342364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:40:25.342744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:25.342744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:40:25.343569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:25.343569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:40:25.343569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:40:25.343569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:40:25.343569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:40:25.343569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:40:25.343569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:25.343569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:40:25.343569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:40:25.343569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:40:25.343569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:40:25.343569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:40:25.343569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:25.343569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:25.343569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:40:25.343569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:40:25.344189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:40:25.344189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:40:25.344189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:25.344189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:25.344189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:25.344189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:40:25.344189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:25.344189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:25.344189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:25.344189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:40:25.344189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:25.344189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:40:25.344189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:40:25.344189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:25.344189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:40:25.344189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:25.344189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:40:25.344189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:40:25.344189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:25.344729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:40:25.344729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:25.344729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:40:25.346613 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:40:25.346969 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:40:25.347066 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:40:25.347150 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:40:25.347392 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:40:25.347673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:25.348180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:40:25.348180 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:25.348180 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:40:25.348180 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:40:25.348180 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:25.348180 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:40:25.348180 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:25.348180 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:40:25.348988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:25.348988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:40:25.348988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:40:25.348988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:40:25.348988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:25.348988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:40:25.348988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:25.348988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:40:25.348988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:25.348988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:40:25.348988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:25.348988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:40:25.348988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:25.348988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:40:25.348988 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:40:25.349966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:25.349966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:40:25.349966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:25.349966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:40:25.349966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:40:25.349966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:25.349966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:40:25.349966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:25.349966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:40:25.349966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:25.349966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:40:25.349966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:25.349966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:40:25.349966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:40:25.350352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:40:25.350352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:40:25.350352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:40:25.350352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:40:25.350352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:40:25.350352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:25.350352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:25.350352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:25.350352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:40:25.350352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:25.350352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:25.350352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:25.350352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:40:25.350726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:25.350726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:40:25.350726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:40:25.350726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:25.350726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:40:25.350726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:25.350726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:40:25.350726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:40:25.350726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:25.350726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:40:25.350726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:25.350726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:40:25.351048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:25.351048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:40:25.351048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:40:25.351048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:40:25.351048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:40:25.351048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:40:25.351048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:25.351048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:40:25.351048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:40:25.351048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:40:25.351048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:40:25.351048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:40:25.351048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:25.351048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:25.351048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:40:25.351048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:40:25.351464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:40:25.351464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:40:25.351464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:40:25.351464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:40:25.351464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:40:25.351464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:25.351464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:25.351464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:25.351464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:40:25.351464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:25.351464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:25.351464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:25.351464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:40:25.351464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:25.351464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:40:25.351464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:40:25.351464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:25.351464 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:40:25.351930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:25.351930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:40:25.351930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:40:25.351930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:25.351930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:40:25.351930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:25.351930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:40:25.356830 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:40:25.357122 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:40:25.357220 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:40:25.357527 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:40:25.358005 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:40:25.358421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:25.359284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:40:25.359284 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:25.359284 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:40:25.359284 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:40:25.359284 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:25.359284 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:40:25.359284 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:25.359284 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:40:25.360456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:25.360456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:40:25.360456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:40:25.360456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:40:25.360456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:25.360456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:40:25.360456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:25.360456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:40:25.360456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:25.360456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:40:25.360456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:25.360456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:40:25.360456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:25.360456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:40:25.360456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:40:25.360851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:25.360851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:40:25.360851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:25.360851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:40:25.360851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:40:25.360851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:25.360851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:40:25.360851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:25.360851 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:40:25.361830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:25.361830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:40:25.361830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:25.361830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:40:25.361830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:40:25.361830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:40:25.361830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:40:25.361830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:40:25.361830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:40:25.361830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:25.361830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:25.362180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:25.362180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:40:25.362180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:25.362180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:25.362180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:25.362180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:40:25.362180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:25.362180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:40:25.362180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:40:25.362180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:25.362180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:40:25.362180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:25.362180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:40:25.362180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:40:25.362574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:25.362574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:40:25.362574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:25.362574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:40:25.363525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:25.363525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:40:25.363525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:40:25.363525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:40:25.363525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:40:25.363525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:40:25.363525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:25.363525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:40:25.363525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:40:25.363525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:40:25.363525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:40:25.363525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:40:25.363525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:25.363525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:25.363525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:40:25.363525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:40:25.363525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:40:25.363980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:40:25.363980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:40:25.363980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:40:25.363980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:25.363980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:25.363980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:25.363980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:40:25.363980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:25.363980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:25.363980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:25.363980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:40:25.363980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:25.363980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:40:25.363980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:40:25.363980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:25.363980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:40:25.363980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:25.363980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:40:25.363980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:40:25.364504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:25.364504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:40:25.364504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:25.364504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:40:25.364504 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:40:25.364504 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:40:25.364504 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:40:25.364720 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:40:25.365801 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:40:25.365801 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:40:25.365801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:25.366848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:40:25.366848 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:25.366848 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:40:25.366848 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:40:25.366848 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:25.366848 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:40:25.366848 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:25.366848 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:40:25.368035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:25.368035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:40:25.368035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:25.368035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:40:25.368035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:40:25.368035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:40:25.368035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:40:25.368035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:25.368035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:25.368035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:25.368035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:40:25.368035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:25.368408 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:25.368408 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:25.368408 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:40:25.368408 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:25.368408 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:40:25.368408 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:40:25.368408 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:25.368408 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:40:25.368408 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:25.368408 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:40:25.368408 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:40:25.368408 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:25.368408 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:40:25.368794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:25.368794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:40:25.369052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:25.369052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:40:25.369052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:40:25.369052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:40:25.369052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:40:25.369052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:40:25.369052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:25.369052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:40:25.369052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:40:25.369052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:40:25.369052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:40:25.369052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:40:25.369052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:25.369052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:25.369052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:40:25.369052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:40:25.369533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:40:25.369533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:40:25.369533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:25.369533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:25.369533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:25.369533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:40:25.369533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:25.369533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:25.369533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:25.369533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:40:25.369533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:25.369533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:40:25.369533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:40:25.369533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:25.369533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:40:25.369533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:25.369533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:40:25.369533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:40:25.369533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:25.370094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:40:25.370094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:25.370094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:40:30.106688 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:40:30.107157 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:40:30.107381 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:40:30.108605 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:40:30.140244 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:40:30.140521 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:30.141105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:40:30.141105 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:30.141105 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:40:30.141105 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:40:30.141105 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:30.141105 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:40:30.141105 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:30.141105 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:40:30.141874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:30.141874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:40:30.141874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:30.141874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:40:30.141874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:40:30.141874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:40:30.141874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:40:30.141874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:30.141874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:30.141874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:30.141874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:40:30.141874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:30.142219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:30.142219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:30.142219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:40:30.142219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:30.142219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:40:30.142219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:40:30.142219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:30.142219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:40:30.142219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:30.142219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:40:30.142219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:40:30.142219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:30.142219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:40:30.142597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:30.142597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:40:30.142597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:30.142597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:40:30.142597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:40:30.142597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:40:30.142597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:40:30.142597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:40:30.142597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:30.142597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:40:30.142597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:40:30.142597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:40:30.142597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:40:30.142597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:40:30.142597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:30.143053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:30.143053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:40:30.143053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:40:30.143053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:40:30.143053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:40:30.143053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:30.143053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:30.143053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:30.143053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:40:30.143053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:30.143053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:30.143053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:30.143053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:40:30.143053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:30.143053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:40:30.143053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:40:30.143053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:30.143053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:40:30.143053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:30.143605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:40:30.143605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:40:30.143605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:30.143605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:40:30.143605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:30.143605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:40:30.338726 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:40:30.339631 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:40:30.339848 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:40:30.341341 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:40:30.341784 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:40:30.342053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:30.343099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:40:30.343099 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:30.343099 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:40:30.343099 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:40:30.343099 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:30.343099 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:40:30.343099 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:30.343099 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:40:30.345064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:30.345064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:40:30.345064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:30.345064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:40:30.345064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:40:30.345064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:40:30.345064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:40:30.345064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:30.345064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:30.345064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:30.345064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:40:30.345064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:30.346210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:30.346210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:30.346210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:40:30.346210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:30.346210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:40:30.346210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:40:30.346210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:30.346210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:40:30.346210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:30.346210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:40:30.346210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:40:30.346210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:30.346210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:40:30.347657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:30.347657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:40:30.347657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:30.347657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:40:30.347657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:40:30.347657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:40:30.347657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:40:30.347657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:40:30.347657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:30.347657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:40:30.347657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:40:30.347657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:40:30.347657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:40:30.347657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:40:30.347657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:30.347974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:30.347974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:40:30.347974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:40:30.347974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:40:30.347974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:40:30.347974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:30.347974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:30.347974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:30.347974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:40:30.347974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:30.347974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:30.347974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:30.347974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:40:30.347974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:30.347974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:40:30.347974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:40:30.347974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:30.347974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:40:30.347974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:30.348326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:40:30.348326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:40:30.348326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:30.348326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:40:30.348326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:30.348326 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:40:30.348326 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:40:30.348647 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:40:30.348845 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:40:30.349076 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:40:30.349529 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:40:30.349712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:30.350342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:40:30.350342 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:30.350342 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:40:30.350342 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:40:30.350342 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:30.350342 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:40:30.350342 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:30.350342 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:40:30.351815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:30.351815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:40:30.351815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:40:30.351815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:40:30.351815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:30.351815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:40:30.351815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:30.351815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:40:30.351815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:30.351815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:40:30.351815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:30.351815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:40:30.351815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:30.351815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:40:30.351815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:40:30.352194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:30.352194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:40:30.352194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:30.352194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:40:30.352194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:40:30.352194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:30.352194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:40:30.352194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:30.352194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:40:30.352814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:30.352814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:40:30.352814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:30.352814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:40:30.352814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:40:30.352814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:40:30.352814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:40:30.352814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:40:30.352814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:40:30.352814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:40:30.352814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:30.352814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:30.353081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:30.353081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:40:30.353081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:30.353081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:30.353081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:30.353081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:40:30.353081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:30.353081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:40:30.353081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:40:30.353081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:30.353081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:40:30.353081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:30.353081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:40:30.353081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:40:30.353350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:30.353350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:40:30.353350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:30.353350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:40:30.354667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:30.354667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:40:30.354667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:40:30.354667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:40:30.354667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:40:30.354667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:40:30.354667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:30.354667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:40:30.354667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:40:30.354667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:40:30.354667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:40:30.354667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:40:30.354667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:30.354667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:30.354667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:40:30.354667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:40:30.355033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:40:30.355033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:40:30.355033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:40:30.355033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:40:30.355033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:40:30.355033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:30.355033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:30.355033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:30.355033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:40:30.355033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:30.355033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:30.355033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:30.355033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:40:30.355033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:30.355033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:40:30.355033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:40:30.355033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:30.355033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:40:30.355396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:30.355396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:40:30.355396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:40:30.355396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:30.355396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:40:30.355396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:30.355396 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:40:30.357379 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:40:30.357718 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:40:30.357803 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:40:30.357905 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:40:30.358268 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:40:30.358733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:30.359286 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:40:30.359286 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:30.359286 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:40:30.359286 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:40:30.359286 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:30.359286 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:40:30.359286 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:30.359286 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:40:30.359983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:30.359983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:40:30.359983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:40:30.359983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:40:30.359983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:30.359983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:40:30.359983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:30.359983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:40:30.359983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:30.359983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:40:30.359983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:30.359983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:40:30.359983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:30.359983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:40:30.359983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:40:30.360395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:30.360395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:40:30.360395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:30.360395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:40:30.360395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:40:30.360395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:30.360395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:40:30.360395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:30.360395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:40:30.361023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:30.361023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:40:30.361023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:30.361023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:40:30.361023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:40:30.361023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:40:30.361023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:40:30.361023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:40:30.361023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:40:30.361023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:30.361023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:30.361386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:30.361386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:40:30.361386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:30.361386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:30.361386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:30.361386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:40:30.361386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:30.361386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:40:30.361386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:40:30.361386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:30.361386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:40:30.361386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:30.361386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:40:30.361386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:40:30.361764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:30.361764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:40:30.361764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:30.361764 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:40:30.362306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:30.362306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:40:30.362306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:40:30.362306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:40:30.362306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:40:30.362306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:40:30.362306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:30.362306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:40:30.362306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:40:30.362306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:40:30.362306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:40:30.362306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:40:30.362306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:30.362306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:30.362306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:40:30.362306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:40:30.362306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:40:30.362882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:40:30.362882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:40:30.362882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:40:30.362882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:30.362882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:30.362882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:30.362882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:40:30.362882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:30.362882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:30.362882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:30.362882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:40:30.362882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:30.362882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:40:30.362882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:40:30.362882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:30.362882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:40:30.362882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:30.362882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:40:30.362882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:40:30.363479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:30.363479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:40:30.363479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:30.363479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:40:30.364322 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:40:30.364707 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:40:30.364824 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:40:30.364983 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:40:30.365677 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:40:30.365920 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:40:30.366078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:30.366776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:40:30.366776 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:30.366776 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:40:30.366776 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:40:30.366776 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:30.366776 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:40:30.366776 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:30.366776 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:40:30.367872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:30.367872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:40:30.367872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:30.367872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:40:30.367872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:40:30.367872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:40:30.367872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:40:30.367872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:30.367872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:30.367872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:30.367872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:40:30.367872 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:30.368247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:30.368247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:30.368247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:40:30.368247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:30.368247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:40:30.368247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:40:30.368247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:30.368247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:40:30.368247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:30.368247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:40:30.368247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:40:30.368247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:30.368247 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:40:30.368592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:30.368592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:40:30.369228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:30.369228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:40:30.369228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:40:30.369228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:40:30.369228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:40:30.369228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:40:30.369228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:30.369228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:40:30.369228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:40:30.369228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:40:30.369228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:40:30.369228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:40:30.369228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:30.369228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:30.369228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:40:30.369228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:40:30.369728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:40:30.369728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:40:30.369728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:30.369728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:30.369728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:30.369728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:40:30.369728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:30.369728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:30.369728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:30.369728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:40:30.369728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:30.369728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:40:30.369728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:40:30.369728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:30.369728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:40:30.369728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:30.369728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:40:30.369728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:40:30.369728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:30.370237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:40:30.370237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:30.370237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:40:35.107220 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:40:35.108307 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:40:35.108307 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:40:35.108656 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:40:35.145324 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:40:35.145762 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:35.146746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:40:35.146746 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:35.146746 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:40:35.146746 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:40:35.146746 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:35.146746 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:40:35.146746 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:35.146746 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:40:35.147810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:35.147810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:40:35.147810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:35.147810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:40:35.147810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:40:35.147810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:40:35.147810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:40:35.147810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:35.147810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:35.147810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:35.147810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:40:35.147810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:35.148246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:35.148246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:35.148246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:40:35.148246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:35.148246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:40:35.148246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:40:35.148246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:35.148246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:40:35.148246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:35.148246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:40:35.148246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:40:35.148246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:35.148246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:40:35.148735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:35.148735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:40:35.148940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:35.148940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:40:35.148940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:40:35.148940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:40:35.148940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:40:35.148940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:40:35.148940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:35.148940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:40:35.148940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:40:35.148940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:40:35.148940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:40:35.148940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:40:35.148940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:35.148940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:35.148940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:40:35.148940 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:40:35.149562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:40:35.149562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:40:35.149562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:35.149562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:35.149562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:35.149562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:40:35.149562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:35.149562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:35.149562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:35.149562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:40:35.149562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:35.149562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:40:35.149562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:40:35.149562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:35.149562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:40:35.149562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:35.149562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:40:35.149562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:40:35.149562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:35.150258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:40:35.150258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:35.150258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:40:35.339472 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:40:35.340019 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:40:35.340113 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:40:35.341009 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:40:35.341243 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:40:35.341385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:35.342020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:40:35.342020 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:35.342020 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:40:35.342020 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:40:35.342020 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:35.342020 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:40:35.342020 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:35.342020 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:40:35.343047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:35.343047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:40:35.343047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:35.343047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:40:35.343047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:40:35.343047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:40:35.343047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:40:35.343047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:35.343047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:35.343047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:35.343047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:40:35.343047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:35.343644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:35.343644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:35.343644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:40:35.343644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:35.343644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:40:35.343644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:40:35.343644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:35.343644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:40:35.343644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:35.343644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:40:35.343644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:40:35.343644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:35.343644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:40:35.344915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:35.344915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:40:35.344915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:35.344915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:40:35.344915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:40:35.344915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:40:35.344915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:40:35.344915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:40:35.344915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:35.344915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:40:35.344915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:40:35.344915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:40:35.344915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:40:35.344915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:40:35.344915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:35.345724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:35.345724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:40:35.345724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:40:35.345724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:40:35.345724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:40:35.345724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:35.345724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:35.345724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:35.345724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:40:35.345724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:35.345724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:35.345724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:35.345724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:40:35.345724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:35.345724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:40:35.345724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:40:35.345724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:35.345724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:40:35.345724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:35.346392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:40:35.346392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:40:35.346392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:35.346392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:40:35.346392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:35.346392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:40:35.348632 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:40:35.348966 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:40:35.349087 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:40:35.349213 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:40:35.349582 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:40:35.349771 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:35.350255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:40:35.350255 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:35.350255 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:40:35.350255 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:40:35.350255 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:35.350255 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:40:35.350255 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:35.350255 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:40:35.351060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:35.351060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:40:35.351060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:40:35.351060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:40:35.351060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:35.351060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:40:35.351060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:35.351060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:40:35.351060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:35.351060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:40:35.351060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:35.351060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:40:35.351060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:35.351060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:40:35.351060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:40:35.351618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:35.351618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:40:35.351618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:35.351618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:40:35.351618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:40:35.351618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:35.351618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:40:35.351618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:35.351618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:40:35.351931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:35.351931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:40:35.351931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:35.351931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:40:35.351931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:40:35.351931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:40:35.351931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:40:35.351931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:40:35.351931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:40:35.351931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:40:35.351931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:35.351931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:35.352306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:35.352306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:40:35.352306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:35.352306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:35.352306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:35.352306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:40:35.352306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:35.352306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:40:35.352306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:40:35.352306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:35.352306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:40:35.352306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:35.352306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:40:35.352306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:40:35.352697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:35.352697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:40:35.352697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:35.352697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:40:35.353058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:35.353058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:40:35.353058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:40:35.353058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:40:35.353058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:40:35.353058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:40:35.353058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:35.353058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:40:35.353058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:40:35.353058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:40:35.353058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:40:35.353058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:40:35.353058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:35.353058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:35.353058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:40:35.353058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:40:35.353568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:40:35.353568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:40:35.353568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:40:35.353568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:40:35.353568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:40:35.353568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:35.353568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:35.353568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:35.353568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:40:35.353568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:35.353568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:35.353568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:35.353568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:40:35.353568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:35.353568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:40:35.353568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:40:35.353568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:35.353568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:40:35.354189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:35.354189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:40:35.354189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:40:35.354189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:35.354189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:40:35.354189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:35.354189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:40:35.357868 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:40:35.358152 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:40:35.358257 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:40:35.358389 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:40:35.358719 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:40:35.358945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:35.359416 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:40:35.359416 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:35.359416 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:40:35.359416 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:40:35.359416 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:35.359416 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:40:35.359416 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:35.359416 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:40:35.360142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:35.360142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:40:35.360142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:40:35.360142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:40:35.360142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:35.360142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:40:35.360142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:35.360142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:40:35.360142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:35.360142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:40:35.360142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:35.360142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:40:35.360142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:35.360142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:40:35.360142 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:40:35.360564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:35.360564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:40:35.360564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:35.360564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:40:35.360564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:40:35.360564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:35.360564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:40:35.360564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:35.360564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:40:35.361124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:35.361124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:40:35.361124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:35.361124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:40:35.361124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:40:35.361124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:40:35.361124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:40:35.361124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:40:35.361124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:40:35.361124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:35.361124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:35.362032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:35.362032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:40:35.362032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:35.362032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:35.362032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:35.362032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:40:35.362032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:35.362032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:40:35.362032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:40:35.362032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:35.362032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:40:35.362032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:35.362032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:40:35.362032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:40:35.362549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:35.362549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:40:35.362549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:35.362549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:40:35.362549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:35.362549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:40:35.362549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:40:35.362549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:40:35.362549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:40:35.362549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:40:35.362549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:35.362549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:40:35.362549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:40:35.362549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:40:35.363023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:40:35.363023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:40:35.363023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:35.363023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:35.363023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:40:35.363023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:40:35.363023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:40:35.363023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:40:35.363023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:40:35.363023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:40:35.363023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:35.363023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:35.363023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:35.363023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:40:35.363023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:35.363023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:35.363023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:35.363023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:40:35.363023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:35.363535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:40:35.363535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:40:35.363535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:35.363535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:40:35.363535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:35.363535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:40:35.363535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:40:35.363535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:35.363535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:40:35.363535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:35.363535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:40:35.364792 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:40:35.365098 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:40:35.365217 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:40:35.365376 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:40:35.365794 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:40:35.366119 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:40:35.366336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:35.367008 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:40:35.367008 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:35.367008 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:40:35.367008 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:40:35.367008 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:35.367008 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:40:35.367008 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:35.367008 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:40:35.367865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:35.367865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:40:35.367865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:35.367865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:40:35.367865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:40:35.367865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:40:35.367865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:40:35.367865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:35.367865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:35.367865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:35.367865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:40:35.367865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:35.368662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:35.368662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:35.368662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:40:35.368662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:35.368662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:40:35.368662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:40:35.368662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:35.368662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:40:35.368662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:35.368662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:40:35.368662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:40:35.368662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:35.368662 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:40:35.369460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:35.369460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:40:35.369460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:35.369460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:40:35.369460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:40:35.369460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:40:35.369460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:40:35.369460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:40:35.369460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:35.369460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:40:35.369460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:40:35.369460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:40:35.369460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:40:35.369460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:40:35.369460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:35.370003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:35.370003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:40:35.370003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:40:35.370003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:40:35.370003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:40:35.370003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:35.370003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:35.370003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:35.370003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:40:35.370003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:35.370003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:35.370003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:35.370003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:40:35.370003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:35.370003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:40:35.370003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:40:35.370003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:35.370003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:40:35.370003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:35.370512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:40:35.370512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:40:35.370512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:35.370512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:40:35.370512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:35.370512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:40:40.108132 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:40:40.108935 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:40:40.109007 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:40:40.110358 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:40:40.142515 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:40:40.142908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:40.143843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:40:40.143843 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:40.143843 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:40:40.143843 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:40:40.143843 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:40.143843 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:40:40.143843 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:40.143843 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:40:40.144876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:40.144876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:40:40.144876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:40.144876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:40:40.144876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:40:40.144876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:40:40.144876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:40:40.144876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:40.144876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:40.144876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:40.144876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:40:40.144876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:40.145511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:40.145511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:40.145511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:40:40.145511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:40.145511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:40:40.145511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:40:40.145511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:40.145511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:40:40.145511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:40.145511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:40:40.145511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:40:40.145511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:40.145511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:40:40.146065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:40.146065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:40:40.146065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:40.146065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:40:40.146065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:40:40.146065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:40:40.146065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:40:40.146065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:40:40.146065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:40.146065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:40:40.146065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:40:40.146065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:40:40.146065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:40:40.146065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:40:40.146065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:40.146790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:40.146790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:40:40.146790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:40:40.146790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:40:40.146790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:40:40.146790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:40.146790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:40.146790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:40.146790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:40:40.146790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:40.146790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:40.146790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:40.146790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:40:40.146790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:40.146790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:40:40.146790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:40:40.146790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:40.146790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:40:40.146790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:40.147652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:40:40.147652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:40:40.147652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:40.147652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:40:40.147652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:40.147652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:40:40.340676 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:40:40.341076 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:40:40.341721 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:40:40.342611 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:40:40.342959 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:40:40.343225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:40.344191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:40:40.344191 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:40.344191 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:40:40.344191 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:40:40.344191 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:40.344191 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:40:40.344191 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:40.344191 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:40:40.345304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:40.345304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:40:40.345304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:40.345304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:40:40.345304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:40:40.345304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:40:40.345304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:40:40.345304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:40.345304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:40.345304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:40.345304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:40:40.345304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:40.346052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:40.346052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:40.346052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:40:40.346052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:40.346052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:40:40.346052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:40:40.346052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:40.346052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:40:40.346052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:40.346052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:40:40.346052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:40:40.346052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:40.346052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:40:40.346680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:40.346680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:40:40.346799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:40.346799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:40:40.346799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:40:40.346799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:40:40.346799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:40:40.346799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:40:40.346799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:40.346799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:40:40.346799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:40:40.346799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:40:40.346799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:40:40.346799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:40:40.346799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:40.346799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:40.346799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:40:40.346799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:40:40.347567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:40:40.347567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:40:40.347567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:40.347567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:40.347567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:40.347567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:40:40.347567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:40.347567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:40.347567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:40.347567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:40:40.347567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:40.347567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:40:40.347567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:40:40.347567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:40.347567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:40:40.347567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:40.347567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:40:40.347567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:40:40.347567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:40.348498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:40:40.348498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:40.348498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:40:40.349125 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:40:40.349590 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:40:40.349775 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:40:40.349955 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:40:40.350497 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:40:40.350898 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:40.351634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:40:40.351634 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:40.351634 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:40:40.351634 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:40:40.351634 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:40.351634 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:40:40.351634 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:40.351634 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:40:40.352554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:40.352554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:40:40.352554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:40:40.352554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:40:40.352554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:40.352554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:40:40.352554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:40.352554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:40:40.352554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:40.352554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:40:40.352554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:40.352554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:40:40.352554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:40.352554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:40:40.352554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:40:40.352995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:40.352995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:40:40.352995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:40.352995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:40:40.352995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:40:40.352995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:40.352995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:40:40.352995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:40.352995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:40:40.353543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:40.353543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:40:40.353543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:40.353543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:40:40.353543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:40:40.353543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:40:40.353543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:40:40.353543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:40:40.353543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:40:40.353543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:40:40.353543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:40.353543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:40.353953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:40.353953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:40:40.353953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:40.353953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:40.353953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:40.353953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:40:40.353953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:40.353953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:40:40.353953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:40:40.353953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:40.353953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:40:40.353953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:40.353953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:40:40.353953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:40:40.354350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:40.354350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:40:40.354350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:40.354350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:40:40.354651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:40.354651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:40:40.354651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:40:40.354651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:40:40.354651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:40:40.354651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:40:40.354651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:40.354651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:40:40.354651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:40:40.354651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:40:40.354651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:40:40.354651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:40:40.354651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:40.354651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:40.354651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:40:40.354651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:40:40.355117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:40:40.355117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:40:40.355117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:40:40.355117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:40:40.355117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:40:40.355117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:40.355117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:40.355117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:40.355117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:40:40.355117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:40.355117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:40.355117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:40.355117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:40:40.355117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:40.355117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:40:40.355117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:40:40.355117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:40.355117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:40:40.355633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:40.355633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:40:40.355633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:40:40.355633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:40.355633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:40:40.355633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:40.355633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:40:40.358427 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:40:40.358646 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:40:40.358761 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:40:40.358888 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:40:40.359346 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:40:40.359774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:40.360501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:40:40.360501 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:40.360501 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:40:40.360501 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:40:40.360501 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:40.360501 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:40:40.360501 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:40.360501 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:40:40.361215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:40.361215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:40:40.361215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:40:40.361215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:40:40.361215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:40.361215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:40:40.361215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:40.361215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:40:40.361215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:40.361215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:40:40.361215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:40.361215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:40:40.361215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:40.361215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:40:40.361215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:40:40.361647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:40.361647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:40:40.361647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:40.361647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:40:40.361647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:40:40.361647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:40.361647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:40:40.361647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:40.361647 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:40:40.362088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:40.362088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:40:40.362088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:40.362088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:40:40.362088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:40:40.362088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:40:40.362088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:40:40.362088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:40:40.362088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:40:40.362088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:40.362088 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:40.362399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:40.362399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:40:40.362399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:40.362399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:40.362399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:40.362399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:40:40.362399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:40.362399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:40:40.362399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:40:40.362399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:40.362399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:40:40.362399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:40.362399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:40:40.362399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:40:40.362782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:40.362782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:40:40.362782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:40.362782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:40:40.363127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:40.363127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:40:40.363127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:40:40.363127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:40:40.363127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:40:40.363127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:40:40.363127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:40.363127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:40:40.363127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:40:40.363127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:40:40.363127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:40:40.363127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:40:40.363127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:40.363127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:40.363127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:40:40.363127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:40:40.363127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:40:40.363616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:40:40.363616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:40:40.363616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:40:40.363616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:40.363616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:40.363616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:40.363616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:40:40.363616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:40.363616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:40.363616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:40.363616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:40:40.363616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:40.363616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:40:40.363616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:40:40.363616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:40.363616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:40:40.363616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:40.363616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:40:40.363616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:40:40.364105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:40.364105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:40:40.364105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:40.364105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:40:40.365340 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:40:40.365658 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:40:40.365766 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:40:40.365913 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:40:40.366217 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:40:40.366465 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:40:40.366631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:40.367117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:40:40.367117 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:40.367117 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:40:40.367117 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:40:40.367117 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:40.367117 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:40:40.367117 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:40.367117 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:40:40.367943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:40.367943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:40:40.367943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:40.367943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:40:40.367943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:40:40.367943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:40:40.367943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:40:40.367943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:40.367943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:40.367943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:40.367943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:40:40.367943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:40.368321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:40.368321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:40.368321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:40:40.368321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:40.368321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:40:40.368321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:40:40.368321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:40.368321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:40:40.368321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:40.368321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:40:40.368321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:40:40.368321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:40.368321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:40:40.368774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:40.368774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:40:40.368927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:40.368927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:40:40.368927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:40:40.368927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:40:40.368927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:40:40.368927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:40:40.368927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:40.368927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:40:40.368927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:40:40.368927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:40:40.368927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:40:40.368927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:40:40.368927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:40.368927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:40.368927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:40:40.368927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:40:40.369375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:40:40.369375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:40:40.369375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:40.369375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:40.369375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:40.369375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:40:40.369375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:40.369375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:40.369375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:40.369375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:40:40.369375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:40.369375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:40:40.369375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:40:40.369375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:40.369375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:40:40.369375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:40.369375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:40:40.369375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:40:40.369375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:40.369938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:40:40.369938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:40.369938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:40:45.108810 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:40:45.109487 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:40:45.109665 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:40:45.110891 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:40:45.145550 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:40:45.145837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:45.146539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:40:45.146539 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:45.146539 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:40:45.146539 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:40:45.146539 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:45.146539 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:40:45.146539 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:45.146539 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:40:45.147534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:45.147534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:40:45.147534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:45.147534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:40:45.147534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:40:45.147534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:40:45.147534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:40:45.147534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:45.147534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:45.147534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:45.147534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:40:45.147534 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:45.148224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:45.148224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:45.148224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:40:45.148224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:45.148224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:40:45.148224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:40:45.148224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:45.148224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:40:45.148224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:45.148224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:40:45.148224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:40:45.148224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:45.148224 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:40:45.148891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:45.148891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:40:45.148891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:45.148891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:40:45.148891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:40:45.148891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:40:45.148891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:40:45.148891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:40:45.148891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:45.148891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:40:45.148891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:40:45.148891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:40:45.148891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:40:45.148891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:40:45.148891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:45.149680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:45.149680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:40:45.149680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:40:45.149680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:40:45.149680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:40:45.149680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:45.149680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:45.149680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:45.149680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:40:45.149680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:45.149680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:45.149680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:45.149680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:40:45.149680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:45.149680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:40:45.149680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:40:45.149680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:45.149680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:40:45.149680 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:45.150566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:40:45.150566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:40:45.150566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:45.150566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:40:45.150566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:45.150566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:40:45.340882 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:40:45.341293 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:40:45.341526 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:40:45.342501 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:40:45.342741 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:40:45.342910 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:45.343589 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:40:45.343589 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:45.343589 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:40:45.343589 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:40:45.343589 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:45.343589 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:40:45.343589 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:45.343589 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:40:45.344990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:45.344990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:40:45.344990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:45.344990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:40:45.344990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:40:45.344990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:40:45.344990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:40:45.344990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:45.344990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:45.344990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:45.344990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:40:45.344990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:45.345635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:45.345635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:45.345635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:40:45.345635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:45.345635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:40:45.345635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:40:45.345635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:45.345635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:40:45.345635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:45.345635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:40:45.345635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:40:45.345635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:45.345635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:40:45.346197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:45.346197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:40:45.346673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:45.346673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:40:45.346673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:40:45.346673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:40:45.346673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:40:45.346673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:40:45.346673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:45.346673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:40:45.346673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:40:45.346673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:40:45.346673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:40:45.346673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:40:45.346673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:45.346673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:45.346673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:40:45.346673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:40:45.347448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:40:45.347448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:40:45.347448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:45.347448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:45.347448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:45.347448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:40:45.347448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:45.347448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:45.347448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:45.347448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:40:45.347448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:45.347448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:40:45.347448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:40:45.347448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:45.347448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:40:45.347448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:45.347448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:40:45.347448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:40:45.347448 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:45.348257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:40:45.348257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:45.348257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:40:45.349635 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:40:45.349954 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:40:45.350055 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:40:45.350180 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:40:45.350536 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:40:45.350730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:45.351313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:40:45.351313 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:45.351313 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:40:45.351313 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:40:45.351313 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:45.351313 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:40:45.351313 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:45.351313 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:40:45.352070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:45.352070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:40:45.352070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:40:45.352070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:40:45.352070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:45.352070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:40:45.352070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:45.352070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:40:45.352070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:45.352070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:40:45.352070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:45.352070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:40:45.352070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:45.352070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:40:45.352070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:40:45.352834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:45.352834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:40:45.352834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:45.352834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:40:45.352834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:40:45.352834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:45.352834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:40:45.352834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:45.352834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:40:45.352834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:45.352834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:40:45.352834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:45.352834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:40:45.352834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:40:45.353552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:40:45.353552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:40:45.353552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:40:45.353552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:40:45.353552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:40:45.353552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:45.353552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:45.353552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:45.353552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:40:45.353552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:45.353552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:45.353552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:45.353552 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:40:45.354386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:45.354386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:40:45.354386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:40:45.354386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:45.354386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:40:45.354386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:45.354386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:40:45.354386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:40:45.354386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:45.354386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:40:45.354386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:45.354386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:40:45.354918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:45.354918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:40:45.354918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:40:45.354918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:40:45.354918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:40:45.354918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:40:45.354918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:45.354918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:40:45.354918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:40:45.354918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:40:45.354918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:40:45.354918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:40:45.354918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:45.354918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:45.354918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:40:45.354918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:40:45.355339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:40:45.355339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:40:45.355339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:40:45.355339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:40:45.355339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:40:45.355339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:45.355339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:45.355339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:45.355339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:40:45.355339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:45.355339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:45.355339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:45.355339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:40:45.355339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:45.355339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:40:45.355339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:40:45.355339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:45.355339 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:40:45.355900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:45.355900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:40:45.355900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:40:45.355900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:45.355900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:40:45.355900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:45.355900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:40:45.358864 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:40:45.359134 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:40:45.359247 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:40:45.359366 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:40:45.359670 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:40:45.359887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:45.360344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:40:45.360344 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:45.360344 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:40:45.360344 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:40:45.360344 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:45.360344 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:40:45.360344 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:45.360344 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:40:45.361046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:45.361046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:40:45.361046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:40:45.361046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:40:45.361046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:45.361046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:40:45.361046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:45.361046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:40:45.361046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:45.361046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:40:45.361046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:45.361046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:40:45.361046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:45.361046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:40:45.361046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:40:45.361454 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:45.361454 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:40:45.361454 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:45.361454 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:40:45.361454 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:40:45.361454 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:45.361454 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:40:45.361454 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:45.361454 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:40:45.361893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:45.361893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:40:45.361893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:45.361893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:40:45.361893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:40:45.361893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:40:45.361893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:40:45.361893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:40:45.361893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:40:45.361893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:45.361893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:45.362257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:45.362257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:40:45.362257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:45.362257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:45.362257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:45.362257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:40:45.362257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:45.362257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:40:45.362257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:40:45.362257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:45.362257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:40:45.362257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:45.362257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:40:45.362257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:40:45.362636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:45.362636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:40:45.362636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:45.362636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:40:45.363002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:45.363002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:40:45.363002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:40:45.363002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:40:45.363002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:40:45.363002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:40:45.363002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:45.363002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:40:45.363002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:40:45.363002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:40:45.363002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:40:45.363002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:40:45.363002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:45.363002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:45.363002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:40:45.363002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:40:45.363002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:40:45.363490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:40:45.363490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:40:45.363490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:40:45.363490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:45.363490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:45.363490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:45.363490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:40:45.363490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:45.363490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:45.363490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:45.363490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:40:45.363490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:45.363490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:40:45.363490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:40:45.363490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:45.363490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:40:45.363490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:45.363490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:40:45.363490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:40:45.363970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:45.363970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:40:45.363970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:45.363970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:40:45.365785 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:40:45.366085 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:40:45.366206 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:40:45.366349 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:40:45.366647 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:40:45.366860 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:40:45.366995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:45.367548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:40:45.367548 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:45.367548 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:40:45.367548 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:40:45.367548 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:45.367548 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:40:45.367548 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:45.367548 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:40:45.368989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:45.368989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:40:45.368989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:45.368989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:40:45.368989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:40:45.368989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:40:45.368989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:40:45.368989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:45.368989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:45.368989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:45.368989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:40:45.368989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:45.369325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:45.369325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:45.369325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:40:45.369325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:45.369325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:40:45.369325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:40:45.369325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:45.369325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:40:45.369325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:45.369325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:40:45.369325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:40:45.369325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:45.369325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:40:45.369664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:45.369664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:40:45.369934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:45.369934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:40:45.369934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:40:45.369934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:40:45.369934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:40:45.369934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:40:45.369934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:45.369934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:40:45.369934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:40:45.369934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:40:45.369934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:40:45.369934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:40:45.369934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:45.369934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:45.369934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:40:45.369934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:40:45.370421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:40:45.370421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:40:45.370421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:45.370421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:45.370421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:45.370421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:40:45.370421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:45.370421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:45.370421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:45.370421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:40:45.370421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:45.370421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:40:45.370421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:40:45.370421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:45.370421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:40:45.370421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:45.370421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:40:45.370421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:40:45.370421 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:45.370916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:40:45.370916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:45.370916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:40:50.109752 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:40:50.110590 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:40:50.110847 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:40:50.112294 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:40:50.140804 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:40:50.141344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:50.142738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:40:50.142738 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:50.142738 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:40:50.142738 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:40:50.142738 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:50.142738 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:40:50.142738 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:50.142738 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:40:50.143503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:50.143503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:40:50.143503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:50.143503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:40:50.143503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:40:50.143503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:40:50.143503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:40:50.143503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:50.143503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:50.143503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:50.143503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:40:50.143503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:50.144109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:50.144109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:50.144109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:40:50.144109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:50.144109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:40:50.144109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:40:50.144109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:50.144109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:40:50.144109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:50.144109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:40:50.144109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:40:50.144109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:50.144109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:40:50.145293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:50.145293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:40:50.145293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:50.145293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:40:50.145293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:40:50.145293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:40:50.145293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:40:50.145293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:40:50.145293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:50.145293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:40:50.145293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:40:50.145293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:40:50.145293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:40:50.145293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:40:50.145293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:50.146821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:50.146821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:40:50.146821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:40:50.146821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:40:50.146821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:40:50.146821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:50.146821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:50.146821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:50.146821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:40:50.146821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:50.146821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:50.146821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:50.146821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:40:50.146821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:50.146821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:40:50.146821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:40:50.146821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:50.146821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:40:50.146821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:50.147935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:40:50.147935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:40:50.147935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:50.147935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:40:50.147935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:50.147935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:40:50.341961 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:40:50.342781 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:40:50.342968 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:40:50.344231 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:40:50.344657 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:40:50.344948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:50.345903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:40:50.345903 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:50.345903 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:40:50.345903 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:40:50.345903 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:50.345903 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:40:50.345903 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:50.345903 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:40:50.347663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:50.347663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:40:50.347663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:50.347663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:40:50.347663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:40:50.347663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:40:50.347663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:40:50.347663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:50.347663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:50.347663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:50.347663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:40:50.347663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:50.349370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:50.349370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:50.349370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:40:50.349370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:50.349370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:40:50.349370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:40:50.349370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:50.349370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:40:50.349370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:50.349370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:40:50.349370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:40:50.349370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:50.349370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:40:50.350007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:50.350007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:40:50.350007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:50.350007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:40:50.350007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:40:50.350007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:40:50.350007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:40:50.350007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:40:50.350007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:50.350007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:40:50.350007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:40:50.350007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:40:50.350007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:40:50.350007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:40:50.350007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:50.351354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:50.351354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:40:50.351354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:40:50.351354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:40:50.351354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:40:50.351354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:50.351354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:50.351354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:50.351354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:40:50.351354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:50.351354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:50.351354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:50.351354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:40:50.351354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:50.351354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:40:50.351354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:40:50.351354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:50.351354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:40:50.351354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:50.351951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:40:50.351951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:40:50.351951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:50.351951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:40:50.351951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:50.351951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:40:50.351951 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:40:50.351951 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:40:50.351951 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:40:50.351951 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:40:50.352565 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:40:50.352565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:50.352565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:40:50.352565 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:50.352565 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:40:50.352565 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:40:50.352565 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:50.352565 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:40:50.352565 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:50.352565 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:40:50.352967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:50.352967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:40:50.352967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:40:50.352967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:40:50.352967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:50.352967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:40:50.352967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:50.352967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:40:50.352967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:50.352967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:40:50.352967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:50.352967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:40:50.352967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:50.352967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:40:50.352967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:40:50.353525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:50.353525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:40:50.353525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:50.353525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:40:50.353525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:40:50.353525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:50.353525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:40:50.353525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:50.353525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:40:50.353780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:50.353780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:40:50.353780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:50.353780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:40:50.353780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:40:50.353780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:40:50.353780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:40:50.353780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:40:50.353780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:40:50.353780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:40:50.353780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:50.353780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:50.354120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:50.354120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:40:50.354120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:50.354120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:50.354120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:50.354120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:40:50.354120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:50.354120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:40:50.354120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:40:50.354120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:50.354120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:40:50.354120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:50.354120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:40:50.354120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:40:50.354547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:50.354547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:40:50.354547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:50.354547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:40:50.354782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:50.354782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:40:50.354782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:40:50.354782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:40:50.354782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:40:50.354782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:40:50.354782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:50.354782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:40:50.354782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:40:50.354782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:40:50.354782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:40:50.354782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:40:50.354782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:50.354782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:50.354782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:40:50.354782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:40:50.355709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:40:50.355709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:40:50.355709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:40:50.355709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:40:50.355709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:40:50.355709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:50.355709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:50.355709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:50.355709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:40:50.355709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:50.355709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:50.355709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:50.355709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:40:50.355709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:50.355709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:40:50.355709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:40:50.355709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:50.355709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:40:50.356195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:50.356195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:40:50.356195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:40:50.356195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:50.356195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:40:50.356195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:50.356195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:40:50.359369 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:40:50.359816 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:40:50.359870 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:40:50.360025 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:40:50.360553 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:40:50.361346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:50.361701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:40:50.361701 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:50.361701 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:40:50.361701 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:40:50.361701 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:50.361701 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:40:50.361701 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:50.361701 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:40:50.362277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:50.362277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:40:50.362277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:40:50.362277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:40:50.362277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:50.362277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:40:50.362277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:50.362277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:40:50.362277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:50.362277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:40:50.362277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:50.362277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:40:50.362277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:50.362277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:40:50.362277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:40:50.362857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:50.362857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:40:50.362857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:50.362857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:40:50.362857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:40:50.362857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:50.362857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:40:50.362857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:50.362857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:40:50.363135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:50.363135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:40:50.363135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:50.363135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:40:50.363135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:40:50.363135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:40:50.363135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:40:50.363135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:40:50.363135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:40:50.363135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:50.363135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:50.363438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:50.363438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:40:50.363438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:50.363438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:50.363438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:50.363438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:40:50.363438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:50.363438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:40:50.363438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:40:50.363438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:50.363438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:40:50.363438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:50.363438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:40:50.363438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:40:50.363924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:50.363924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:40:50.363924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:50.363924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:40:50.363924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:50.363924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:40:50.363924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:40:50.363924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:40:50.363924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:40:50.363924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:40:50.363924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:50.363924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:40:50.363924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:40:50.363924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:40:50.364351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:40:50.364351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:40:50.364351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:50.364351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:50.364351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:40:50.364351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:40:50.364351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:40:50.364351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:40:50.364351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:40:50.364351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:40:50.364351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:50.364351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:50.364351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:50.364351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:40:50.364351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:50.364351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:50.364351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:50.364351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:40:50.364351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:50.364901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:40:50.364901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:40:50.364901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:50.364901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:40:50.364901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:50.364901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:40:50.364901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:40:50.364901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:50.364901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:40:50.364901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:50.364901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:40:50.366213 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:40:50.366528 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:40:50.366601 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:40:50.366718 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:40:50.367211 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:40:50.367650 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:40:50.367887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:50.368582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:40:50.368582 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:50.368582 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:40:50.368582 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:40:50.368582 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:50.368582 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:40:50.368582 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:50.368582 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:40:50.369165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:50.369165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:40:50.369165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:50.369165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:40:50.369165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:40:50.369165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:40:50.369165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:40:50.369165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:50.369165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:50.369165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:50.369165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:40:50.369165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:50.369548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:50.369548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:50.369548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:40:50.369548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:50.369548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:40:50.369548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:40:50.369548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:50.369548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:40:50.369548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:50.369548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:40:50.369548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:40:50.369548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:50.369548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:40:50.369884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:50.369884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:40:50.370044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:50.370044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:40:50.370044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:40:50.370044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:40:50.370044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:40:50.370044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:40:50.370044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:50.370044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:40:50.370044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:40:50.370044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:40:50.370044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:40:50.370044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:40:50.370044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:50.370044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:50.370044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:40:50.370044 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:40:50.370555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:40:50.370555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:40:50.370555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:50.370555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:50.370555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:50.370555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:40:50.370555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:50.370555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:50.370555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:50.370555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:40:50.370555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:50.370555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:40:50.370555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:40:50.370555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:50.370555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:40:50.370555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:50.370555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:40:50.370555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:40:50.370555 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:50.371067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:40:50.371067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:50.371067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:40:55.110611 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:40:55.111556 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:40:55.111556 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:40:55.112611 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:40:55.145541 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:40:55.145866 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:55.147559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:40:55.147559 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:55.147559 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:40:55.147559 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:40:55.147559 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:55.147559 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:40:55.147559 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:55.147559 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:40:55.148144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:55.148144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:40:55.148144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:55.148144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:40:55.148144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:40:55.148144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:40:55.148144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:40:55.148144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:55.148144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:55.148144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:55.148144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:40:55.148144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:55.148825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:55.148825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:55.148825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:40:55.148825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:55.148825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:40:55.148825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:40:55.148825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:55.148825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:40:55.148825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:55.148825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:40:55.148825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:40:55.148825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:55.148825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:40:55.149531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:55.149531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:40:55.149531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:55.149531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:40:55.149531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:40:55.149531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:40:55.149531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:40:55.149531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:40:55.149531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:55.149531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:40:55.149531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:40:55.149531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:40:55.149531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:40:55.149531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:40:55.149531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:55.150290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:55.150290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:40:55.150290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:40:55.150290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:40:55.150290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:40:55.150290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:55.150290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:55.150290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:55.150290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:40:55.150290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:55.150290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:55.150290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:55.150290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:40:55.150290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:55.150290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:40:55.150290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:40:55.150290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:55.150290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:40:55.150290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:55.150964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:40:55.150964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:40:55.150964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:55.150964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:40:55.150964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:55.150964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:40:55.342619 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:40:55.343268 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:40:55.343268 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:40:55.345606 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:40:55.345606 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:40:55.345606 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:55.345957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:40:55.345957 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:55.345957 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:40:55.345957 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:40:55.345957 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:55.345957 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:40:55.345957 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:55.345957 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:40:55.347202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:55.347202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:40:55.347202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:55.347202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:40:55.347202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:40:55.347202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:40:55.347202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:40:55.347202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:55.347202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:55.347202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:55.347202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:40:55.347202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:55.347967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:55.347967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:55.347967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:40:55.347967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:55.347967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:40:55.347967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:40:55.347967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:55.347967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:40:55.347967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:55.347967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:40:55.347967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:40:55.347967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:55.347967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:40:55.348607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:55.348607 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:40:55.348786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:55.348786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:40:55.348786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:40:55.348786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:40:55.348786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:40:55.348786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:40:55.348786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:55.348786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:40:55.348786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:40:55.348786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:40:55.348786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:40:55.348786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:40:55.348786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:55.348786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:55.348786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:40:55.348786 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:40:55.349610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:40:55.349610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:40:55.349610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:55.349610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:55.349610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:55.349610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:40:55.349610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:55.349610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:55.349610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:55.349610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:40:55.349610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:55.349610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:40:55.349610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:40:55.349610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:55.349610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:40:55.349610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:55.349610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:40:55.349610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:40:55.349610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:55.350374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:40:55.350374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:55.350374 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:40:55.350852 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:40:55.351141 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:40:55.351245 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:40:55.351357 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:40:55.351716 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:40:55.352002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:55.352566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:40:55.352566 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:55.352566 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:40:55.352566 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:40:55.352566 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:55.352566 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:40:55.352566 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:55.352566 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:40:55.353276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:55.353276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:40:55.353276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:40:55.353276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:40:55.353276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:55.353276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:40:55.353276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:55.353276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:40:55.353276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:55.353276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:40:55.353276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:55.353276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:40:55.353276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:55.353276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:40:55.353276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:40:55.353862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:55.353862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:40:55.353862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:55.353862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:40:55.353862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:40:55.353862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:55.353862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:40:55.353862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:55.353862 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:40:55.354164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:55.354164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:40:55.354164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:55.354164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:40:55.354164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:40:55.354164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:40:55.354164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:40:55.354164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:40:55.354164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:40:55.354164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:40:55.354164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:55.354164 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:55.354571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:55.354571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:40:55.354571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:55.354571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:55.354571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:55.354571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:40:55.354571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:55.354571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:40:55.354571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:40:55.354571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:55.354571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:40:55.354571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:55.354571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:40:55.354571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:40:55.354966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:55.354966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:40:55.354966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:55.354966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:40:55.355082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:55.355082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:40:55.355082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:40:55.355082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:40:55.355082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:40:55.355082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:40:55.355082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:55.355082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:40:55.355082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:40:55.355082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:40:55.355082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:40:55.355082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:40:55.355082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:55.355082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:55.355082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:40:55.355082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:40:55.355528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:40:55.355528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:40:55.355528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:40:55.355528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:40:55.355528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:40:55.355528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:55.355528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:55.355528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:55.355528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:40:55.355528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:55.355528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:55.355528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:55.355528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:40:55.355528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:55.355528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:40:55.355528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:40:55.355528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:55.355528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:40:55.356029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:55.356029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:40:55.356029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:40:55.356029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:55.356029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:40:55.356029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:55.356029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:40:55.359989 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:40:55.360261 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:40:55.360348 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:40:55.360502 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:40:55.361384 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:40:55.361384 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:55.361559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:40:55.361559 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:55.361559 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:40:55.361559 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:40:55.361559 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:55.361559 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:40:55.361559 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:55.361559 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:40:55.362215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:55.362215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:40:55.362215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:40:55.362215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:40:55.362215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:55.362215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:40:55.362215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:55.362215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:40:55.362215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:55.362215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:40:55.362215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:55.362215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:40:55.362215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:55.362215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:40:55.362215 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:40:55.362668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:55.362668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:40:55.362668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:55.362668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:40:55.362668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:40:55.362668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:55.362668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:40:55.362668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:55.362668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:40:55.363054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:55.363054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:40:55.363054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:55.363054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:40:55.363054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:40:55.363054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:40:55.363054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:40:55.363054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:40:55.363054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:40:55.363054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:55.363054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:55.363547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:55.363547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:40:55.363547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:55.363547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:55.363547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:55.363547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:40:55.363547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:55.363547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:40:55.363547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:40:55.363547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:55.363547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:40:55.363547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:55.363547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:40:55.363547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:40:55.363937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:55.363937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:40:55.363937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:55.363937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:40:55.364096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:55.364096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:40:55.364096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:40:55.364096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:40:55.364096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:40:55.364096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:40:55.364096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:55.364096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:40:55.364096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:40:55.364096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:40:55.364096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:40:55.364096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:40:55.364096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:55.364096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:55.364096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:40:55.364096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:40:55.364096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:40:55.364598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:40:55.364598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:40:55.364598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:40:55.364598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:55.364598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:55.364598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:55.364598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:40:55.364598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:55.364598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:55.364598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:55.364598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:40:55.364598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:55.364598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:40:55.364598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:40:55.364598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:55.364598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:40:55.364598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:55.364598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:40:55.364598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:40:55.365096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:55.365096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:40:55.365096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:55.365096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:40:55.366705 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:40:55.366994 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:40:55.367102 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:40:55.367250 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:40:55.367634 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:40:55.367977 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:40:55.368212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:55.368768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:40:55.368768 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:55.368768 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:40:55.368768 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:40:55.368768 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:55.368768 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:40:55.368768 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:55.368768 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:40:55.369614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:55.369614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:40:55.369614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:55.369614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:40:55.369614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:40:55.369614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:40:55.369614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:40:55.369614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:55.369614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:55.369614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:55.369614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:40:55.369614 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:55.370013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:40:55.370013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:55.370013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:40:55.370013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:55.370013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:40:55.370013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:40:55.370013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:55.370013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:40:55.370013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:55.370013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:40:55.370013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:40:55.370013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:55.370013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:40:55.370391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:55.370391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:40:55.370641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:55.370641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:40:55.370641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:40:55.370641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:40:55.370641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:40:55.370641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:40:55.370641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:55.370641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:40:55.370641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:40:55.370641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:40:55.370641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:40:55.370641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:40:55.370641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:40:55.370641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:40:55.370641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:40:55.370641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:40:55.371084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:40:55.371084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:40:55.371084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:55.371084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:55.371084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:40:55.371084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:40:55.371084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:40:55.371084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:40:55.371084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:40:55.371084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:40:55.371084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:40:55.371084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:40:55.371084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:40:55.371084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:40:55.371084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:40:55.371084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:40:55.371084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:40:55.371084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:40:55.371084 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:40:55.371690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:40:55.371690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:40:55.371690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:41:00.111253 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:41:00.112068 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:41:00.112068 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:41:00.113308 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:41:00.141169 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:41:00.141528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:00.142364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:41:00.142364 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:00.142364 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:41:00.142364 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:41:00.142364 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:00.142364 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:41:00.142364 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:00.142364 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:41:00.143428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.030 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:00.143428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:41:00.143428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:00.143428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:41:00.143428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:41:00.143428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:41:00.143428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:41:00.143428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:00.143428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:00.143428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:00.143428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:41:00.143428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:00.144182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:00.144182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:00.144182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:41:00.144182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:00.144182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:41:00.144182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:41:00.144182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:00.144182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:41:00.144182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:00.144182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:41:00.144182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:41:00.144182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:00.144182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:41:00.146042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:00.146042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:41:00.146042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:00.146042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:41:00.146042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:41:00.146042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:41:00.146042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:41:00.146042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:41:00.146042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:00.146042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:41:00.146042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:41:00.146042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:41:00.146042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:41:00.146042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:41:00.146042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:00.147499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:00.147499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:41:00.147499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:41:00.147499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:41:00.147499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:41:00.147499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:00.147499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:00.147499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:00.147499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:41:00.147499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:00.147499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:00.147499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:00.147499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:41:00.147499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:00.147499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:41:00.147499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:41:00.147499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:00.147499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:41:00.147499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:00.148603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:41:00.148603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:41:00.148603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:00.148603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:41:00.148603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:00.148603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:41:00.343625 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:41:00.344090 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:41:00.344316 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:41:00.345494 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:41:00.346064 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:41:00.346207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:00.347706 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:41:00.347706 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:00.347706 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:41:00.347706 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:41:00.347706 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:00.347706 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:41:00.347706 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:00.347706 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:41:00.349578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:00.349578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:41:00.349578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:00.349578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:41:00.349578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:41:00.349578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:41:00.349578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:41:00.349578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:00.349578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:00.349578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:00.349578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:41:00.349578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:00.350352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:00.350352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:00.350352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:41:00.350352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:00.350352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:41:00.350352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:41:00.350352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:00.350352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:41:00.350352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:00.350352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:41:00.350352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:41:00.350352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:00.350352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:41:00.351026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:00.351026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:41:00.351194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:00.351194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:41:00.351194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:41:00.351194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:41:00.351194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:41:00.351194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:41:00.351194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:00.351194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:41:00.351194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:41:00.351194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:41:00.351194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:41:00.351194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:41:00.351194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:00.351194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:00.351194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:41:00.351194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:41:00.352200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:41:00.352200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:41:00.352200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:00.352200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:00.352200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:00.352200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:41:00.352200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:00.352200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:00.352200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:00.352200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:41:00.352200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:00.352200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:41:00.352200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:41:00.352200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:00.352200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:41:00.352200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:00.352200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:41:00.352200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:41:00.352200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:00.353226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:41:00.353226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:00.353226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:41:00.353226 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:41:00.353226 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:41:00.353226 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:41:00.353226 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:41:00.353226 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:41:00.353519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:00.354139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:41:00.354139 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:00.354139 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:41:00.354139 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:41:00.354139 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:00.354139 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:41:00.354139 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:00.354139 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:41:00.354714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:00.354714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:41:00.354714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:41:00.354714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:41:00.354714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:00.354714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:41:00.354714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:00.354714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:41:00.354714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:00.354714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:41:00.354714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:00.354714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:41:00.354714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:00.354714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:41:00.354714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:41:00.355450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:00.355450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:41:00.355450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:00.355450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:41:00.355450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:41:00.355450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:00.355450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:41:00.355450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:00.355450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:41:00.355874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:00.355874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:41:00.355874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:00.355874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:41:00.355874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:41:00.355874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:41:00.355874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:41:00.355874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:41:00.355874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:41:00.355874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:41:00.355874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:00.355874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:00.356431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:00.356431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:41:00.356431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:00.356431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:00.356431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:00.356431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:41:00.356431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:00.356431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:41:00.356431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:41:00.356431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:00.356431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:41:00.356431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:00.356431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:41:00.356431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:41:00.357338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:00.357338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:41:00.357338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:00.357338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:41:00.357338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:00.357338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:41:00.357338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:41:00.357338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:41:00.357338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:41:00.357338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:41:00.357338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:00.357338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:41:00.357338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:41:00.357338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:41:00.358179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:41:00.358179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:41:00.358179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:00.358179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:00.358179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:41:00.358179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:41:00.358179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:41:00.358179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:41:00.358179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:41:00.358179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:41:00.358179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:41:00.358179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:00.358179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:00.358179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:00.358179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:41:00.358179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:00.358179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:00.358179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:00.358179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:41:00.358977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:00.358977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:41:00.358977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:41:00.358977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:00.358977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:41:00.358977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:00.358977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:41:00.358977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:41:00.358977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:00.358977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:41:00.358977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:00.358977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:41:00.360441 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:41:00.360705 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:41:00.360803 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:41:00.360947 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:41:00.361624 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:41:00.362099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:00.362777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:41:00.362777 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:00.362777 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:41:00.362777 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:41:00.362777 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:00.362777 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:41:00.362777 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:00.362777 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:41:00.363643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:00.363643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:41:00.363643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:41:00.363643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:41:00.363643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:00.363643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:41:00.363643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:00.363643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:41:00.363643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:00.363643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:41:00.363643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:00.363643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:41:00.363643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:00.363643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:41:00.363643 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:41:00.364234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:00.364234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:41:00.364234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:00.364234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:41:00.364234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:41:00.364234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:00.364234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:41:00.364234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:00.364234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:41:00.364617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:00.364617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:41:00.364617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:00.364617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:41:00.364617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:41:00.364617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:41:00.364617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:41:00.364617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:41:00.364617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:41:00.364617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:00.364617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:00.365842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:00.365842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:41:00.365842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:00.365842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:00.365842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:00.365842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:41:00.365842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:00.365842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:41:00.365842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:41:00.365842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:00.365842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:41:00.365842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:00.365842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:41:00.365842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:41:00.366400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:00.366400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:41:00.366400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:00.366400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:41:00.366400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:00.366400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:41:00.366400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:41:00.366400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:41:00.366400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:41:00.366400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:41:00.366400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:00.366400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:41:00.366400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:41:00.366400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:41:00.367018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:41:00.367018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:41:00.367018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:00.367018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:00.367018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:41:00.367018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:41:00.367018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:41:00.367018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:41:00.367018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:41:00.367018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:41:00.367018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:00.367018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:00.367018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:00.367018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:41:00.367018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:00.367018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:00.367018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:00.367018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:41:00.367018 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:00.367905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:41:00.367905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:41:00.367905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:00.367905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:41:00.367905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:00.367905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:41:00.367905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:41:00.367905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:00.367905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:41:00.367905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:00.367905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:41:00.367905 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:41:00.367905 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:41:00.368485 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:41:00.368485 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:41:00.368485 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:41:00.368485 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:41:00.368700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:00.369343 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:41:00.369343 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:00.369343 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:41:00.369343 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:41:00.369343 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:00.369343 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:41:00.369343 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:00.369343 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:41:00.370139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:00.370139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:41:00.370139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:00.370139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:41:00.370139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:41:00.370139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:41:00.370139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:41:00.370139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:00.370139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:00.370139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:00.370139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:41:00.370139 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:00.370682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:00.370682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:00.370682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:41:00.370682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:00.370682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:41:00.370682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:41:00.370682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:00.370682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:41:00.370682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:00.370682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:41:00.370682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:41:00.370682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:00.370682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:41:00.371217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:00.371217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:41:00.371217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:00.371217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:41:00.371217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:41:00.371217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:41:00.371217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:41:00.371217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:41:00.371217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:00.371217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:41:00.371217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:41:00.371217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:41:00.371217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:41:00.371217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:41:00.371217 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:00.371852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:00.371852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:41:00.371852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:41:00.371852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:41:00.371852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:41:00.371852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:00.371852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:00.371852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:00.371852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:41:00.371852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:00.371852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:00.371852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:00.371852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:41:00.371852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:00.371852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:41:00.371852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:41:00.371852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:00.371852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:41:00.371852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:00.372630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:41:00.372630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:41:00.372630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:00.372630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:41:00.372630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:00.372630 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:41:05.112091 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:41:05.112909 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:41:05.112909 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:41:05.113558 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:41:05.144842 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:41:05.145230 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:05.146214 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:41:05.146214 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:05.146214 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:41:05.146214 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:41:05.146214 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:05.146214 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:41:05.146214 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:05.146214 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:41:05.147625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:05.147625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:41:05.147625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:05.147625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:41:05.147625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:41:05.147625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:41:05.147625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:41:05.147625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:05.147625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:05.147625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:05.147625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:41:05.147625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:05.148100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:05.148100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:05.148100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:41:05.148100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:05.148100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:41:05.148100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:41:05.148100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:05.148100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:41:05.148100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:05.148100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:41:05.148100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:41:05.148100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:05.148100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:41:05.149033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:05.149033 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:41:05.149256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:05.149256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:41:05.149256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:41:05.149256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:41:05.149256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:41:05.149256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:41:05.149256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:05.149256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:41:05.149256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:41:05.149256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:41:05.149256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:41:05.149256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:41:05.149256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:05.149256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:05.149256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:41:05.149256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:41:05.150289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:41:05.150289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:41:05.150289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:05.150289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:05.150289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:05.150289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:41:05.150289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:05.150289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:05.150289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:05.150289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:41:05.150289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:05.150289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:41:05.150289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:41:05.150289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:05.150289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:41:05.150289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:05.150289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:41:05.150289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:41:05.150289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:05.151361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:41:05.151361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:05.151361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:41:05.344065 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:41:05.344464 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:41:05.344628 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:41:05.346299 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:41:05.346299 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:41:05.346299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:05.347151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:41:05.347151 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:05.347151 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:41:05.347151 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:41:05.347151 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:05.347151 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:41:05.347151 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:05.347151 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:41:05.348130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:05.348130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:41:05.348130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:05.348130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:41:05.348130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:41:05.348130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:41:05.348130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:41:05.348130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:05.348130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:05.348130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:05.348130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:41:05.348130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:05.348774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:05.348774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:05.348774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:41:05.348774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:05.348774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:41:05.348774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:41:05.348774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:05.348774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:41:05.348774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:05.348774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:41:05.348774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:41:05.348774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:05.348774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:41:05.349447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:05.349447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:41:05.349447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:05.349447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:41:05.349447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:41:05.349447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:41:05.349447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:41:05.349447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:41:05.349447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:05.349447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:41:05.349447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:41:05.349447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:41:05.349447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:41:05.349447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:41:05.349447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:05.350170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:05.350170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:41:05.350170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:41:05.350170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:41:05.350170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:41:05.350170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:05.350170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:05.350170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:05.350170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:41:05.350170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:05.350170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:05.350170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:05.350170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:41:05.350170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:05.350170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:41:05.350170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:41:05.350170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:05.350170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:41:05.350170 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:05.351045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:41:05.351045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:41:05.351045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:05.351045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:41:05.351045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:05.351045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:41:05.351923 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:41:05.352247 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:41:05.352433 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:41:05.352581 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:41:05.353083 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:41:05.353375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:05.354119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:41:05.354119 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:05.354119 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:41:05.354119 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:41:05.354119 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:05.354119 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:41:05.354119 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:05.354119 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:41:05.354765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:05.354765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:41:05.354765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:41:05.354765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:41:05.354765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:05.354765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:41:05.354765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:05.354765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:41:05.354765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:05.354765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:41:05.354765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:05.354765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:41:05.354765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:05.354765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:41:05.354765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:41:05.355167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:05.355167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:41:05.355167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:05.355167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:41:05.355167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:41:05.355167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:05.355167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:41:05.355167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:05.355167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:41:05.355635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:05.355635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:41:05.355635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:05.355635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:41:05.355635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:41:05.355635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:41:05.355635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:41:05.355635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:41:05.355635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:41:05.355635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:41:05.355635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:05.355635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:05.355979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:05.355979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:41:05.355979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:05.355979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:05.355979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:05.355979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:41:05.355979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:05.355979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:41:05.355979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:41:05.355979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:05.355979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:41:05.355979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:05.355979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:41:05.355979 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:41:05.356340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:05.356340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:41:05.356340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:05.356340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:41:05.356674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:05.356674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:41:05.356674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:41:05.356674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:41:05.356674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:41:05.356674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:41:05.356674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:05.356674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:41:05.356674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:41:05.356674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:41:05.356674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:41:05.356674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:41:05.356674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:05.356674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:05.356674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:41:05.356674 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:41:05.357186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:41:05.357186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:41:05.357186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:41:05.357186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:41:05.357186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:41:05.357186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:05.357186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:05.357186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:05.357186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:41:05.357186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:05.357186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:05.357186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:05.357186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:41:05.357186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:05.357186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:41:05.357186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:41:05.357186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:05.357186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:41:05.357740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:05.357740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:41:05.357740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:41:05.357740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:05.357740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:41:05.357740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:05.357740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:41:05.360711 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:41:05.360950 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:41:05.361045 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:41:05.361204 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:41:05.361661 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:41:05.362016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:05.362772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:41:05.362772 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:05.362772 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:41:05.362772 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:41:05.362772 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:05.362772 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:41:05.362772 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:05.362772 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:41:05.363505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:05.363505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:41:05.363505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:41:05.363505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:41:05.363505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:05.363505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:41:05.363505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:05.363505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:41:05.363505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:05.363505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:41:05.363505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:05.363505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:41:05.363505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:05.363505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:41:05.363505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:41:05.363951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:05.363951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:41:05.363951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:05.363951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:41:05.363951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:41:05.363951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:05.363951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:41:05.363951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:05.363951 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:41:05.364377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:05.364377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:41:05.364377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:05.364377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:41:05.364377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:41:05.364377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:41:05.364377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:41:05.364377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:41:05.364377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:41:05.364377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:05.364377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:05.364772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:05.364772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:41:05.364772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:05.364772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:05.364772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:05.364772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:41:05.364772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:05.364772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:41:05.364772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:41:05.364772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:05.364772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:41:05.364772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:05.364772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:41:05.364772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:41:05.365197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:05.365197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:41:05.365197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:05.365197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:41:05.365518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:05.365518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:41:05.365518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:41:05.365518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:41:05.365518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:41:05.365518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:41:05.365518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:05.365518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:41:05.365518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:41:05.365518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:41:05.365518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:41:05.365518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:41:05.365518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:05.365518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:05.365518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:41:05.365518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:41:05.365518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:41:05.366183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:41:05.366183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:41:05.366183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:41:05.366183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:05.366183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:05.366183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:05.366183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:41:05.366183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:05.366183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:05.366183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:05.366183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:41:05.366183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:05.366183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:41:05.366183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:41:05.366183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:05.366183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:41:05.366183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:05.366183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:41:05.366183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:41:05.366838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:05.366838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:41:05.366838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:05.366838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:41:05.367352 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:41:05.367580 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:41:05.367686 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:41:05.367852 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:41:05.368308 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:41:05.368700 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:41:05.368943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:05.369609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:41:05.369609 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:05.369609 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:41:05.369609 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:41:05.369609 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:05.369609 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:41:05.369609 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:05.369609 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:41:05.370428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:05.370428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:41:05.370428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:05.370428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:41:05.370428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:41:05.370428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:41:05.370428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:41:05.370428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:05.370428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:05.370428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:05.370428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:41:05.370428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:05.370838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:05.370838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:05.370838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:41:05.370838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:05.370838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:41:05.370838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:41:05.370838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:05.370838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:41:05.370838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:05.370838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:41:05.370838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:41:05.370838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:05.370838 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:41:05.371210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:05.371210 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:41:05.371416 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:05.371416 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:41:05.371416 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:41:05.371416 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:41:05.371416 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:41:05.371416 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:41:05.371416 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:05.371416 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:41:05.371416 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:41:05.371416 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:41:05.371416 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:41:05.371416 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:41:05.371416 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:05.371416 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:05.371416 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:41:05.371416 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:41:05.371981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:41:05.371981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:41:05.371981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:05.371981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:05.371981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:05.371981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:41:05.371981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:05.371981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:05.371981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:05.371981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:41:05.371981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:05.371981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:41:05.371981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:41:05.371981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:05.371981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:41:05.371981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:05.371981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:41:05.371981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:41:05.371981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:05.372783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:41:05.372783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:05.372783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:41:10.112815 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:41:10.113945 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:41:10.113945 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:41:10.115126 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:41:10.150984 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:41:10.151441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:10.152022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:41:10.152022 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:10.152022 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:41:10.152022 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:41:10.152022 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:10.152022 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:41:10.152022 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:10.152022 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:41:10.152642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:10.152642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:41:10.152642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:10.152642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:41:10.152642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:41:10.152642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:41:10.152642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:41:10.152642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:10.152642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:10.152642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:10.152642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:41:10.152642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:10.153574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:10.153574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:10.153574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:41:10.153574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:10.153574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:41:10.153574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:41:10.153574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:10.153574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:41:10.153574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:10.153574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:41:10.153574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:41:10.153574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:10.153574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:41:10.154328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:10.154328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:41:10.154328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:10.154328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:41:10.154328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:41:10.154328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:41:10.154328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:41:10.154328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:41:10.154328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:10.154328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:41:10.154328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:41:10.154328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:41:10.154328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:41:10.154328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:41:10.154328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:10.155261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:10.155261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:41:10.155261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:41:10.155261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:41:10.155261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:41:10.155261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:10.155261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:10.155261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:10.155261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:41:10.155261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:10.155261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:10.155261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:10.155261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:41:10.155261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:10.155261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:41:10.155261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:41:10.155261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:10.155261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:41:10.155261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:10.156328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:41:10.156328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:41:10.156328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:10.156328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:41:10.156328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:10.156328 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:41:10.344707 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:41:10.345177 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:41:10.345267 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:41:10.346097 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:41:10.346492 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:41:10.346832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:10.347746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:41:10.347746 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:10.347746 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:41:10.347746 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:41:10.347746 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:10.347746 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:41:10.347746 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:10.347746 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:41:10.348584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:10.348584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:41:10.348584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:10.348584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:41:10.348584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:41:10.348584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:41:10.348584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:41:10.348584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:10.348584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:10.348584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:10.348584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:41:10.348584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:10.349197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:10.349197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:10.349197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:41:10.349197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:10.349197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:41:10.349197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:41:10.349197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:10.349197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:41:10.349197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:10.349197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:41:10.349197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:41:10.349197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:10.349197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:41:10.349829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:10.349829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:41:10.349829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:10.349829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:41:10.349829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:41:10.349829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:41:10.349829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:41:10.349829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:41:10.349829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:10.349829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:41:10.349829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:41:10.349829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:41:10.349829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:41:10.349829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:41:10.349829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:10.350669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:10.350669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:41:10.350669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:41:10.350669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:41:10.350669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:41:10.350669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:10.350669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:10.350669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:10.350669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:41:10.350669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:10.350669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:10.350669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:10.350669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:41:10.350669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:10.350669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:41:10.350669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:41:10.350669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:10.350669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:41:10.350669 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:10.351269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:41:10.351269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:41:10.351269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:10.351269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:41:10.351269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:10.351269 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:41:10.352361 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:41:10.352656 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:41:10.352713 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:41:10.352817 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:41:10.353261 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:41:10.353595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:10.354238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:41:10.354238 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:10.354238 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:41:10.354238 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:41:10.354238 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:10.354238 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:41:10.354238 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:10.354238 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:41:10.354808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:10.354808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:41:10.354808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:41:10.354808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:41:10.354808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:10.354808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:41:10.354808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:10.354808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:41:10.354808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:10.354808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:41:10.354808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:10.354808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:41:10.354808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:10.354808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:41:10.354808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:41:10.355336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:10.355336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:41:10.355336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:10.355336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:41:10.355336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:41:10.355336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:10.355336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:41:10.355336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:10.355336 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:41:10.355664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:10.355664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:41:10.355664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:10.355664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:41:10.355664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:41:10.355664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:41:10.355664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:41:10.355664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:41:10.355664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:41:10.355664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:41:10.355664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:10.355664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:10.356003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:10.356003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:41:10.356003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:10.356003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:10.356003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:10.356003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:41:10.356003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:10.356003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:41:10.356003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:41:10.356003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:10.356003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:41:10.356003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:10.356003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:41:10.356003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:41:10.356359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:10.356359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:41:10.356359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:10.356359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:41:10.356359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:10.356359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:41:10.356359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:41:10.356359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:41:10.356359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:41:10.356359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:41:10.356359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:10.356359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:41:10.356359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:41:10.356359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:41:10.356772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:41:10.356772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:41:10.356772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:10.356772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:10.356772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:41:10.356772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:41:10.356772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:41:10.356772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:41:10.356772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:41:10.356772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:41:10.356772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:41:10.356772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:10.356772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:10.356772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:10.356772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:41:10.356772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:10.356772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:10.356772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:10.356772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:41:10.357337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:10.357337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:41:10.357337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:41:10.357337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:10.357337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:41:10.357337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:10.357337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:41:10.357337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:41:10.357337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:10.357337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:41:10.357337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:10.357337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:41:10.361145 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:41:10.361427 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:41:10.361503 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:41:10.361587 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:41:10.361935 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:41:10.362108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:10.362550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:41:10.362550 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:10.362550 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:41:10.362550 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:41:10.362550 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:10.362550 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:41:10.362550 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:10.362550 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:41:10.363085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:10.363085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:41:10.363085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:41:10.363085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:41:10.363085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:10.363085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:41:10.363085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:10.363085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:41:10.363085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:10.363085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:41:10.363085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:10.363085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:41:10.363085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:10.363085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:41:10.363085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:41:10.363538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:10.363538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:41:10.363538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:10.363538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:41:10.363538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:41:10.363538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:10.363538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:41:10.363538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:10.363538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:41:10.363784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:10.363784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:41:10.363784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:10.363784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:41:10.363784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:41:10.363784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:41:10.363784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:41:10.363784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:41:10.363784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:41:10.363784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:10.363784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:10.364130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:10.364130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:41:10.364130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:10.364130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:10.364130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:10.364130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:41:10.364130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:10.364130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:41:10.364130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:41:10.364130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:10.364130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:41:10.364130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:10.364130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:41:10.364130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:41:10.364553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:10.364553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:41:10.364553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:10.364553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:41:10.364553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:10.364553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:41:10.364553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:41:10.364553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:41:10.364553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:41:10.364553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:41:10.364553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:10.364553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:41:10.364553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:41:10.364553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:41:10.364980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:41:10.364980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:41:10.364980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:10.364980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:10.364980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:41:10.364980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:41:10.364980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:41:10.364980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:41:10.364980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:41:10.364980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:41:10.364980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:10.364980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:10.364980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:10.364980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:41:10.364980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:10.364980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:10.364980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:10.364980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:41:10.364980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:10.365538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:41:10.365538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:41:10.365538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:10.365538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:41:10.365538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:10.365538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:41:10.365538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:41:10.365538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:10.365538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:41:10.365538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:10.365538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:41:10.367760 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:41:10.367994 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:41:10.368067 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:41:10.368178 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:41:10.368756 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:41:10.369087 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:41:10.369340 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:10.370077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:41:10.370077 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:10.370077 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:41:10.370077 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:41:10.370077 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:10.370077 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:41:10.370077 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:10.370077 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:41:10.370651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:10.370651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:41:10.370651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:10.370651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:41:10.370651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:41:10.370651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:41:10.370651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:41:10.370651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:10.370651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:10.370651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:10.370651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:41:10.370651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:10.371031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:10.371031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:10.371031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:41:10.371031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:10.371031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:41:10.371031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:41:10.371031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:10.371031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:41:10.371031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:10.371031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:41:10.371031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:41:10.371031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:10.371031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:41:10.371395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:10.371395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:41:10.371395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:10.371395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:41:10.371395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:41:10.371395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:41:10.371395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:41:10.371395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:41:10.371395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:10.371395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:41:10.371395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:41:10.371395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:41:10.371395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:41:10.371395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:41:10.371395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:10.371874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:10.371874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:41:10.371874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:41:10.371874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:41:10.371874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:41:10.371874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:10.371874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:10.371874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:10.371874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:41:10.371874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:10.371874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:10.371874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:10.371874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:41:10.371874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:10.371874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:41:10.371874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:41:10.371874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:10.371874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:41:10.371874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:10.372428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:41:10.372428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:41:10.372428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:10.372428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:41:10.372428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:10.372428 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:41:15.113532 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:41:15.114569 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:41:15.114569 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:41:15.115437 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:41:15.147683 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:41:15.148076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:15.148924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:41:15.148924 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:15.148924 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:41:15.148924 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:41:15.148924 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:15.148924 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:41:15.148924 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:15.148924 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:41:15.149705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:15.149705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:41:15.149705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:15.149705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:41:15.149705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:41:15.149705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:41:15.149705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:41:15.149705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:15.149705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:15.149705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:15.149705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:41:15.149705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:15.150356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:15.150356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:15.150356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:41:15.150356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:15.150356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:41:15.150356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:41:15.150356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:15.150356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:41:15.150356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:15.150356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:41:15.150356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:41:15.150356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:15.150356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:41:15.151041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:15.151041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:41:15.151041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:15.151041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:41:15.151041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:41:15.151041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:41:15.151041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:41:15.151041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:41:15.151041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:15.151041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:41:15.151041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:41:15.151041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:41:15.151041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:41:15.151041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:41:15.151041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:15.151867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:15.151867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:41:15.151867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:41:15.151867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:41:15.151867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:41:15.151867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:15.151867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:15.151867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:15.151867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:41:15.151867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:15.151867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:15.151867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:15.151867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:41:15.151867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:15.151867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:41:15.151867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:41:15.151867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:15.151867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:41:15.151867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:15.152846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:41:15.152846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:41:15.152846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:15.152846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:41:15.152846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:15.152846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:41:15.345356 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:41:15.345748 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:41:15.345904 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:41:15.346793 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:41:15.346997 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:41:15.347090 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:15.347975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:41:15.347975 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:15.347975 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:41:15.347975 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:41:15.347975 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:15.347975 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:41:15.347975 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:15.347975 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:41:15.349591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:15.349591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:41:15.349591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:15.349591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:41:15.349591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:41:15.349591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:41:15.349591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:41:15.349591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:15.349591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:15.349591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:15.349591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:41:15.349591 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:15.350199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:15.350199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:15.350199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:41:15.350199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:15.350199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:41:15.350199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:41:15.350199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:15.350199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:41:15.350199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:15.350199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:41:15.350199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:41:15.350199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:15.350199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:41:15.350842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:15.350842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:41:15.351501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:15.351501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:41:15.351501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:41:15.351501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:41:15.351501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:41:15.351501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:41:15.351501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:15.351501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:41:15.351501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:41:15.351501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:41:15.351501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:41:15.351501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:41:15.351501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:15.351501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:15.351501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:41:15.351501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:41:15.352064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:41:15.352064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:41:15.352064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:15.352064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:15.352064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:15.352064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:41:15.352064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:15.352064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:15.352064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:15.352064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:41:15.352064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:15.352064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:41:15.352064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:41:15.352064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:15.352064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:41:15.352064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:15.352064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:41:15.352064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:41:15.352064 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:15.352609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:41:15.352609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:15.352609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:41:15.352911 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:41:15.353214 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:41:15.353322 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:41:15.353465 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:41:15.353874 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:41:15.353954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:15.354312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:41:15.354312 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:15.354312 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:41:15.354312 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:41:15.354312 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:15.354312 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:41:15.354312 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:15.354312 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:41:15.355196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:15.355196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:41:15.355196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:41:15.355196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:41:15.355196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:15.355196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:41:15.355196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:15.355196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:41:15.355196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:15.355196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:41:15.355196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:15.355196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:41:15.355196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:15.355196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:41:15.355196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:41:15.355702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:15.355702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:41:15.355702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:15.355702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:41:15.355702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:41:15.355702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:15.355702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:41:15.355702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:15.355702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:41:15.356009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:15.356009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:41:15.356009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:15.356009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:41:15.356009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:41:15.356009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:41:15.356009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:41:15.356009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:41:15.356009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:41:15.356009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:41:15.356009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:15.356009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:15.356360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:15.356360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:41:15.356360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:15.356360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:15.356360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:15.356360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:41:15.356360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:15.356360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:41:15.356360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:41:15.356360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:15.356360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:41:15.356360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:15.356360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:41:15.356360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:41:15.356761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:15.356761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:41:15.356761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:15.356761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:41:15.357027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:15.357027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:41:15.357027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:41:15.357027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:41:15.357027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:41:15.357027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:41:15.357027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:15.357027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:41:15.357027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:41:15.357027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:41:15.357027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:41:15.357027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:41:15.357027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:15.357027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:15.357027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:41:15.357027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:41:15.357533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:41:15.357533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:41:15.357533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:41:15.357533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:41:15.357533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:41:15.357533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:15.357533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:15.357533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:15.357533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:41:15.357533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:15.357533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:15.357533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:15.357533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:41:15.357533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:15.357533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:41:15.357533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:41:15.357533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:15.357533 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:41:15.358065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:15.358065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:41:15.358065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:41:15.358065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:15.358065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:41:15.358065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:15.358065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:41:15.361545 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:41:15.361784 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:41:15.361900 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:41:15.362019 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:41:15.362525 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:41:15.362878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:15.363874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:41:15.363874 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:15.363874 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:41:15.363874 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:41:15.363874 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:15.363874 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:41:15.363874 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:15.363874 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:41:15.364546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:15.364546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:41:15.364546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:41:15.364546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:41:15.364546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:15.364546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:41:15.364546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:15.364546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:41:15.364546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:15.364546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:41:15.364546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:15.364546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:41:15.364546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:15.364546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:41:15.364546 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:41:15.364974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:15.364974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:41:15.364974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:15.364974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:41:15.364974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:41:15.364974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:15.364974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:41:15.364974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:15.364974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:41:15.365392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:15.365392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:41:15.365392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:15.365392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:41:15.365392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:41:15.365392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:41:15.365392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:41:15.365392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:41:15.365392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:41:15.365392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:15.365392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:15.365755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:15.365755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:41:15.365755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:15.365755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:15.365755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:15.365755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:41:15.365755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:15.365755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:41:15.365755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:41:15.365755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:15.365755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:41:15.365755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:15.365755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:41:15.365755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:41:15.366123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:15.366123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:41:15.366123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:15.366123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:41:15.366483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:15.366483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:41:15.366483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:41:15.366483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:41:15.366483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:41:15.366483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:41:15.366483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:15.366483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:41:15.366483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:41:15.366483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:41:15.366483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:41:15.366483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:41:15.366483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:15.366483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:15.366483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:41:15.366483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:41:15.366483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:41:15.366955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:41:15.366955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:41:15.366955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:41:15.366955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:15.366955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:15.366955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:15.366955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:41:15.366955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:15.366955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:15.366955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:15.366955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:41:15.366955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:15.366955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:41:15.366955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:41:15.366955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:15.366955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:41:15.366955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:15.366955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:41:15.366955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:41:15.367495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:15.367495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:41:15.367495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:15.367495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:41:15.368100 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:41:15.368366 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:41:15.368493 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:41:15.368651 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:41:15.369064 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:41:15.369394 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:41:15.369663 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:15.370297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:41:15.370297 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:15.370297 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:41:15.370297 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:41:15.370297 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:15.370297 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:41:15.370297 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:15.370297 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:41:15.371112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:15.371112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:41:15.371112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:15.371112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:41:15.371112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:41:15.371112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:41:15.371112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:41:15.371112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:15.371112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:15.371112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:15.371112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:41:15.371112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:15.371471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:15.371471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:15.371471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:41:15.371471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:15.371471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:41:15.371471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:41:15.371471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:15.371471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:41:15.371471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:15.371471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:41:15.371471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:41:15.371471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:15.371471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:41:15.371843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:15.371843 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:41:15.372099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:15.372099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:41:15.372099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:41:15.372099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:41:15.372099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:41:15.372099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:41:15.372099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:15.372099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:41:15.372099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:41:15.372099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:41:15.372099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:41:15.372099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:41:15.372099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:15.372099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:15.372099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:41:15.372099 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:41:15.372553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:41:15.372553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:41:15.372553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:15.372553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:15.372553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:15.372553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:41:15.372553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:15.372553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:15.372553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:15.372553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:41:15.372553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:15.372553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:41:15.372553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:41:15.372553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:15.372553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:41:15.372553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:15.372553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:41:15.372553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:41:15.372553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:15.373140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:41:15.373140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:15.373140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:41:20.114060 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:41:20.115114 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:41:20.115114 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:41:20.115956 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:41:20.153436 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:41:20.153665 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:20.154212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:41:20.154212 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:20.154212 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:41:20.154212 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:41:20.154212 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:20.154212 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:41:20.154212 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:20.154212 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:41:20.155281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:20.155281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:41:20.155281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:20.155281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:41:20.155281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:41:20.155281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:41:20.155281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:41:20.155281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:20.155281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:20.155281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:20.155281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:41:20.155281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:20.156572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:20.156572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:20.156572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:41:20.156572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:20.156572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:41:20.156572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:41:20.156572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:20.156572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:41:20.156572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:20.156572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:41:20.156572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:41:20.156572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:20.156572 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:41:20.157632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:20.157632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:41:20.157632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:20.157632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:41:20.157632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:41:20.157632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:41:20.157632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:41:20.157632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:41:20.157632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:20.157632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:41:20.157632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:41:20.157632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:41:20.157632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:41:20.157632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:41:20.157632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:20.158364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:20.158364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:41:20.158364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:41:20.158364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:41:20.158364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:41:20.158364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:20.158364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:20.158364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:20.158364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:41:20.158364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:20.158364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:20.158364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:20.158364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:41:20.158364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:20.158364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:41:20.158364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:41:20.158364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:20.158364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:41:20.158364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:20.159321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:41:20.159321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:41:20.159321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:20.159321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:41:20.159321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:20.159321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:41:20.346013 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:41:20.346749 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:41:20.346942 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:41:20.348273 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:41:20.348537 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:41:20.348777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:20.349666 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:41:20.349666 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:20.349666 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:41:20.349666 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:41:20.349666 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:20.349666 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:41:20.349666 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:20.349666 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:41:20.353431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:20.353431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:41:20.353431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:20.353431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:41:20.353431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:41:20.353431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:41:20.353431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:41:20.353431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:20.353431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:20.353431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:20.353431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:41:20.353431 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:20.353817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:20.353817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:20.353817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:41:20.353817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:20.353817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:41:20.353817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:41:20.353817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:20.353817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:41:20.353817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:20.353817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:41:20.353817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:41:20.353817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:20.353817 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:41:20.354154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:20.354154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:41:20.354154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:20.354154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:41:20.354154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:41:20.354154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:41:20.354154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:41:20.354154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:41:20.354154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:20.354154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:41:20.354154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:41:20.354154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:41:20.354154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:41:20.354154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:41:20.354154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:20.354893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:20.354893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:41:20.354893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:41:20.354893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:41:20.354893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:41:20.354893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:20.354893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:20.354893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:20.354893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:41:20.354893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:20.354893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:20.354893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:20.354893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:41:20.354893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:20.354893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:41:20.354893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:41:20.354893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:20.354893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:41:20.354893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:20.355444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:41:20.355444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:41:20.355444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:20.355444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:41:20.355444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:20.355444 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:41:20.355444 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:41:20.355444 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:41:20.355444 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:41:20.355444 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:41:20.355733 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:41:20.355733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:20.355733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:41:20.355733 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:20.355733 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:41:20.355733 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:41:20.355733 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:20.355733 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:41:20.355733 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:20.355733 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:41:20.356017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:20.356017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:41:20.356017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:41:20.356017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:41:20.356017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:20.356017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:41:20.356017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:20.356017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:41:20.356017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:20.356017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:41:20.356017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:20.356017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:41:20.356017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:20.356017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:41:20.356017 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:41:20.357163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:20.357163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:41:20.357163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:20.357163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:41:20.357163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:41:20.357163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:20.357163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:41:20.357163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:20.357163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:41:20.357163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:20.357163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:41:20.357163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:20.357163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:41:20.357163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:41:20.357616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:41:20.357616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:41:20.357616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:41:20.357616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:41:20.357616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:41:20.357616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:20.357616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:20.357616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:20.357616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:41:20.357616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:20.357616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:20.357616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:20.357616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:41:20.357989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:20.357989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:41:20.357989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:41:20.357989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:20.357989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:41:20.357989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:20.357989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:41:20.357989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:41:20.357989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:20.357989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:41:20.357989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:20.357989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:41:20.358359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:20.358359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:41:20.358359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:41:20.358359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:41:20.358359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:41:20.358359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:41:20.358359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:20.358359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:41:20.358359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:41:20.358359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:41:20.358359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:41:20.358359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:41:20.358359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:20.358359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:20.358359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:41:20.358359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:41:20.358896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:41:20.358896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:41:20.358896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:41:20.358896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:41:20.358896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:41:20.358896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:20.358896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:20.358896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:20.358896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:41:20.358896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:20.358896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:20.358896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:20.358896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:41:20.358896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:20.358896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:41:20.358896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:41:20.358896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:20.358896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:41:20.359439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:20.359439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:41:20.359439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:41:20.359439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:20.359439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:41:20.359439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:20.359439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:41:20.361899 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:41:20.363642 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:41:20.363753 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:41:20.363871 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:41:20.364491 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:41:20.364780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:20.365550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:41:20.365550 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:20.365550 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:41:20.365550 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:41:20.365550 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:20.365550 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:41:20.365550 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:20.365550 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:41:20.366262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:20.366262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:41:20.366262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:41:20.366262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:41:20.366262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:20.366262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:41:20.366262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:20.366262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:41:20.366262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:20.366262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:41:20.366262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:20.366262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:41:20.366262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:20.366262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:41:20.366262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:41:20.366569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:20.366569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:41:20.366569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:20.366569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:41:20.366569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:41:20.366569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:20.366569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:41:20.366569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:20.366569 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:41:20.367161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:20.367161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:41:20.367161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:20.367161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:41:20.367161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:41:20.367161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:41:20.367161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:41:20.367161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:41:20.367161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:41:20.367161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:20.367161 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:20.367383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:20.367383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:41:20.367383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:20.367383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:20.367383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:20.367383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:41:20.367383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:20.367383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:41:20.367383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:41:20.367383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:20.367383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:41:20.367383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:20.367383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:41:20.367383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:41:20.367651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:20.367651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:41:20.367651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:20.367651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:41:20.368243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:20.368243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:41:20.368243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:41:20.368243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:41:20.368243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:41:20.368243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:41:20.368243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:20.368243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:41:20.368243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:41:20.368243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:41:20.368243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:41:20.368243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:41:20.368243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:20.368243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:20.368243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:41:20.368243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:41:20.368243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:41:20.368710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:41:20.368710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:41:20.368710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:41:20.368710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:20.368710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:20.368710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:20.368710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:41:20.368710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:20.368710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:20.368710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:20.368710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:41:20.368710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:20.368710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:41:20.368710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:41:20.368710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:20.368710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:41:20.368710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:20.368710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:41:20.368710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:41:20.369220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:20.369220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:41:20.369220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:20.369220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:41:20.369220 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:41:20.369220 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:41:20.369220 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:41:20.369220 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:41:20.370202 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:41:20.370764 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:41:20.370987 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:20.371708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:41:20.371708 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:20.371708 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:41:20.371708 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:41:20.371708 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:20.371708 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:41:20.371708 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:20.371708 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:41:20.372519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:20.372519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:41:20.372519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:20.372519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:41:20.372519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:41:20.372519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:41:20.372519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:41:20.372519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:20.372519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:20.372519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:20.372519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:41:20.372519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:20.372867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:20.372867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:20.372867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:41:20.372867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:20.372867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:41:20.372867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:41:20.372867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:20.372867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:41:20.372867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:20.372867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:41:20.372867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:41:20.372867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:20.372867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:41:20.373207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:20.373207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:41:20.373461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:20.373461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:41:20.373461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:41:20.373461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:41:20.373461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:41:20.373461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:41:20.373461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:20.373461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:41:20.373461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:41:20.373461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:41:20.373461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:41:20.373461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:41:20.373461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:20.373461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:20.373461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:41:20.373461 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:41:20.373900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:41:20.373900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:41:20.373900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:20.373900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:20.373900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:20.373900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:41:20.373900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:20.373900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:20.373900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:20.373900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:41:20.373900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:20.373900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:41:20.373900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:41:20.373900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:20.373900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:41:20.373900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:20.373900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:41:20.373900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:41:20.373900 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:20.374443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:41:20.374443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:20.374443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:41:25.115072 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:41:25.116220 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:41:25.116220 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:41:25.117167 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:41:25.123670 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG dbcounter [-] [105581] Writing DB stats neutron:SELECT=241 {{(pid=105581) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 14:41:25.153781 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:41:25.154242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:25.155212 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:41:25.155212 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:25.155212 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:41:25.155212 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:41:25.155212 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:25.155212 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:41:25.155212 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:25.155212 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:41:25.156562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:25.156562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:41:25.156562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:25.156562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:41:25.156562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:41:25.156562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:41:25.156562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:41:25.156562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:25.156562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:25.156562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:25.156562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:41:25.156562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:25.157208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:25.157208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:25.157208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:41:25.157208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:25.157208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:41:25.157208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:41:25.157208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:25.157208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:41:25.157208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:25.157208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:41:25.157208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:41:25.157208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:25.157208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:41:25.157864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:25.157864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:41:25.157864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:25.157864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:41:25.157864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:41:25.157864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:41:25.157864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:41:25.157864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:41:25.157864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:25.157864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:41:25.157864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:41:25.157864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:41:25.157864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:41:25.157864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:41:25.157864 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:25.158725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:25.158725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:41:25.158725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:41:25.158725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:41:25.158725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:41:25.158725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:25.158725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:25.158725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:25.158725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:41:25.158725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:25.158725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:25.158725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:25.158725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:41:25.158725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:25.158725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:41:25.158725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:41:25.158725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:25.158725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:41:25.158725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:25.160077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:41:25.160077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:41:25.160077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:25.160077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:41:25.160077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:25.160077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:41:25.346698 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:41:25.347109 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:41:25.347314 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:41:25.348004 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:41:25.348277 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:41:25.348585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:25.349320 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:41:25.349320 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:25.349320 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:41:25.349320 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:41:25.349320 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:25.349320 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:41:25.349320 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:25.349320 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:41:25.351415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:25.351415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:41:25.351415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:25.351415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:41:25.351415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:41:25.351415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:41:25.351415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:41:25.351415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:25.351415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:25.351415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:25.351415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:41:25.351415 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:25.352840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:25.352840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:25.352840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:41:25.352840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:25.352840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:41:25.352840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:41:25.352840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:25.352840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:41:25.352840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:25.352840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:41:25.352840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:41:25.352840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:25.352840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:41:25.353729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:25.353729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:41:25.353729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:25.353729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:41:25.353729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:41:25.353729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:41:25.353729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:41:25.353729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:41:25.353729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:25.353729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:41:25.353729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:41:25.353729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:41:25.353729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:41:25.353729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:41:25.353729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:25.354443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:25.354443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:41:25.354443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:41:25.354443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:41:25.354443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:41:25.354443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:25.354443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:25.354443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:25.354443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:41:25.354443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:25.354443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:25.354443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:25.354443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:41:25.354443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:25.354443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:41:25.354443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:41:25.354443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:25.354443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:41:25.354443 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:25.355223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:41:25.355223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:41:25.355223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:25.355223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:41:25.355223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:25.355223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:41:25.355223 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:41:25.355223 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:41:25.355223 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:41:25.355223 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:41:25.356165 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:41:25.356165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:25.356165 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:41:25.356165 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:25.356165 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:41:25.356165 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:41:25.356165 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:25.356165 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:41:25.356165 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:25.356165 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:41:25.356848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:25.356848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:41:25.356848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:41:25.356848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:41:25.356848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:25.356848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:41:25.356848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:25.356848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:41:25.356848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:25.356848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:41:25.356848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:25.356848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:41:25.356848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:25.356848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:41:25.356848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:41:25.357928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:25.357928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:41:25.357928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:25.357928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:41:25.357928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:41:25.357928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:25.357928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:41:25.357928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:25.357928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:41:25.357928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:25.357928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:41:25.357928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:25.357928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:41:25.357928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:41:25.358386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:41:25.358386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:41:25.358386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:41:25.358386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:41:25.358386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:41:25.358386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:25.358386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:25.358386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:25.358386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:41:25.358386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:25.358386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:25.358386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:25.358386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:41:25.358753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:25.358753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:41:25.358753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:41:25.358753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:25.358753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:41:25.358753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:25.358753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:41:25.358753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:41:25.358753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:25.358753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:41:25.358753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:25.358753 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:41:25.359054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:25.359054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:41:25.359054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:41:25.359054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:41:25.359054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:41:25.359054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:41:25.359054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:25.359054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:41:25.359054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:41:25.359054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:41:25.359054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:41:25.359054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:41:25.359054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:25.359054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:25.359054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:41:25.359054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:41:25.359473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:41:25.359473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:41:25.359473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:41:25.359473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:41:25.359473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:41:25.359473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:25.359473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:25.359473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:25.359473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:41:25.359473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:25.359473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:25.359473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:25.359473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:41:25.359473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:25.359473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:41:25.359473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:41:25.359473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:25.359473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:41:25.360731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:25.360731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:41:25.360731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:41:25.360731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:25.360731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:41:25.360731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:25.360731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:41:25.363761 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:41:25.364043 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:41:25.364197 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:41:25.364302 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:41:25.364684 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:41:25.364903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:25.365505 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:41:25.365505 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:25.365505 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:41:25.365505 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:41:25.365505 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:25.365505 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:41:25.365505 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:25.365505 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:41:25.366122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:25.366122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:41:25.366122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:41:25.366122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:41:25.366122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:25.366122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:41:25.366122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:25.366122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:41:25.366122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:25.366122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:41:25.366122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:25.366122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:41:25.366122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:25.366122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:41:25.366122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:41:25.366634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:25.366634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:41:25.366634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:25.366634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:41:25.366634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:41:25.366634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:25.366634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:41:25.366634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:25.366634 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:41:25.366915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:25.366915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:41:25.366915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:25.366915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:41:25.366915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:41:25.366915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:41:25.366915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:41:25.366915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:41:25.366915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:41:25.366915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:25.366915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:25.367451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:25.367451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:41:25.367451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:25.367451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:25.367451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:25.367451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:41:25.367451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:25.367451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:41:25.367451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:41:25.367451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:25.367451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:41:25.367451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:25.367451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:41:25.367451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:41:25.367796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:25.367796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:41:25.367796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:25.367796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:41:25.368062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:25.368062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:41:25.368062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:41:25.368062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:41:25.368062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:41:25.368062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:41:25.368062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:25.368062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:41:25.368062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:41:25.368062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:41:25.368062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:41:25.368062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:41:25.368062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:25.368062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:25.368062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:41:25.368062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:41:25.368062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:41:25.368880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:41:25.368880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:41:25.368880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:41:25.368880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:25.368880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:25.368880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:25.368880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:41:25.368880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:25.368880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:25.368880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:25.368880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:41:25.368880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:25.368880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:41:25.368880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:41:25.368880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:25.368880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:41:25.368880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:25.368880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:41:25.368880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:41:25.369282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:25.369282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:41:25.369282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:25.369282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:41:25.369282 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:41:25.369282 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:41:25.369423 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:41:25.369658 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:41:25.370024 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:41:25.370284 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:41:25.370452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:25.370945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:41:25.370945 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:25.370945 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:41:25.370945 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:41:25.370945 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:25.370945 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:41:25.370945 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:25.370945 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:41:25.371676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:25.371676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:41:25.371676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:25.371676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:41:25.371676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:41:25.371676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:41:25.371676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:41:25.371676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:25.371676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:25.371676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:25.371676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:41:25.371676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:25.372241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:25.372241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:25.372241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:41:25.372241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:25.372241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:41:25.372241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:41:25.372241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:25.372241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:41:25.372241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:25.372241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:41:25.372241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:41:25.372241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:25.372241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:41:25.372483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:25.372483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:41:25.372483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:25.372483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:41:25.372483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:41:25.372483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:41:25.372483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:41:25.372483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:41:25.372483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:25.372483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:41:25.372483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:41:25.372483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:41:25.372483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:41:25.372483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:41:25.372483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:25.373098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:25.373098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:41:25.373098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:41:25.373098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:41:25.373098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:41:25.373098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:25.373098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:25.373098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:25.373098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:41:25.373098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:25.373098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:25.373098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:25.373098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:41:25.373098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:25.373098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:41:25.373098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:41:25.373098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:25.373098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:41:25.373098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:25.373711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:41:25.373711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:41:25.373711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:25.373711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:41:25.373711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:25.373711 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:41:30.115796 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:41:30.117084 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:41:30.117084 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:41:30.118179 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:41:30.152672 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:41:30.153002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:30.153465 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:41:30.153465 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:30.153465 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:41:30.153465 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:41:30.153465 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:30.153465 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:41:30.153465 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:30.153465 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:41:30.154672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:30.154672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:41:30.154672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:30.154672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:41:30.154672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:41:30.154672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:41:30.154672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:41:30.154672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:30.154672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:30.154672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:30.154672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:41:30.154672 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:30.156360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:30.156360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:30.156360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:41:30.156360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:30.156360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:41:30.156360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:41:30.156360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:30.156360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:41:30.156360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:30.156360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:41:30.156360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:41:30.156360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:30.156360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:41:30.157351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:30.157351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:41:30.157351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:30.157351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:41:30.157351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:41:30.157351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:41:30.157351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:41:30.157351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:41:30.157351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:30.157351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:41:30.157351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:41:30.157351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:41:30.157351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:41:30.157351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:41:30.157351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:30.158802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:30.158802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:41:30.158802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:41:30.158802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:41:30.158802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:41:30.158802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:30.158802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:30.158802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:30.158802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:41:30.158802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:30.158802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:30.158802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:30.158802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:41:30.158802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:30.158802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:41:30.158802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:41:30.158802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:30.158802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:41:30.158802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:30.160292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:41:30.160292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:41:30.160292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:30.160292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:41:30.160292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:30.160292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:41:30.347515 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:41:30.351479 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:41:30.351479 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:41:30.351479 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:41:30.351479 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:41:30.351479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:30.351479 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:41:30.351479 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:30.351479 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:41:30.351479 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:41:30.352228 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:30.352228 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:41:30.352228 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:30.352228 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:41:30.352584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:30.352584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:41:30.352584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:30.352584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:41:30.352584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:41:30.352584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:41:30.352584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:41:30.352584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:30.352584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:30.352584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:30.352584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:41:30.352584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:30.353460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:30.353460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:30.353460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:41:30.353460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:30.353460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:41:30.353460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:41:30.353460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:30.353460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:41:30.353460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:30.353460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:41:30.353460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:41:30.353460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:30.353460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:41:30.354395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:30.354395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:41:30.354789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:30.354789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:41:30.354789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:41:30.354789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:41:30.354789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:41:30.354789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:41:30.354789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:30.354789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:41:30.354789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:41:30.354789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:41:30.354789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:41:30.354789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:41:30.354789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:30.354789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:30.354789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:41:30.354789 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:41:30.355743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:41:30.355743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:41:30.355743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:30.355743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:30.355743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:30.355743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:41:30.355743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:30.355743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:30.355743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:30.355743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:41:30.355743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:30.355743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:41:30.355743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:41:30.355743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:30.355743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:41:30.355743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:30.355743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:41:30.355743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:41:30.355743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:30.356276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:41:30.356276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:30.356276 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:41:30.356276 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:41:30.356276 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:41:30.356276 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:41:30.356276 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:41:30.356694 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:41:30.358323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:30.358323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:41:30.358323 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:30.358323 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:41:30.358323 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:41:30.358323 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:30.358323 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:41:30.358323 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:30.358323 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:41:30.358323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:30.358323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:41:30.358323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:41:30.358323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:41:30.358323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:30.358323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:41:30.358323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:30.358323 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:41:30.358784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:30.358784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:41:30.358784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:30.358784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:41:30.358784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:30.358784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:41:30.358784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:41:30.358784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:30.358784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:41:30.358784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:30.358784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:41:30.358784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:41:30.358784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:30.358784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:41:30.358784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:30.358784 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:41:30.360298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:30.360298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:41:30.360298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:30.360298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:41:30.360298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:41:30.360298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:41:30.360298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:41:30.360298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:41:30.360298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:41:30.360298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:41:30.360298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:30.360298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:30.360685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:30.360685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:41:30.360685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:30.360685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:30.360685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:30.360685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:41:30.360685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:30.360685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:41:30.360685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:41:30.360685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:30.360685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:41:30.360685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:30.360685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:41:30.360685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:41:30.361741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:30.361741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:41:30.361741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:30.361741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:41:30.362157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:30.362157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:41:30.362157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:41:30.362157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:41:30.362157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:41:30.362157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:41:30.362157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:30.362157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:41:30.362157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:41:30.362157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:41:30.362157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:41:30.362157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:41:30.362157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:30.362157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:30.362157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:41:30.362157 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:41:30.362487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:41:30.362487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:41:30.362487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:41:30.362487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:41:30.362487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:41:30.362487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:30.362487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:30.362487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:30.362487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:41:30.362487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:30.362487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:30.362487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:30.362487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:41:30.362487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:30.362487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:41:30.362487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:41:30.362487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:30.362487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:41:30.362873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:30.362873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:41:30.362873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:41:30.362873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:30.362873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:41:30.362873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:30.362873 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:41:30.364183 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:41:30.364550 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:41:30.364645 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:41:30.364788 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:41:30.365352 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:41:30.365661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:30.366096 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:41:30.366096 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:30.366096 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:41:30.366096 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:41:30.366096 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:30.366096 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:41:30.366096 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:30.366096 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:41:30.366818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:30.366818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:41:30.366818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:41:30.366818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:41:30.366818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:30.366818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:41:30.366818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:30.366818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:41:30.366818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:30.366818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:41:30.366818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:30.366818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:41:30.366818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:30.366818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:41:30.366818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:41:30.367114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:30.367114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:41:30.367114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:30.367114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:41:30.367114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:41:30.367114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:30.367114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:41:30.367114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:30.367114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:41:30.367671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:30.367671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:41:30.367671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:30.367671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:41:30.367671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:41:30.367671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:41:30.367671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:41:30.367671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:41:30.367671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:41:30.367671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:30.367671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:30.367916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:30.367916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:41:30.367916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:30.367916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:30.367916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:30.367916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:41:30.367916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:30.367916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:41:30.367916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:41:30.367916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:30.367916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:41:30.367916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:30.367916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:41:30.367916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:41:30.368192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:30.368192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:41:30.368192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:30.368192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:41:30.368641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:30.368641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:41:30.368641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:41:30.368641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:41:30.368641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:41:30.368641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:41:30.368641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:30.368641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:41:30.368641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:41:30.368641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:41:30.368641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:41:30.368641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:41:30.368641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:30.368641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:30.368641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:41:30.368641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:41:30.368641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:41:30.369041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:41:30.369041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:41:30.369041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:41:30.369041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:30.369041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:30.369041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:30.369041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:41:30.369041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:30.369041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:30.369041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:30.369041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:41:30.369041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:30.369041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:41:30.369041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:41:30.369041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:30.369041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:41:30.369041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:30.369041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:41:30.369041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:41:30.369383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:30.369383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:41:30.369383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:30.369383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:41:30.369383 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:41:30.369703 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:41:30.369811 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:41:30.369952 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:41:30.370243 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:41:30.370486 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:41:30.370631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:30.371026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:41:30.371026 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:30.371026 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:41:30.371026 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:41:30.371026 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:30.371026 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:41:30.371026 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:30.371026 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:41:30.371719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:30.371719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:41:30.371719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:30.371719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:41:30.371719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:41:30.371719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:41:30.371719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:41:30.371719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:30.371719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:30.371719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:30.371719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:41:30.371719 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:30.371994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:30.371994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:30.371994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:41:30.371994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:30.371994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:41:30.371994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:41:30.371994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:30.371994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:41:30.371994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:30.371994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:41:30.371994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:41:30.371994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:30.371994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:41:30.372256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:30.372256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:41:30.373844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:30.373844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:41:30.373844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:41:30.373844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:41:30.373844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:41:30.373844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:41:30.373844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:30.373844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:41:30.373844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:41:30.373844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:41:30.373844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:41:30.373844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:41:30.373844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:30.373844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:30.373844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:41:30.373844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:41:30.375348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:41:30.375348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:41:30.375348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:30.375348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:30.375348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:30.375348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:41:30.375348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:30.375348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:30.375348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:30.375348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:41:30.375348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:30.375348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:41:30.375348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:41:30.375348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:30.375348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:41:30.375348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:30.375348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:41:30.375348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:41:30.375348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:30.376358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:41:30.376358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:30.376358 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:41:35.116369 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:41:35.117103 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:41:35.117103 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:41:35.118268 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:41:35.154731 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:41:35.154899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:35.156293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:41:35.156293 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:35.156293 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:41:35.156293 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:41:35.156293 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:35.156293 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:41:35.156293 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:35.156293 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:41:35.157081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:35.157081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:41:35.157081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:35.157081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:41:35.157081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:41:35.157081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:41:35.157081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:41:35.157081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:35.157081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:35.157081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:35.157081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:41:35.157081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:35.157547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:35.157547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:35.157547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:41:35.157547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:35.157547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:41:35.157547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:41:35.157547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:35.157547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:41:35.157547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:35.157547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:41:35.157547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:41:35.157547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:35.157547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:41:35.158059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:35.158059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:41:35.158194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:35.158194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:41:35.158194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:41:35.158194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:41:35.158194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:41:35.158194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:41:35.158194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:35.158194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:41:35.158194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:41:35.158194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:41:35.158194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:41:35.158194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:41:35.158194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:35.158194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:35.158194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:41:35.158194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:41:35.158667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:41:35.158667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:41:35.158667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:35.158667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:35.158667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:35.158667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:41:35.158667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:35.158667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:35.158667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:35.158667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:41:35.158667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:35.158667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:41:35.158667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:41:35.158667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:35.158667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:41:35.158667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:35.158667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:41:35.158667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:41:35.158667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:35.159233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:41:35.159233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:35.159233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:41:35.348390 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:41:35.349753 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:41:35.349753 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:41:35.350378 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:41:35.350721 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:41:35.351162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:35.351857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:41:35.351857 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:35.351857 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:41:35.351857 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:41:35.351857 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:35.351857 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:41:35.351857 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:35.351857 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:41:35.353244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:35.353244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:41:35.353244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:35.353244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:41:35.353244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:41:35.353244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:41:35.353244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:41:35.353244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:35.353244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:35.353244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:35.353244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:41:35.353244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:35.353896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:35.353896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:35.353896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:41:35.353896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:35.353896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:41:35.353896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:41:35.353896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:35.353896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:41:35.353896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:35.353896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:41:35.353896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:41:35.353896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:35.353896 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:41:35.354476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:35.354476 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:41:35.354886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:35.354886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:41:35.354886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:41:35.354886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:41:35.354886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:41:35.354886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:41:35.354886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:35.354886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:41:35.354886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:41:35.354886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:41:35.354886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:41:35.354886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:41:35.354886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:35.354886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:35.354886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:41:35.354886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:41:35.355983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:41:35.355983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:41:35.355983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:35.355983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:35.355983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:35.355983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:41:35.355983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:35.355983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:35.355983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:35.355983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:41:35.355983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:35.355983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:41:35.355983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:41:35.355983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:35.355983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:41:35.355983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:35.355983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:41:35.355983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:41:35.355983 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:35.356911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:41:35.356911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:35.356911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:41:35.356911 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:41:35.356911 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:41:35.356911 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:41:35.356911 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:41:35.357284 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:41:35.357349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:35.358034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:41:35.358034 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:35.358034 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:41:35.358034 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:41:35.358034 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:35.358034 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:41:35.358034 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:35.358034 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:41:35.358738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:35.358738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:41:35.358738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:41:35.358738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:41:35.358738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:35.358738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:41:35.358738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:35.358738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:41:35.358738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:35.358738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:41:35.358738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:35.358738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:41:35.358738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:35.358738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:41:35.358738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:41:35.359579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:35.359579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:41:35.359579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:35.359579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:41:35.359579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:41:35.359579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:35.359579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:41:35.359579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:35.359579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:41:35.359579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:35.359579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:41:35.359579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:35.359579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:41:35.359579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:41:35.360011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:41:35.360011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:41:35.360011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:41:35.360011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:41:35.360011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:41:35.360011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:35.360011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:35.360011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:35.360011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:41:35.360011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:35.360011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:35.360011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:35.360011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:41:35.360375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:35.360375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:41:35.360375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:41:35.360375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:35.360375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:41:35.360375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:35.360375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:41:35.360375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:41:35.360375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:35.360375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:41:35.360375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:35.360375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:41:35.360720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:35.360720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:41:35.360720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:41:35.360720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:41:35.360720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:41:35.360720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:41:35.360720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:35.360720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:41:35.360720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:41:35.360720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:41:35.360720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:41:35.360720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:41:35.360720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:35.360720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:35.360720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:41:35.360720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:41:35.361148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:41:35.361148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:41:35.361148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:41:35.361148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:41:35.361148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:41:35.361148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:35.361148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:35.361148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:35.361148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:41:35.361148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:35.361148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:35.361148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:35.361148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:41:35.361148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:35.361148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:41:35.361148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:41:35.361148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:35.361148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:41:35.361802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:35.361802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:41:35.361802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:41:35.361802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:35.361802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:41:35.361802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:35.361802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:41:35.364688 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:41:35.364955 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:41:35.365059 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:41:35.365181 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:41:35.365630 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:41:35.365986 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:35.366683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:41:35.366683 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:35.366683 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:41:35.366683 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:41:35.366683 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:35.366683 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:41:35.366683 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:35.366683 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:41:35.367378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:35.367378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:41:35.367378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:41:35.367378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:41:35.367378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:35.367378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:41:35.367378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:35.367378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:41:35.367378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:35.367378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:41:35.367378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:35.367378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:41:35.367378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:35.367378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:41:35.367378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:41:35.367807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:35.367807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:41:35.367807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:35.367807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:41:35.367807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:41:35.367807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:35.367807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:41:35.367807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:35.367807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:41:35.368220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:35.368220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:41:35.368220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:35.368220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:41:35.368220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:41:35.368220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:41:35.368220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:41:35.368220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:41:35.368220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:41:35.368220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:35.368220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:35.368558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:35.368558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:41:35.368558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:35.368558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:35.368558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:35.368558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:41:35.368558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:35.368558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:41:35.368558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:41:35.368558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:35.368558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:41:35.368558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:35.368558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:41:35.368558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:41:35.368907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:35.368907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:41:35.368907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:35.368907 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:41:35.369240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:35.369240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:41:35.369240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:41:35.369240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:41:35.369240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:41:35.369240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:41:35.369240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:35.369240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:41:35.369240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:41:35.369240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:41:35.369240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:41:35.369240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:41:35.369240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:35.369240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:35.369240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:41:35.369240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:41:35.369240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:41:35.369695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:41:35.369695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:41:35.369695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:41:35.369695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:35.369695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:35.369695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:35.369695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:41:35.369695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:35.369695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:35.369695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:35.369695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:41:35.369695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:35.369695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:41:35.369695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:41:35.369695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:35.369695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:41:35.369695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:35.369695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:41:35.369695 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:41:35.370211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:35.370211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:41:35.370211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:35.370211 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:41:35.370211 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:41:35.370211 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:41:35.370397 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:41:35.370397 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:41:35.370782 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:41:35.371160 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:41:35.371377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:35.372012 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:41:35.372012 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:35.372012 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:41:35.372012 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:41:35.372012 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:35.372012 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:41:35.372012 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:35.372012 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:41:35.372802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:35.372802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:41:35.372802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:35.372802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:41:35.372802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:41:35.372802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:41:35.372802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:41:35.372802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:35.372802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:35.372802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:35.372802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:41:35.372802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:35.373153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:35.373153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:35.373153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:41:35.373153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:35.373153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:41:35.373153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:41:35.373153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:35.373153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:41:35.373153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:35.373153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:41:35.373153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:41:35.373153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:35.373153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:41:35.373527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:35.373527 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:41:35.373745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:35.373745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:41:35.373745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:41:35.373745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:41:35.373745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:41:35.373745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:41:35.373745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:35.373745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:41:35.373745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:41:35.373745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:41:35.373745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:41:35.373745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:41:35.373745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:35.373745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:35.373745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:41:35.373745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:41:35.374176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:41:35.374176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:41:35.374176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:35.374176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:35.374176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:35.374176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:41:35.374176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:35.374176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:35.374176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:35.374176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:41:35.374176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:35.374176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:41:35.374176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:41:35.374176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:35.374176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:41:35.374176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:35.374176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:41:35.374176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:41:35.374176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:35.374698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:41:35.374698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:35.374698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:41:40.117165 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:41:40.118120 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:41:40.118120 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:41:40.118920 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:41:40.154794 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:41:40.154794 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:40.154978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:41:40.154978 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:40.154978 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:41:40.154978 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:41:40.154978 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:40.154978 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:41:40.154978 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:40.154978 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:41:40.155889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:40.155889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:41:40.155889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:40.155889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:41:40.155889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:41:40.155889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:41:40.155889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:41:40.155889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:40.155889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:40.155889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:40.155889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:41:40.155889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:40.156231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:40.156231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:40.156231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:41:40.156231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:40.156231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:41:40.156231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:41:40.156231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:40.156231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:41:40.156231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:40.156231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:41:40.156231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:41:40.156231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:40.156231 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:41:40.156563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:40.156563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:41:40.157887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:40.157887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:41:40.157887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:41:40.157887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:41:40.157887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:41:40.157887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:41:40.157887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:40.157887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:41:40.157887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:41:40.157887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:41:40.157887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:41:40.157887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:41:40.157887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:40.157887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:40.157887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:41:40.157887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:41:40.158241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:41:40.158241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:41:40.158241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:40.158241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:40.158241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:40.158241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:41:40.158241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:40.158241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:40.158241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:40.158241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:41:40.158241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:40.158241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:41:40.158241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:41:40.158241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:40.158241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:41:40.158241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:40.158241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:41:40.158241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:41:40.158241 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:40.158730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:41:40.158730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:40.158730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:41:40.349146 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:41:40.349907 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:41:40.350557 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:41:40.352595 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:41:40.352595 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:41:40.352595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:40.353039 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:41:40.353039 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:40.353039 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:41:40.353039 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:41:40.353039 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:40.353039 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:41:40.353039 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:40.353039 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:41:40.355126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:40.355126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:41:40.355126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:40.355126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:41:40.355126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:41:40.355126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:41:40.355126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:41:40.355126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:40.355126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:40.355126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:40.355126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:41:40.355126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:40.356083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:40.356083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:40.356083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:41:40.356083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:40.356083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:41:40.356083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:41:40.356083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:40.356083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:41:40.356083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:40.356083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:41:40.356083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:41:40.356083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:40.356083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:41:40.357123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:40.357123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:41:40.358292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:40.358292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:41:40.358292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:41:40.358292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:41:40.358292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:41:40.358292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:41:40.358292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:40.358292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:41:40.358292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:41:40.358292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:41:40.358292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:41:40.358292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:41:40.358292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:40.358292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:40.358292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:41:40.358292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:41:40.359035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:41:40.359035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:41:40.359035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:40.359035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:40.359035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:40.359035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:41:40.359035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:40.359035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:40.359035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:40.359035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:41:40.359035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:40.359035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:41:40.359035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:41:40.359035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:40.359035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:41:40.359035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:40.359035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:41:40.359035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:41:40.359035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:40.359588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:41:40.359588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:40.359588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:41:40.359588 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:41:40.359588 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:41:40.359588 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:41:40.359588 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:41:40.360372 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:41:40.360691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:40.361544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:41:40.361544 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:40.361544 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:41:40.361544 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:41:40.361544 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:40.361544 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:41:40.361544 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:40.361544 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:41:40.362577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:40.362577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:41:40.362577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:41:40.362577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:41:40.362577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:40.362577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:41:40.362577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:40.362577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:41:40.362577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:40.362577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:41:40.362577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:40.362577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:41:40.362577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:40.362577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:41:40.362577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:41:40.363315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:40.363315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:41:40.363315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:40.363315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:41:40.363315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:41:40.363315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:40.363315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:41:40.363315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:40.363315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:41:40.363799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:40.363799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:41:40.363799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:40.363799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:41:40.363799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:41:40.363799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:41:40.363799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:41:40.363799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:41:40.363799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:41:40.363799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:41:40.363799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:40.363799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:40.364344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:40.364344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:41:40.364344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:40.364344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:40.364344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:40.364344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:41:40.364344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:40.364344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:41:40.364344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:41:40.364344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:40.364344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:41:40.364344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:40.364344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:41:40.364344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:41:40.365060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:40.365060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:41:40.365060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:40.365060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:41:40.365060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:40.365060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:41:40.365060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:41:40.365060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:41:40.365060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:41:40.365060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:41:40.365060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:40.365060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:41:40.365060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:41:40.365060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:41:40.366050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:41:40.366050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:41:40.366050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:40.366050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:40.366050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:41:40.366050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:41:40.366050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:41:40.366050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:41:40.366050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:41:40.366050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:41:40.366050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:41:40.366050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:40.366050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:40.366050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:40.366050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:41:40.366050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:40.366050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:40.366050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:40.366050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:41:40.367835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:40.367835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:41:40.367835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:41:40.367835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:40.367835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:41:40.367835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:40.367835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:41:40.367835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:41:40.367835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:40.367835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:41:40.367835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:40.367835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:41:40.367835 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:41:40.367835 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:41:40.368300 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:41:40.368300 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:41:40.368300 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:41:40.368300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:40.368300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:41:40.368300 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:40.368300 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:41:40.368300 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:41:40.368300 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:40.368300 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:41:40.368300 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:40.368300 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:41:40.368713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:40.368713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:41:40.368713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:41:40.368713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:41:40.368713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:40.368713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:41:40.368713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:40.368713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:41:40.368713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:40.368713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:41:40.368713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:40.368713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:41:40.368713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:40.368713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:41:40.368713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:41:40.369735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:40.369735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:41:40.369735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:40.369735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:41:40.369735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:41:40.369735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:40.369735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:41:40.369735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:40.369735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:41:40.369735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:40.369735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:41:40.369735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:40.369735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:41:40.369735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:41:40.370228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:41:40.370228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:41:40.370228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:41:40.370228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:41:40.370228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:40.370228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:40.370228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:40.370228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:41:40.370228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:40.370228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:40.370228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:40.370228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:41:40.370228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:40.371079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:41:40.371079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:41:40.371079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:40.371079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:41:40.371079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:40.371079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:41:40.371079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:41:40.371079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:40.371079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:41:40.371079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:40.371079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:41:40.371079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:40.371370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:41:40.371370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:41:40.371370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:41:40.371370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:41:40.371370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:41:40.371370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:40.371370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:41:40.371370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:41:40.371370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:41:40.371370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:41:40.371370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:41:40.371370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:40.371370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:40.371370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:41:40.371370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:41:40.371370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:41:40.371370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:41:40.371370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:41:40.371370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:41:40.371766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:40.371766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:40.371766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:40.371766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:41:40.371766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:40.371766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:40.371766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:40.371766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:41:40.371766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:40.371766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:41:40.371766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:41:40.371766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:40.371766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:41:40.371766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:40.371766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:41:40.371766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:41:40.371766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:40.371766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:41:40.371766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:40.371766 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:41:40.372154 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:41:40.372154 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:41:40.372154 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:41:40.372154 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:41:40.372967 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:41:40.372967 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:41:40.372967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:40.373127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:41:40.373127 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:40.373127 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:41:40.373127 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:41:40.373127 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:40.373127 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:41:40.373127 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:40.373127 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:41:40.373881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:40.373881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:41:40.373881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:40.373881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:41:40.373881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:41:40.373881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:41:40.373881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:41:40.373881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:40.373881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:40.373881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:40.373881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:41:40.373881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:40.374266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:40.374266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:40.374266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:41:40.374266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:40.374266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:41:40.374266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:41:40.374266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:40.374266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:41:40.374266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:40.374266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:41:40.374266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:41:40.374266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:40.374266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:41:40.374698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:40.374698 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:41:40.374867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:40.374867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:41:40.374867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:41:40.374867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:41:40.374867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:41:40.374867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:41:40.374867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:40.374867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:41:40.374867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:41:40.374867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:41:40.374867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:41:40.374867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:41:40.374867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:40.374867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:40.374867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:41:40.374867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:41:40.375233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:41:40.375233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:41:40.375233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:40.375233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:40.375233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:40.375233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:41:40.375233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:40.375233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:40.375233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:40.375233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:41:40.375233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:40.375233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:41:40.375233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:41:40.375233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:40.375233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:41:40.375233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:40.375233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:41:40.375233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:41:40.375233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:40.375636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:41:40.375636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:40.375636 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:41:45.117913 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:41:45.118588 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:41:45.118588 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:41:45.119875 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:41:45.153599 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:41:45.153599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:45.154306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:41:45.154306 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:45.154306 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:41:45.154306 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:41:45.154306 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:45.154306 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:41:45.154306 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:45.154306 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:41:45.156557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:45.156557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:41:45.156557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:45.156557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:41:45.156557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:41:45.156557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:41:45.156557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:41:45.156557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:45.156557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:45.156557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:45.156557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:41:45.156557 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:45.156996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:45.156996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:45.156996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:41:45.156996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:45.156996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:41:45.156996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:41:45.156996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:45.156996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:41:45.156996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:45.156996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:41:45.156996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:41:45.156996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:45.156996 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:41:45.157355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:45.157355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:41:45.159475 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:45.159475 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:41:45.159475 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:41:45.159475 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:41:45.159475 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:41:45.159475 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:41:45.159475 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:45.159475 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:41:45.159475 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:41:45.159475 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:41:45.159475 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:41:45.159475 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:41:45.159475 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:45.159475 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:45.159475 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:41:45.159475 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:41:45.159950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:41:45.159950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:41:45.159950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:45.159950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:45.159950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:45.159950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:41:45.159950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:45.159950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:45.159950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:45.159950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:41:45.159950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:45.159950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:41:45.159950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:41:45.159950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:45.159950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:41:45.159950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:45.159950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:41:45.159950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:41:45.159950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:45.160595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:41:45.160595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:45.160595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:41:45.349979 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:41:45.350526 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:41:45.350526 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:41:45.351359 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:41:45.351700 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:41:45.351928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:45.353559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:41:45.353559 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:45.353559 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:41:45.353559 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:41:45.353559 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:45.353559 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:41:45.353559 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:45.353559 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:41:45.354055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:45.354055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:41:45.354055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:45.354055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:41:45.354055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:41:45.354055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:41:45.354055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:41:45.354055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:45.354055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:45.354055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:45.354055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:41:45.354055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:45.354618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:45.354618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:45.354618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:41:45.354618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:45.354618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:41:45.354618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:41:45.354618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:45.354618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:41:45.354618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:45.354618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:41:45.354618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:41:45.354618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:45.354618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:41:45.355149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:45.355149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:41:45.355149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:45.355149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:41:45.355149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:41:45.355149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:41:45.355149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:41:45.355149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:41:45.355149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:45.355149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:41:45.355149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:41:45.355149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:41:45.355149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:41:45.355149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:41:45.355149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:45.355849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:45.355849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:41:45.355849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:41:45.355849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:41:45.355849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:41:45.355849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:45.355849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:45.355849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:45.355849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:41:45.355849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:45.355849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:45.355849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:45.355849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:41:45.355849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:45.355849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:41:45.355849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:41:45.355849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:45.355849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:41:45.355849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:45.356651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:41:45.356651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:41:45.356651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:45.356651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:41:45.356651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:45.356651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:41:45.358801 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:41:45.359339 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:41:45.359650 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:41:45.359926 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:41:45.360824 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:41:45.360824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:45.361257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:41:45.361257 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:45.361257 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:41:45.361257 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:41:45.361257 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:45.361257 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:41:45.361257 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:45.361257 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:41:45.362627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:45.362627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:41:45.362627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:41:45.362627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:41:45.362627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:45.362627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:41:45.362627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:45.362627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:41:45.362627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:45.362627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:41:45.362627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:45.362627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:41:45.362627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:45.362627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:41:45.362627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:41:45.363032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:45.363032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:41:45.363032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:45.363032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:41:45.363032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:41:45.363032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:45.363032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:41:45.363032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:45.363032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:41:45.363775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:45.363775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:41:45.363775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:45.363775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:41:45.363775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:41:45.363775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:41:45.363775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:41:45.363775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:41:45.363775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:41:45.363775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:41:45.363775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:45.363775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:45.364122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:45.364122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:41:45.364122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:45.364122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:45.364122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:45.364122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:41:45.364122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:45.364122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:41:45.364122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:41:45.364122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:45.364122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:41:45.364122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:45.364122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:41:45.364122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:41:45.364504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:45.364504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:41:45.364504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:45.364504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:41:45.365289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:45.365289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:41:45.365289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:41:45.365289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:41:45.365289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:41:45.365289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:41:45.365289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:45.365289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:41:45.365289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:41:45.365289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:41:45.365289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:41:45.365289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:41:45.365289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:45.365289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:45.365289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:41:45.365289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:41:45.365697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:41:45.365697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:41:45.365697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:41:45.365697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:41:45.365697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:41:45.365697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:45.365697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:45.365697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:45.365697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:41:45.365697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:45.365697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:45.365697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:45.365697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:41:45.365697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:45.365697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:41:45.365697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:41:45.365697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:45.365697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:41:45.366152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:45.366152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:41:45.366152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:41:45.366152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:45.366152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:41:45.366152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:45.366152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:41:45.366152 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:41:45.366152 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:41:45.366152 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:41:45.366472 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:41:45.367153 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:41:45.367619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:45.368522 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:41:45.368522 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:45.368522 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:41:45.368522 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:41:45.368522 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:45.368522 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:41:45.368522 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:45.368522 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:41:45.369559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:45.369559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:41:45.369559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:41:45.369559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:41:45.369559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:45.369559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:41:45.369559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:45.369559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:41:45.369559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:45.369559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:41:45.369559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:45.369559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:41:45.369559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:45.369559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:41:45.369559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:41:45.369999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:45.369999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:41:45.369999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:45.369999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:41:45.369999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:41:45.369999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:45.369999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:41:45.369999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:45.369999 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:41:45.370503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:45.370503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:41:45.370503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:45.370503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:41:45.370503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:41:45.370503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:41:45.370503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:41:45.370503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:41:45.370503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:41:45.370503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:45.370503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:45.370835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:45.370835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:41:45.370835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:45.370835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:45.370835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:45.370835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:41:45.370835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:45.370835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:41:45.370835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:41:45.370835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:45.370835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:41:45.370835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:45.370835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:41:45.370835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:41:45.371180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:45.371180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:41:45.371180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:45.371180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:41:45.371859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:45.371859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:41:45.371859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:41:45.371859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:41:45.371859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:41:45.371859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:41:45.371859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:45.371859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:41:45.371859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:41:45.371859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:41:45.371859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:41:45.371859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:41:45.371859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:45.371859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:45.371859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:41:45.371859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:41:45.371859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:41:45.372293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:41:45.372293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:41:45.372293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:41:45.372293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:45.372293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:45.372293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:45.372293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:41:45.372293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:45.372293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:45.372293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:45.372293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:41:45.372293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:45.372293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:41:45.372293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:41:45.372293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:45.372293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:41:45.372293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:45.372293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:41:45.372293 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:41:45.372790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:45.372790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:41:45.372790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:45.372790 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:41:45.372790 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:41:45.372790 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:41:45.372790 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:41:45.372790 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:41:45.373017 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:41:45.373418 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:41:45.373685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:45.374219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:41:45.374219 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:45.374219 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:41:45.374219 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:41:45.374219 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:45.374219 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:41:45.374219 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:45.374219 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:41:45.374966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:45.374966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:41:45.374966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:45.374966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:41:45.374966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:41:45.374966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:41:45.374966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:41:45.374966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:45.374966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:45.374966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:45.374966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:41:45.374966 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:45.375299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:45.375299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:45.375299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:41:45.375299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:45.375299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:41:45.375299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:41:45.375299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:45.375299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:41:45.375299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:45.375299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:41:45.375299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:41:45.375299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:45.375299 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:41:45.375651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:45.375651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:41:45.375816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:45.375816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:41:45.375816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:41:45.375816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:41:45.375816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:41:45.375816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:41:45.375816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:45.375816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:41:45.375816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:41:45.375816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:41:45.375816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:41:45.375816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:41:45.375816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:45.375816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:45.375816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:41:45.375816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:41:45.376236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:41:45.376236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:41:45.376236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:45.376236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:45.376236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:45.376236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:41:45.376236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:45.376236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:45.376236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:45.376236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:41:45.376236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:45.376236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:41:45.376236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:41:45.376236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:45.376236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:41:45.376236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:45.376236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:41:45.376236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:41:45.376236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:45.376728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:41:45.376728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:45.376728 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:41:50.118605 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:41:50.119552 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:41:50.119552 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:41:50.120450 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:41:50.155671 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:41:50.155947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:50.156857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:41:50.156857 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:50.156857 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:41:50.156857 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:41:50.156857 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:50.156857 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:41:50.156857 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:50.156857 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:41:50.157820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:50.157820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:41:50.157820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:50.157820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:41:50.157820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:41:50.157820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:41:50.157820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:41:50.157820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:50.157820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:50.157820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:50.157820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:41:50.157820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:50.158200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:50.158200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:50.158200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:41:50.158200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:50.158200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:41:50.158200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:41:50.158200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:50.158200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:41:50.158200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:50.158200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:41:50.158200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:41:50.158200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:50.158200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:41:50.158908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:50.158908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:41:50.158908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:50.158908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:41:50.158908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:41:50.158908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:41:50.158908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:41:50.158908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:41:50.158908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:50.158908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:41:50.158908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:41:50.158908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:41:50.158908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:41:50.158908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:41:50.158908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:50.159897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:50.159897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:41:50.159897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:41:50.159897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:41:50.159897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:41:50.159897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:50.159897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:50.159897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:50.159897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:41:50.159897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:50.159897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:50.159897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:50.159897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:41:50.159897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:50.159897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:41:50.159897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:41:50.159897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:50.159897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:41:50.159897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:50.160847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:41:50.160847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:41:50.160847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:50.160847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:41:50.160847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:50.160847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:41:50.350665 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:41:50.351447 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:41:50.351719 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:41:50.353154 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:41:50.353457 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:41:50.353718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:50.354400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:41:50.354400 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:50.354400 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:41:50.354400 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:41:50.354400 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:50.354400 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:41:50.354400 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:50.354400 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:41:50.355758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:50.355758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:41:50.355758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:50.355758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:41:50.355758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:41:50.355758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:41:50.355758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:41:50.355758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:50.355758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:50.355758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:50.355758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:41:50.355758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:50.356441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:50.356441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:50.356441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:41:50.356441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:50.356441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:41:50.356441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:41:50.356441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:50.356441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:41:50.356441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:50.356441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:41:50.356441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:41:50.356441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:50.356441 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:41:50.356867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:50.356867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:41:50.357089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:50.357089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:41:50.357089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:41:50.357089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:41:50.357089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:41:50.357089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:41:50.357089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:50.357089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:41:50.357089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:41:50.357089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:41:50.357089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:41:50.357089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:41:50.357089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:50.357089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:50.357089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:41:50.357089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:41:50.357609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:41:50.357609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:41:50.357609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:50.357609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:50.357609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:50.357609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:41:50.357609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:50.357609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:50.357609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:50.357609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:41:50.357609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:50.357609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:41:50.357609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:41:50.357609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:50.357609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:41:50.357609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:50.357609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:41:50.357609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:41:50.357609 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:50.358151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:41:50.358151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:50.358151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:41:50.359304 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:41:50.359606 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:41:50.359847 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:41:50.359847 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:41:50.360198 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:41:50.360460 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:50.360905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:41:50.360905 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:50.360905 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:41:50.360905 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:41:50.360905 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:50.360905 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:41:50.360905 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:50.360905 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:41:50.361621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:50.361621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:41:50.361621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:41:50.361621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:41:50.361621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:50.361621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:41:50.361621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:50.361621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:41:50.361621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:50.361621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:41:50.361621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:50.361621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:41:50.361621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:50.361621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:41:50.361621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:41:50.362150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:50.362150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:41:50.362150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:50.362150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:41:50.362150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:41:50.362150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:50.362150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:41:50.362150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:50.362150 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:41:50.362667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:50.362667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:41:50.362667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:50.362667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:41:50.362667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:41:50.362667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:41:50.362667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:41:50.362667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:41:50.362667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:41:50.362667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:41:50.362667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:50.362667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:50.363028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:50.363028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:41:50.363028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:50.363028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:50.363028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:50.363028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:41:50.363028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:50.363028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:41:50.363028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:41:50.363028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:50.363028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:41:50.363028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:50.363028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:41:50.363028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:41:50.363542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:50.363542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:41:50.363542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:50.363542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:41:50.363542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:50.363542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:41:50.363542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:41:50.363542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:41:50.363542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:41:50.363542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:41:50.363542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:50.363542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:41:50.363542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:41:50.363542 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:41:50.364109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:41:50.364109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:41:50.364109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:50.364109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:50.364109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:41:50.364109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:41:50.364109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:41:50.364109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:41:50.364109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:41:50.364109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:41:50.364109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:41:50.364109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:50.364109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:50.364109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:50.364109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:41:50.364109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:50.364109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:50.364109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:50.364109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:41:50.364683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:50.364683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:41:50.364683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:41:50.364683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:50.364683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:41:50.364683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:50.364683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:41:50.364683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:41:50.364683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:50.364683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:41:50.364683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:50.364683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:41:50.365936 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:41:50.366254 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:41:50.366315 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:41:50.366420 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:41:50.366779 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:41:50.366944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:50.367416 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:41:50.367416 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:50.367416 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:41:50.367416 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:41:50.367416 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:50.367416 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:41:50.367416 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:50.367416 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:41:50.367964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:50.367964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:41:50.367964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:41:50.367964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:41:50.367964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:50.367964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:41:50.367964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:50.367964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:41:50.367964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:50.367964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:41:50.367964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:50.367964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:41:50.367964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:50.367964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:41:50.367964 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:41:50.368345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:50.368345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:41:50.368345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:50.368345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:41:50.368345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:41:50.368345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:50.368345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:41:50.368345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:50.368345 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:41:50.368678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:50.368678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:41:50.368678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:50.368678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:41:50.368678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:41:50.368678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:41:50.368678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:41:50.368678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:41:50.368678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:41:50.368678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:50.368678 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:50.369023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:50.369023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:41:50.369023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:50.369023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:50.369023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:50.369023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:41:50.369023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:50.369023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:41:50.369023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:41:50.369023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:50.369023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:41:50.369023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:50.369023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:41:50.369023 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:41:50.369368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:50.369368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:41:50.369368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:50.369368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:41:50.369495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:50.369495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:41:50.369495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:41:50.369495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:41:50.369495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:41:50.369495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:41:50.369495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:50.369495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:41:50.369495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:41:50.369495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:41:50.369495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:41:50.369495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:41:50.369495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:50.369495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:50.369495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:41:50.369495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:41:50.369495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:41:50.369980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:41:50.369980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:41:50.369980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:41:50.369980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:50.369980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:50.369980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:50.369980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:41:50.369980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:50.369980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:50.369980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:50.369980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:41:50.369980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:50.369980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:41:50.369980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:41:50.369980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:50.369980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:41:50.369980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:50.369980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:41:50.369980 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:41:50.370501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:50.370501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:41:50.370501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:50.370501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:41:50.372429 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:41:50.372718 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:41:50.372957 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:41:50.373040 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:41:50.373274 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:41:50.373503 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:41:50.373642 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:50.374123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:41:50.374123 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:50.374123 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:41:50.374123 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:41:50.374123 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:50.374123 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:41:50.374123 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:50.374123 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:41:50.375182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:50.375182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:41:50.375182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:50.375182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:41:50.375182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:41:50.375182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:41:50.375182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:41:50.375182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:50.375182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:50.375182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:50.375182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:41:50.375182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:50.375553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:50.375553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:50.375553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:41:50.375553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:50.375553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:41:50.375553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:41:50.375553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:50.375553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:41:50.375553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:50.375553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:41:50.375553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:41:50.375553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:50.375553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:41:50.375914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:50.375914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:41:50.376318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:50.376318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:41:50.376318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:41:50.376318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:41:50.376318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:41:50.376318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:41:50.376318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:50.376318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:41:50.376318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:41:50.376318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:41:50.376318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:41:50.376318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:41:50.376318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:50.376318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:50.376318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:41:50.376318 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:41:50.376767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:41:50.376767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:41:50.376767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:50.376767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:50.376767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:50.376767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:41:50.376767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:50.376767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:50.376767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:50.376767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:41:50.376767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:50.376767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:41:50.376767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:41:50.376767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:50.376767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:41:50.376767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:50.376767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:41:50.376767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:41:50.376767 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:50.377245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:41:50.377245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:50.377245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:41:55.119260 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:41:55.120255 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:41:55.120620 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:41:55.121945 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:41:55.162311 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:41:55.162691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:55.163512 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:41:55.163512 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:55.163512 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:41:55.163512 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:41:55.163512 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:55.163512 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:41:55.163512 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:55.163512 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:41:55.164324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:55.164324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:41:55.164324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:55.164324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:41:55.164324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:41:55.164324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:41:55.164324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:41:55.164324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:55.164324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:55.164324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:55.164324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:41:55.164324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:55.164845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:55.164845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:55.164845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:41:55.164845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:55.164845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:41:55.164845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:41:55.164845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:55.164845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:41:55.164845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:55.164845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:41:55.164845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:41:55.164845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:55.164845 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:41:55.165312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:55.165312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:41:55.165312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:55.165312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:41:55.165312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:41:55.165312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:41:55.165312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:41:55.165312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:41:55.165312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:55.165312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:41:55.165312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:41:55.165312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:41:55.165312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:41:55.165312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:41:55.165312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:55.165916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:55.165916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:41:55.165916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:41:55.165916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:41:55.165916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:41:55.165916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:55.165916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:55.165916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:55.165916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:41:55.165916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:55.165916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:55.165916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:55.165916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:41:55.165916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:55.165916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:41:55.165916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:41:55.165916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:55.165916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:41:55.165916 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:55.167076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:41:55.167076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:41:55.167076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:55.167076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:41:55.167076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:55.167076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:41:55.351437 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:41:55.352084 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:41:55.352271 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:41:55.353626 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:41:55.354080 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:41:55.354236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:55.355250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:41:55.355250 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:55.355250 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:41:55.355250 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:41:55.355250 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:55.355250 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:41:55.355250 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:55.355250 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:41:55.356869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:55.356869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:41:55.356869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:55.356869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:41:55.356869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:41:55.356869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:41:55.356869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:41:55.356869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:55.356869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:55.356869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:55.356869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:41:55.356869 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:55.357530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:55.357530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:55.357530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:41:55.357530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:55.357530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:41:55.357530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:41:55.357530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:55.357530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:41:55.357530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:55.357530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:41:55.357530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:41:55.357530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:55.357530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:41:55.358152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:55.358152 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:41:55.358688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:55.358688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:41:55.358688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:41:55.358688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:41:55.358688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:41:55.358688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:41:55.358688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:55.358688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:41:55.358688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:41:55.358688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:41:55.358688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:41:55.358688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:41:55.358688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:55.358688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:55.358688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:41:55.358688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:41:55.359303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:41:55.359303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:41:55.359303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:55.359303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:55.359303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:55.359303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:41:55.359303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:55.359303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:55.359303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:55.359303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:41:55.359303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:55.359303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:41:55.359303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:41:55.359303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:55.359303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:41:55.359303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:55.359303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:41:55.359303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:41:55.359303 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:55.359818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:41:55.359818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:55.359818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:41:55.359818 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:41:55.360262 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:41:55.360473 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:41:55.360673 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:41:55.361256 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:41:55.361657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:55.362707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:41:55.362707 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:55.362707 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:41:55.362707 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:41:55.362707 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:55.362707 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:41:55.362707 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:55.362707 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:41:55.363652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:55.363652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:41:55.363652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:41:55.363652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:41:55.363652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:55.363652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:41:55.363652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:55.363652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:41:55.363652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:55.363652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:41:55.363652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:55.363652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:41:55.363652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:55.363652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:41:55.363652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:41:55.364167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:55.364167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:41:55.364167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:55.364167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:41:55.364167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:41:55.364167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:55.364167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:41:55.364167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:55.364167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:41:55.364956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:55.364956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:41:55.364956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:55.364956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:41:55.364956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:41:55.364956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:41:55.364956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:41:55.364956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:41:55.364956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:41:55.364956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:41:55.364956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:55.364956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:55.365335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:55.365335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:41:55.365335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:55.365335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:55.365335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:55.365335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:41:55.365335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:55.365335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:41:55.365335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:41:55.365335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:55.365335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:41:55.365335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:55.365335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:41:55.365335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:41:55.365729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:55.365729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:41:55.365729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:55.365729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:41:55.366019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:55.366019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:41:55.366019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:41:55.366019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:41:55.366019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:41:55.366019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:41:55.366019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:55.366019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:41:55.366019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:41:55.366019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:41:55.366019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:41:55.366019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:41:55.366019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:55.366019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:55.366019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:41:55.366019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:41:55.366506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:41:55.366506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:41:55.366506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:41:55.366506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:41:55.366506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:41:55.366506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:55.366506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:55.366506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:55.366506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:41:55.366506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:55.366506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:55.366506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:55.366506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:41:55.366506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:55.366506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:41:55.366506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:41:55.366506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:55.366506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:41:55.367030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:55.367030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:41:55.367030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:41:55.367030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:55.367030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:41:55.367030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:55.367030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:41:55.367030 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:41:55.367030 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:41:55.367030 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:41:55.367334 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:41:55.367533 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:41:55.367892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:55.368578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:41:55.368578 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:55.368578 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:41:55.368578 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:41:55.368578 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:55.368578 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:41:55.368578 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:55.368578 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:41:55.369574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:55.369574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:41:55.369574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:41:55.369574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:41:55.369574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:55.369574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:41:55.369574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:55.369574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:41:55.369574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:55.369574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:41:55.369574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:55.369574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:41:55.369574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:55.369574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:41:55.369574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:41:55.370009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:55.370009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:41:55.370009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:55.370009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:41:55.370009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:41:55.370009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:55.370009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:41:55.370009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:55.370009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:41:55.370712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:55.370712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:41:55.370712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:55.370712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:41:55.370712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:41:55.370712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:41:55.370712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:41:55.370712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:41:55.370712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:41:55.370712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:55.370712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:55.371085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:55.371085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:41:55.371085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:55.371085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:55.371085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:55.371085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:41:55.371085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:55.371085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:41:55.371085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:41:55.371085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:55.371085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:41:55.371085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:55.371085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:41:55.371085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:41:55.371488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:55.371488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:41:55.371488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:55.371488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:41:55.371793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:55.371793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:41:55.371793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:41:55.371793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:41:55.371793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:41:55.371793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:41:55.371793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:55.371793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:41:55.371793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:41:55.371793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:41:55.371793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:41:55.371793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:41:55.371793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:55.371793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:55.371793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:41:55.371793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:41:55.371793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:41:55.372264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:41:55.372264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:41:55.372264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:41:55.372264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:55.372264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:55.372264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:55.372264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:41:55.372264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:55.372264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:55.372264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:55.372264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:41:55.372264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:55.372264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:41:55.372264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:41:55.372264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:55.372264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:41:55.372264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:55.372264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:41:55.372264 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:41:55.372799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:55.372799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:41:55.372799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:55.372799 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:41:55.372912 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:41:55.373147 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:41:55.373254 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:41:55.373416 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:41:55.373896 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:41:55.374429 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:41:55.374821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:55.375578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:41:55.375578 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:55.375578 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:41:55.375578 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:41:55.375578 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:55.375578 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:41:55.375578 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:55.375578 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:41:55.376469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:55.376469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:41:55.376469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:55.376469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:41:55.376469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:41:55.376469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:41:55.376469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:41:55.376469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:55.376469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:55.376469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:55.376469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:41:55.376469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:55.376828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:41:55.376828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:55.376828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:41:55.376828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:55.376828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:41:55.376828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:41:55.376828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:55.376828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:41:55.376828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:55.376828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:41:55.376828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:41:55.376828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:55.376828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:41:55.377177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:55.377177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:41:55.377495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:55.377495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:41:55.377495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:41:55.377495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:41:55.377495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:41:55.377495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:41:55.377495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:55.377495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:41:55.377495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:41:55.377495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:41:55.377495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:41:55.377495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:41:55.377495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:41:55.377495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:41:55.377495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:41:55.377495 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:41:55.377949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:41:55.377949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:41:55.377949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:55.377949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:55.377949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:41:55.377949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:41:55.377949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:41:55.377949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:41:55.377949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:41:55.377949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:41:55.377949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:41:55.377949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:41:55.377949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:41:55.377949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:41:55.377949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:41:55.377949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:41:55.377949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:41:55.377949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:41:55.377949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:41:55.378520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:41:55.378520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:41:55.378520 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:42:00.120051 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:42:00.120762 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:42:00.120882 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:42:00.122092 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:42:00.159484 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:42:00.159681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:00.160433 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:42:00.160433 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:00.160433 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:42:00.160433 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:42:00.160433 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:00.160433 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:42:00.160433 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:00.160433 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:42:00.161273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:00.161273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:42:00.161273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:00.161273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:42:00.161273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:42:00.161273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:42:00.161273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:42:00.161273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:00.161273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:00.161273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:00.161273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:42:00.161273 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:00.162424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:00.162424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:00.162424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:42:00.162424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:00.162424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:42:00.162424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:42:00.162424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:00.162424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:42:00.162424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:00.162424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:42:00.162424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:42:00.162424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:00.162424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:42:00.163128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:00.163128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:42:00.163128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:00.163128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:42:00.163128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:42:00.163128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:42:00.163128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:42:00.163128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:42:00.163128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:00.163128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:42:00.163128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:42:00.163128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:42:00.163128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:42:00.163128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:42:00.163128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:00.164300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:00.164300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:42:00.164300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:42:00.164300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:42:00.164300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:42:00.164300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:00.164300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:00.164300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:00.164300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:42:00.164300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:00.164300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:00.164300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:00.164300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:42:00.164300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:00.164300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:42:00.164300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:42:00.164300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:00.164300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:42:00.164300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:00.165598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:42:00.165598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:42:00.165598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:00.165598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:42:00.165598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:00.165598 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:42:00.352211 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:42:00.352789 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:42:00.352966 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:42:00.354243 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:42:00.357422 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:42:00.357422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:00.357422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:42:00.357422 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:00.357422 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:42:00.357422 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:42:00.357422 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:00.357422 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:42:00.357422 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:00.357422 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:42:00.357783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:00.357783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:42:00.357783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:00.357783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:42:00.357783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:42:00.357783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:42:00.357783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:42:00.357783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:00.357783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:00.357783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:00.357783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:42:00.357783 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:00.358181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:00.358181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:00.358181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:42:00.358181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:00.358181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:42:00.358181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:42:00.358181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:00.358181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:42:00.358181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:00.358181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:42:00.358181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:42:00.358181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:00.358181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:42:00.358529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:00.358529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:42:00.359005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:00.359005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:42:00.359005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:42:00.359005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:42:00.359005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:42:00.359005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:42:00.359005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:00.359005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:42:00.359005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:42:00.359005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:42:00.359005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:42:00.359005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:42:00.359005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:00.359005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:00.359005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:42:00.359005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:42:00.359462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:42:00.359462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:42:00.359462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:00.359462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:00.359462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:00.359462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:42:00.359462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:00.359462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:00.359462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:00.359462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:42:00.359462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:00.359462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:42:00.359462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:42:00.359462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:00.359462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:42:00.359462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:00.359462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:42:00.359462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:42:00.359462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:00.359928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:42:00.359928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:00.359928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:42:00.360207 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:42:00.360561 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:42:00.360668 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:42:00.360782 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:42:00.361273 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:42:00.361621 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:00.362524 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:42:00.362524 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:00.362524 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:42:00.362524 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:42:00.362524 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:00.362524 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:42:00.362524 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:00.362524 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:42:00.363194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:00.363194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:42:00.363194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:42:00.363194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:42:00.363194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:00.363194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:42:00.363194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:00.363194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:42:00.363194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:00.363194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:42:00.363194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:00.363194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:42:00.363194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:00.363194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:42:00.363194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:42:00.363860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:00.363860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:42:00.363860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:00.363860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:42:00.363860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:42:00.363860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:00.363860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:42:00.363860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:00.363860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:42:00.364128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:00.364128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:42:00.364128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:00.364128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:42:00.364128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:42:00.364128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:42:00.364128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:42:00.364128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:42:00.364128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:42:00.364128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:42:00.364128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:00.364128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:00.364488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:00.364488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:42:00.364488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:00.364488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:00.364488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:00.364488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:42:00.364488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:00.364488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:42:00.364488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:42:00.364488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:00.364488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:42:00.364488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:00.364488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:42:00.364488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:42:00.364861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:00.364861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:42:00.364861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:00.364861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:42:00.364971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:00.364971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:42:00.364971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:42:00.364971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:42:00.364971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:42:00.364971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:42:00.364971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:00.364971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:42:00.364971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:42:00.364971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:42:00.364971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:42:00.364971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:42:00.364971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:00.364971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:00.364971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:42:00.364971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:42:00.365386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:42:00.365386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:42:00.365386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:42:00.365386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:42:00.365386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:42:00.365386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:00.365386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:00.365386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:00.365386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:42:00.365386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:00.365386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:00.365386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:00.365386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:42:00.365386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:00.365386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:42:00.365386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:42:00.365386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:00.365386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:42:00.365890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:00.365890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:42:00.365890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:42:00.365890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:00.365890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:42:00.365890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:00.365890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:42:00.366818 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:42:00.367102 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:42:00.367220 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:42:00.367332 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:42:00.367778 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:42:00.367969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:00.368485 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:42:00.368485 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:00.368485 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:42:00.368485 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:42:00.368485 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:00.368485 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:42:00.368485 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:00.368485 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:42:00.369361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:00.369361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:42:00.369361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:42:00.369361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:42:00.369361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:00.369361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:42:00.369361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:00.369361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:42:00.369361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:00.369361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:42:00.369361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:00.369361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:42:00.369361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:00.369361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:42:00.369361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:42:00.369724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:00.369724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:42:00.369724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:00.369724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:42:00.369724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:42:00.369724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:00.369724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:42:00.369724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:00.369724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:42:00.370244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:00.370244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:42:00.370244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:00.370244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:42:00.370244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:42:00.370244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:42:00.370244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:42:00.370244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:42:00.370244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:42:00.370244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:00.370244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:00.370506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:00.370506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:42:00.370506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:00.370506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:00.370506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:00.370506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:42:00.370506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:00.370506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:42:00.370506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:42:00.370506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:00.370506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:42:00.370506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:00.370506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:42:00.370506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:42:00.370763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:00.370763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:42:00.370763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:00.370763 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:42:00.371368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:00.371368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:42:00.371368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:42:00.371368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:42:00.371368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:42:00.371368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:42:00.371368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:00.371368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:42:00.371368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:42:00.371368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:42:00.371368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:42:00.371368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:42:00.371368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:00.371368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:00.371368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:42:00.371368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:42:00.371368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:42:00.371718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:42:00.371718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:42:00.371718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:42:00.371718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:00.371718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:00.371718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:00.371718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:42:00.371718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:00.371718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:00.371718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:00.371718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:42:00.371718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:00.371718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:42:00.371718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:42:00.371718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:00.371718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:42:00.371718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:00.371718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:42:00.371718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:42:00.372046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:00.372046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:42:00.372046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:00.372046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:42:00.373293 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:42:00.373569 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:42:00.373683 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:42:00.373901 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:42:00.374200 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:42:00.374467 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:42:00.374620 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:00.375228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:42:00.375228 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:00.375228 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:42:00.375228 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:42:00.375228 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:00.375228 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:42:00.375228 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:00.375228 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:42:00.376576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:00.376576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:42:00.376576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:00.376576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:42:00.376576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:42:00.376576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:42:00.376576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:42:00.376576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:00.376576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:00.376576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:00.376576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:42:00.376576 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:00.376925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:00.376925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:00.376925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:42:00.376925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:00.376925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:42:00.376925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:42:00.376925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:00.376925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:42:00.376925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:00.376925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:42:00.376925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:42:00.376925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:00.376925 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:42:00.377258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:00.377258 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:42:00.378122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:00.378122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:42:00.378122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:42:00.378122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:42:00.378122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:42:00.378122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:42:00.378122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:00.378122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:42:00.378122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:42:00.378122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:42:00.378122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:42:00.378122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:42:00.378122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:00.378122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:00.378122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:42:00.378122 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:42:00.378656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:42:00.378656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:42:00.378656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:00.378656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:00.378656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:00.378656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:42:00.378656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:00.378656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:00.378656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:00.378656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:42:00.378656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:00.378656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:42:00.378656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:42:00.378656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:00.378656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:42:00.378656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:00.378656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:42:00.378656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:42:00.378656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:00.379104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:42:00.379104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:00.379104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:42:05.120662 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:42:05.121798 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:42:05.121798 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:42:05.123072 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:42:05.159747 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:42:05.160191 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:05.161256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:42:05.161256 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:05.161256 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:42:05.161256 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:42:05.161256 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:05.161256 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:42:05.161256 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:05.161256 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:42:05.162000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:05.162000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:42:05.162000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:05.162000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:42:05.162000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:42:05.162000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:42:05.162000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:42:05.162000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:05.162000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:05.162000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:05.162000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:42:05.162000 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:05.162751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:05.162751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:05.162751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:42:05.162751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:05.162751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:42:05.162751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:42:05.162751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:05.162751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:42:05.162751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:05.162751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:42:05.162751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:42:05.162751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:05.162751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:42:05.163526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:05.163526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:42:05.163526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:05.163526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:42:05.163526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:42:05.163526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:42:05.163526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:42:05.163526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:42:05.163526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:05.163526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:42:05.163526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:42:05.163526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:42:05.163526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:42:05.163526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:42:05.163526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:05.164530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:05.164530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:42:05.164530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:42:05.164530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:42:05.164530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:42:05.164530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:05.164530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:05.164530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:05.164530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:42:05.164530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:05.164530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:05.164530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:05.164530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:42:05.164530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:05.164530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:42:05.164530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:42:05.164530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:05.164530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:42:05.164530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:05.165560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:42:05.165560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:42:05.165560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:05.165560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:42:05.165560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:05.165560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:42:05.352742 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:42:05.353363 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:42:05.353771 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:42:05.355047 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:42:05.355438 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:42:05.355772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:05.356633 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:42:05.356633 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:05.356633 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:42:05.356633 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:42:05.356633 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:05.356633 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:42:05.356633 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:05.356633 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:42:05.358118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:05.358118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:42:05.358118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:05.358118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:42:05.358118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:42:05.358118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:42:05.358118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:42:05.358118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:05.358118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:05.358118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:05.358118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:42:05.358118 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:05.358718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:05.358718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:05.358718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:42:05.358718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:05.358718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:42:05.358718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:42:05.358718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:05.358718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:42:05.358718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:05.358718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:42:05.358718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:42:05.358718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:05.358718 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:42:05.359089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:05.359089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:42:05.359735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:05.359735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:42:05.359735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:42:05.359735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:42:05.359735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:42:05.359735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:42:05.359735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:05.359735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:42:05.359735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:42:05.359735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:42:05.359735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:42:05.359735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:42:05.359735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:05.359735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:05.359735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:42:05.359735 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:42:05.360262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:42:05.360262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:42:05.360262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:05.360262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:05.360262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:05.360262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:42:05.360262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:05.360262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:05.360262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:05.360262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:42:05.360262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:05.360262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:42:05.360262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:42:05.360262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:05.360262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:42:05.360262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:05.360262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:42:05.360262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:42:05.360262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:05.360826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:42:05.360826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:05.360826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:42:05.360826 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:42:05.361043 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:42:05.361213 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:42:05.361394 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:42:05.361777 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:42:05.361975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:05.362510 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:42:05.362510 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:05.362510 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:42:05.362510 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:42:05.362510 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:05.362510 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:42:05.362510 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:05.362510 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:42:05.363223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:05.363223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:42:05.363223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:42:05.363223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:42:05.363223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:05.363223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:42:05.363223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:05.363223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:42:05.363223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:05.363223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:42:05.363223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:05.363223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:42:05.363223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:05.363223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:42:05.363223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:42:05.363745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:05.363745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:42:05.363745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:05.363745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:42:05.363745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:42:05.363745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:05.363745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:42:05.363745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:05.363745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:42:05.364041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:05.364041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:42:05.364041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:05.364041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:42:05.364041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:42:05.364041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:42:05.364041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:42:05.364041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:42:05.364041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:42:05.364041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:42:05.364041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:05.364041 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:05.364395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:05.364395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:42:05.364395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:05.364395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:05.364395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:05.364395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:42:05.364395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:05.364395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:42:05.364395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:42:05.364395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:05.364395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:42:05.364395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:05.364395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:42:05.364395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:42:05.364810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:05.364810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:42:05.364810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:05.364810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:42:05.365102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:05.365102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:42:05.365102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:42:05.365102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:42:05.365102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:42:05.365102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:42:05.365102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:05.365102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:42:05.365102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:42:05.365102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:42:05.365102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:42:05.365102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:42:05.365102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:05.365102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:05.365102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:42:05.365102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:42:05.365579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:42:05.365579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:42:05.365579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:42:05.365579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:42:05.365579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:42:05.365579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:05.365579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:05.365579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:05.365579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:42:05.365579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:05.365579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:05.365579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:05.365579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:42:05.365579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:05.365579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:42:05.365579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:42:05.365579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:05.365579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:42:05.366120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:05.366120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:42:05.366120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:42:05.366120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:05.366120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:42:05.366120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:05.366120 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:42:05.367284 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:42:05.367818 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:42:05.367818 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:42:05.367818 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:42:05.368612 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:42:05.368612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:05.368954 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:42:05.368954 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:05.368954 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:42:05.368954 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:42:05.368954 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:05.368954 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:42:05.368954 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:05.368954 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:42:05.369700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:05.369700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:42:05.369700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:42:05.369700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:42:05.369700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:05.369700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:42:05.369700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:05.369700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:42:05.369700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:05.369700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:42:05.369700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:05.369700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:42:05.369700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:05.369700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:42:05.369700 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:42:05.370483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:05.370483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:42:05.370483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:05.370483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:42:05.370483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:42:05.370483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:05.370483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:42:05.370483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:05.370483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:42:05.370483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:05.370483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:42:05.370483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:05.370483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:42:05.370483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:42:05.371363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:42:05.371363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:42:05.371363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:42:05.371363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:42:05.371363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:05.371363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:05.371363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:05.371363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:42:05.371363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:05.371363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:05.371363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:05.371363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:42:05.371363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:05.372107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:42:05.372107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:42:05.372107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:05.372107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:42:05.372107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:05.372107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:42:05.372107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:42:05.372107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:05.372107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:42:05.372107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:05.372107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:42:05.372107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:05.372830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:42:05.372830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:42:05.372830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:42:05.372830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:42:05.372830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:42:05.372830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:05.372830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:42:05.372830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:42:05.372830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:42:05.372830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:42:05.372830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:42:05.372830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:05.372830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:05.372830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:42:05.372830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:42:05.372830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:42:05.372830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:42:05.372830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:42:05.372830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:42:05.374585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:05.374585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:05.374585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:05.374585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:42:05.374585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:05.374585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:05.374585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:05.374585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:42:05.374585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:05.374585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:42:05.374585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:42:05.374585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:05.374585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:42:05.374585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:05.374585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:42:05.374585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:42:05.374585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:05.374585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:42:05.374585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:05.374585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:42:05.375591 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:42:05.375591 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:42:05.375591 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:42:05.375591 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:42:05.375591 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:42:05.375591 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:42:05.376006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:05.376006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:42:05.376006 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:05.376006 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:42:05.376006 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:42:05.376006 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:05.376006 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:42:05.376006 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:05.376006 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:42:05.376834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:05.376834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:42:05.376834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:05.376834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:42:05.376834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:42:05.376834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:42:05.376834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:42:05.376834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:05.376834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:05.376834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:05.376834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:42:05.376834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:05.377968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:05.377968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:05.377968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:42:05.377968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:05.377968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:42:05.377968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:42:05.377968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:05.377968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:42:05.377968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:05.377968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:42:05.377968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:42:05.377968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:05.377968 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:42:05.378377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:05.378377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:42:05.378377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:05.378377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:42:05.378377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:42:05.378377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:42:05.378377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:42:05.378377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:42:05.378377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:05.378377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:42:05.378377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:42:05.378377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:42:05.378377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:42:05.378377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:42:05.378377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:05.378715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:05.378715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:42:05.378715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:42:05.378715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:42:05.378715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:42:05.378715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:05.378715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:05.378715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:05.378715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:42:05.378715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:05.378715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:05.378715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:05.378715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:42:05.378715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:05.378715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:42:05.378715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:42:05.378715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:05.378715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:42:05.378715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:05.379109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:42:05.379109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:42:05.379109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:05.379109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:42:05.379109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:05.379109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:42:10.121338 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:42:10.122269 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:42:10.122510 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:42:10.123876 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:42:10.167333 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:42:10.167571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:10.168019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:42:10.168019 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:10.168019 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:42:10.168019 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:42:10.168019 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:10.168019 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:42:10.168019 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:10.168019 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:42:10.168807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.045 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:10.168807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:42:10.168807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:10.168807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:42:10.168807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:42:10.168807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:42:10.168807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:42:10.168807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:10.168807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:10.168807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:10.168807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:42:10.168807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:10.169502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:10.169502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:10.169502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:42:10.169502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:10.169502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:42:10.169502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:42:10.169502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:10.169502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:42:10.169502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:10.169502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:42:10.169502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:42:10.169502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:10.169502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:42:10.170181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:10.170181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:42:10.170181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:10.170181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:42:10.170181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:42:10.170181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:42:10.170181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:42:10.170181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:42:10.170181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:10.170181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:42:10.170181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:42:10.170181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:42:10.170181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:42:10.170181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:42:10.170181 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:10.170981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:10.170981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:42:10.170981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:42:10.170981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:42:10.170981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:42:10.170981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:10.170981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:10.170981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:10.170981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:42:10.170981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:10.170981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:10.170981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:10.170981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:42:10.170981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:10.170981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:42:10.170981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:42:10.170981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:10.170981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:42:10.170981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:10.172087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:42:10.172087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:42:10.172087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:10.172087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:42:10.172087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:10.172087 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:42:10.354577 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:42:10.354577 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:42:10.354577 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:42:10.354994 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:42:10.355519 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:42:10.355671 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:10.356395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:42:10.356395 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:10.356395 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:42:10.356395 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:42:10.356395 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:10.356395 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:42:10.356395 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:10.356395 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:42:10.357604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:10.357604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:42:10.357604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:10.357604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:42:10.357604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:42:10.357604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:42:10.357604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:42:10.357604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:10.357604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:10.357604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:10.357604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:42:10.357604 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:10.358297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:10.358297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:10.358297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:42:10.358297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:10.358297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:42:10.358297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:42:10.358297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:10.358297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:42:10.358297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:10.358297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:42:10.358297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:42:10.358297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:10.358297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:42:10.358962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:10.358962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:42:10.358962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:10.358962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:42:10.358962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:42:10.358962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:42:10.358962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:42:10.358962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:42:10.358962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:10.358962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:42:10.358962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:42:10.358962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:42:10.358962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:42:10.358962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:42:10.358962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:10.359743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:10.359743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:42:10.359743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:42:10.359743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:42:10.359743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:42:10.359743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:10.359743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:10.359743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:10.359743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:42:10.359743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:10.359743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:10.359743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:10.359743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:42:10.359743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:10.359743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:42:10.359743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:42:10.359743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:10.359743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:42:10.359743 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:10.361526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:42:10.361526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:42:10.361526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:10.361526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:42:10.361526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:10.361526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:42:10.361526 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:42:10.361913 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:42:10.361913 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:42:10.361913 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:42:10.362127 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:42:10.362236 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:10.362892 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:42:10.362892 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:10.362892 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:42:10.362892 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:42:10.362892 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:10.362892 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:42:10.362892 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:10.362892 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:42:10.363556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:10.363556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:42:10.363556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:42:10.363556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:42:10.363556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:10.363556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:42:10.363556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:10.363556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:42:10.363556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:10.363556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:42:10.363556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:10.363556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:42:10.363556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:10.363556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:42:10.363556 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:42:10.364034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:10.364034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:42:10.364034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:10.364034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:42:10.364034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:42:10.364034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:10.364034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:42:10.364034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:10.364034 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:42:10.364313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:10.364313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:42:10.364313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:10.364313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:42:10.364313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:42:10.364313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:42:10.364313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:42:10.364313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:42:10.364313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:42:10.364313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:42:10.364313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:10.364313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:10.364772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:10.364772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:42:10.364772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:10.364772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:10.364772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:10.364772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:42:10.364772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:10.364772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:42:10.364772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:42:10.364772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:10.364772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:42:10.364772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:10.364772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:42:10.364772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:42:10.365168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:10.365168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:42:10.365168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:10.365168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:42:10.365313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:10.365313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:42:10.365313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:42:10.365313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:42:10.365313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:42:10.365313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:42:10.365313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:10.365313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:42:10.365313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:42:10.365313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:42:10.365313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:42:10.365313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:42:10.365313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:10.365313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:10.365313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:42:10.365313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:42:10.365760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:42:10.365760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:42:10.365760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:42:10.365760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:42:10.365760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:42:10.365760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:10.365760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:10.365760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:10.365760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:42:10.365760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:10.365760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:10.365760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:10.365760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:42:10.365760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:10.365760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:42:10.365760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:42:10.365760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:10.365760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:42:10.366456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:10.366456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:42:10.366456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:42:10.366456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:10.366456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:42:10.366456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:10.366456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:42:10.367646 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:42:10.367909 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:42:10.368013 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:42:10.368134 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:42:10.368451 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:42:10.368772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:10.369564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:42:10.369564 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:10.369564 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:42:10.369564 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:42:10.369564 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:10.369564 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:42:10.369564 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:10.369564 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:42:10.370242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:10.370242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:42:10.370242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:42:10.370242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:42:10.370242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:10.370242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:42:10.370242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:10.370242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:42:10.370242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:10.370242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:42:10.370242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:10.370242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:42:10.370242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:10.370242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:42:10.370242 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:42:10.370744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:10.370744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:42:10.370744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:10.370744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:42:10.370744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:42:10.370744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:10.370744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:42:10.370744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:10.370744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:42:10.371194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:10.371194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:42:10.371194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:10.371194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:42:10.371194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:42:10.371194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:42:10.371194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:42:10.371194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:42:10.371194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:42:10.371194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:10.371194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:10.371656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:10.371656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:42:10.371656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:10.371656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:10.371656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:10.371656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:42:10.371656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:10.371656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:42:10.371656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:42:10.371656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:10.371656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:42:10.371656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:10.371656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:42:10.371656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:42:10.372077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:10.372077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:42:10.372077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:10.372077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:42:10.372716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:10.372716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:42:10.372716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:42:10.372716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:42:10.372716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:42:10.372716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:42:10.372716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:10.372716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:42:10.372716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:42:10.372716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:42:10.372716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:42:10.372716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:42:10.372716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:10.372716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:10.372716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:42:10.372716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:42:10.372716 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:42:10.373277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:42:10.373277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:42:10.373277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:42:10.373277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:10.373277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:10.373277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:10.373277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:42:10.373277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:10.373277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:10.373277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:10.373277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:42:10.373277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:10.373277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:42:10.373277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:42:10.373277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:10.373277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:42:10.373277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:10.373277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:42:10.373277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:42:10.373842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:10.373842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:42:10.373842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:10.373842 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:42:10.374096 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:42:10.374471 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:42:10.374583 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:42:10.374762 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:42:10.375319 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:42:10.375755 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:42:10.375984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:10.376705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:42:10.376705 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:10.376705 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:42:10.376705 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:42:10.376705 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:10.376705 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:42:10.376705 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:10.376705 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:42:10.377562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:10.377562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:42:10.377562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:10.377562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:42:10.377562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:42:10.377562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:42:10.377562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:42:10.377562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:10.377562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:10.377562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:10.377562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:42:10.377562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:10.378067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:10.378067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:10.378067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:42:10.378067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:10.378067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:42:10.378067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:42:10.378067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:10.378067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:42:10.378067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:10.378067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:42:10.378067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:42:10.378067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:10.378067 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:42:10.378540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:10.378540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:42:10.378540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:10.378540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:42:10.378540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:42:10.378540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:42:10.378540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:42:10.378540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:42:10.378540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:10.378540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:42:10.378540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:42:10.378540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:42:10.378540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:42:10.378540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:42:10.378540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:10.379074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:10.379074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:42:10.379074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:42:10.379074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:42:10.379074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:42:10.379074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:10.379074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:10.379074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:10.379074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:42:10.379074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:10.379074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:10.379074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:10.379074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:42:10.379074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:10.379074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:42:10.379074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:42:10.379074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:10.379074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:42:10.379074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:10.379686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:42:10.379686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:42:10.379686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:10.379686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:42:10.379686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:10.379686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:42:15.122273 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:42:15.122910 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:42:15.123049 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:42:15.124286 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:42:15.161771 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:42:15.162020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:15.162661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:42:15.162661 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:15.162661 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:42:15.162661 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:42:15.162661 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:15.162661 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:42:15.162661 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:15.162661 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:42:15.163392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:15.163392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:42:15.163392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:15.163392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:42:15.163392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:42:15.163392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:42:15.163392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:42:15.163392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:15.163392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:15.163392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:15.163392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:42:15.163392 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:15.163760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:15.163760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:15.163760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:42:15.163760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:15.163760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:42:15.163760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:42:15.163760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:15.163760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:42:15.163760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:15.163760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:42:15.163760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:42:15.163760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:15.163760 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:42:15.164091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:15.164091 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:42:15.164154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:15.164154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:42:15.164154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:42:15.164154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:42:15.164154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:42:15.164154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:42:15.164154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:15.164154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:42:15.164154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:42:15.164154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:42:15.164154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:42:15.164154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:42:15.164154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:15.164154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:15.164154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:42:15.164154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:42:15.164585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:42:15.164585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:42:15.164585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:15.164585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:15.164585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:15.164585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:42:15.164585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:15.164585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:15.164585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:15.164585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:42:15.164585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:15.164585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:42:15.164585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:42:15.164585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:15.164585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:42:15.164585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:15.164585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:42:15.164585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:42:15.164585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:15.165074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:42:15.165074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:15.165074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:42:15.354201 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:42:15.354701 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:42:15.354883 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:42:15.355953 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:42:15.356336 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:42:15.356597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:15.357508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:42:15.357508 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:15.357508 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:42:15.357508 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:42:15.357508 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:15.357508 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:42:15.357508 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:15.357508 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:42:15.358911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:15.358911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:42:15.358911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:15.358911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:42:15.358911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:42:15.358911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:42:15.358911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:42:15.358911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:15.358911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:15.358911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:15.358911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:42:15.358911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:15.359632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:15.359632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:15.359632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:42:15.359632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:15.359632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:42:15.359632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:42:15.359632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:15.359632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:42:15.359632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:15.359632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:42:15.359632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:42:15.359632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:15.359632 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:42:15.360206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:15.360206 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:42:15.360490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:15.360490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:42:15.360490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:42:15.360490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:42:15.360490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:42:15.360490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:42:15.360490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:15.360490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:42:15.360490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:42:15.360490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:42:15.360490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:42:15.360490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:42:15.360490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:15.360490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:15.360490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:42:15.360490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:42:15.361256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:42:15.361256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:42:15.361256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:15.361256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:15.361256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:15.361256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:42:15.361256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:15.361256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:15.361256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:15.361256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:42:15.361256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:15.361256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:42:15.361256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:42:15.361256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:15.361256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:42:15.361256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:15.361256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:42:15.361256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:42:15.361256 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:15.362216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:42:15.362216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:15.362216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:42:15.362216 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:42:15.362216 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:42:15.362216 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:42:15.362529 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:42:15.362804 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:42:15.363127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:15.363894 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:42:15.363894 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:15.363894 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:42:15.363894 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:42:15.363894 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:15.363894 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:42:15.363894 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:15.363894 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:42:15.364645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:15.364645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:42:15.364645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:42:15.364645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:42:15.364645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:15.364645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:42:15.364645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:15.364645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:42:15.364645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:15.364645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:42:15.364645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:15.364645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:42:15.364645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:15.364645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:42:15.364645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:42:15.365385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:15.365385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:42:15.365385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:15.365385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:42:15.365385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:42:15.365385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:15.365385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:42:15.365385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:15.365385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:42:15.365385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:15.365385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:42:15.365385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:15.365385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:42:15.365385 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:42:15.366131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:42:15.366131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:42:15.366131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:42:15.366131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:42:15.366131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:42:15.366131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:15.366131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:15.366131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:15.366131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:42:15.366131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:15.366131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:15.366131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:15.366131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:42:15.366770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:15.366770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:42:15.366770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:42:15.366770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:15.366770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:42:15.366770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:15.366770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:42:15.366770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:42:15.366770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:15.366770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:42:15.366770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:15.366770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:42:15.367113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:15.367113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:42:15.367113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:42:15.367113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:42:15.367113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:42:15.367113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:42:15.367113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:15.367113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:42:15.367113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:42:15.367113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:42:15.367113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:42:15.367113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:42:15.367113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:15.367113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:15.367113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:42:15.367113 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:42:15.367610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:42:15.367610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:42:15.367610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:42:15.367610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:42:15.367610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:42:15.367610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:15.367610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:15.367610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:15.367610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:42:15.367610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:15.367610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:15.367610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:15.367610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:42:15.367610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:15.367610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:42:15.367610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:42:15.367610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:15.367610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:42:15.368107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:15.368107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:42:15.368107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:42:15.368107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:15.368107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:42:15.368107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:15.368107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:42:15.368107 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:42:15.368350 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:42:15.368499 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:42:15.368618 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:42:15.369089 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:42:15.369498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:15.370123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:42:15.370123 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:15.370123 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:42:15.370123 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:42:15.370123 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:15.370123 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:42:15.370123 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:15.370123 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:42:15.370835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:15.370835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:42:15.370835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:42:15.370835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:42:15.370835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:15.370835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:42:15.370835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:15.370835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:42:15.370835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:15.370835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:42:15.370835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:15.370835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:42:15.370835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:15.370835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:42:15.370835 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:42:15.371327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:15.371327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:42:15.371327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:15.371327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:42:15.371327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:42:15.371327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:15.371327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:42:15.371327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:15.371327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:42:15.371685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:15.371685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:42:15.371685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:15.371685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:42:15.371685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:42:15.371685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:42:15.371685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:42:15.371685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:42:15.371685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:42:15.371685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:15.371685 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:15.372007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:15.372007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:42:15.372007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:15.372007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:15.372007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:15.372007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:42:15.372007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:15.372007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:42:15.372007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:42:15.372007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:15.372007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:42:15.372007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:15.372007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:42:15.372007 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:42:15.372482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:15.372482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:42:15.372482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:15.372482 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:42:15.372714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:15.372714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:42:15.372714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:42:15.372714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:42:15.372714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:42:15.372714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:42:15.372714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:15.372714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:42:15.372714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:42:15.372714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:42:15.372714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:42:15.372714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:42:15.372714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:15.372714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:15.372714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:42:15.372714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:42:15.372714 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:42:15.373232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:42:15.373232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:42:15.373232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:42:15.373232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:15.373232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:15.373232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:15.373232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:42:15.373232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:15.373232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:15.373232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:15.373232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:42:15.373232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:15.373232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:42:15.373232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:42:15.373232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:15.373232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:42:15.373232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:15.373232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:42:15.373232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:42:15.373780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:15.373780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:42:15.373780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:15.373780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:42:15.374433 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:42:15.374664 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:42:15.374774 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:42:15.374913 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:42:15.375189 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:42:15.375418 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:42:15.375562 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:15.376032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:42:15.376032 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:15.376032 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:42:15.376032 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:42:15.376032 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:15.376032 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:42:15.376032 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:15.376032 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:42:15.376827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:15.376827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:42:15.376827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:15.376827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:42:15.376827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:42:15.376827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:42:15.376827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:42:15.376827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:15.376827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:15.376827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:15.376827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:42:15.376827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:15.377204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:15.377204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:15.377204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:42:15.377204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:15.377204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:42:15.377204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:42:15.377204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:15.377204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:42:15.377204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:15.377204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:42:15.377204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:42:15.377204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:15.377204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:42:15.377601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:15.377601 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:42:15.377779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:15.377779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:42:15.377779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:42:15.377779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:42:15.377779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:42:15.377779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:42:15.377779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:15.377779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:42:15.377779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:42:15.377779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:42:15.377779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:42:15.377779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:42:15.377779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:15.377779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:15.377779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:42:15.377779 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:42:15.378260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:42:15.378260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:42:15.378260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:15.378260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:15.378260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:15.378260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:42:15.378260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:15.378260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:15.378260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:15.378260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:42:15.378260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:15.378260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:42:15.378260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:42:15.378260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:15.378260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:42:15.378260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:15.378260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:42:15.378260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:42:15.378260 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:15.378804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:42:15.378804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:15.378804 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:42:20.122871 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:42:20.123834 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:42:20.123834 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:42:20.124837 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:42:20.161737 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:42:20.161957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:20.162514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:42:20.162514 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:20.162514 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:42:20.162514 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:42:20.162514 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:20.162514 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:42:20.162514 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:20.162514 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:42:20.163514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:20.163514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:42:20.163514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:20.163514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:42:20.163514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:42:20.163514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:42:20.163514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:42:20.163514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:20.163514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:20.163514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:20.163514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:42:20.163514 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:20.164308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:20.164308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:20.164308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:42:20.164308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:20.164308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:42:20.164308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:42:20.164308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:20.164308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:42:20.164308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:20.164308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:42:20.164308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:42:20.164308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:20.164308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:42:20.165104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:20.165104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:42:20.165104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:20.165104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:42:20.165104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:42:20.165104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:42:20.165104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:42:20.165104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:42:20.165104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:20.165104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:42:20.165104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:42:20.165104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:42:20.165104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:42:20.165104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:42:20.165104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:20.165976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:20.165976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:42:20.165976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:42:20.165976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:42:20.165976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:42:20.165976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:20.165976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:20.165976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:20.165976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:42:20.165976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:20.165976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:20.165976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:20.165976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:42:20.165976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:20.165976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:42:20.165976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:42:20.165976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:20.165976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:42:20.165976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:20.168166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:42:20.168166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:42:20.168166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:20.168166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:42:20.168166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:20.168166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:42:20.354882 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:42:20.358249 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:42:20.358249 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:42:20.358249 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:42:20.358249 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:42:20.358249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:20.358793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:42:20.358793 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:20.358793 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:42:20.358793 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:42:20.358793 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:20.358793 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:42:20.358793 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:20.358793 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:42:20.360302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:20.360302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:42:20.360302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:20.360302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:42:20.360302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:42:20.360302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:42:20.360302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:42:20.360302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:20.360302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:20.360302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:20.360302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:42:20.360302 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:20.360944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:20.360944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:20.360944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:42:20.360944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:20.360944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:42:20.360944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:42:20.360944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:20.360944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:42:20.360944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:20.360944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:42:20.360944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:42:20.360944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:20.360944 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:42:20.361583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:20.361583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:42:20.362622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:20.362622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:42:20.362622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:42:20.362622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:42:20.362622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:42:20.362622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:42:20.362622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:20.362622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:42:20.362622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:42:20.362622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:42:20.362622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:42:20.362622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:42:20.362622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:20.362622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:20.362622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:42:20.362622 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:42:20.363296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:42:20.363296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:42:20.363296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:20.363296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:20.363296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:20.363296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:42:20.363296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:20.363296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:20.363296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:20.363296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:42:20.363296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:20.363296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:42:20.363296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:42:20.363296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:20.363296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:42:20.363296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:20.363296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:42:20.363296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:42:20.363296 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:20.364135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:42:20.364135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:20.364135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:42:20.364135 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:42:20.364135 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:42:20.364135 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:42:20.364135 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:42:20.364688 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:42:20.364688 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:20.365052 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:42:20.365052 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:20.365052 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:42:20.365052 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:42:20.365052 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:20.365052 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:42:20.365052 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:20.365052 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:42:20.365906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:20.365906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:42:20.365906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:42:20.365906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:42:20.365906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:20.365906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:42:20.365906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:20.365906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:42:20.365906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:20.365906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:42:20.365906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:20.365906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:42:20.365906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:20.365906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:42:20.365906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:42:20.366364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:20.366364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:42:20.366364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:20.366364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:42:20.366364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:42:20.366364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:20.366364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:42:20.366364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:20.366364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:42:20.366947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:20.366947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:42:20.366947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:20.366947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:42:20.366947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:42:20.366947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:42:20.366947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:42:20.366947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:42:20.366947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:42:20.366947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:42:20.366947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:20.366947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:20.367310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:20.367310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:42:20.367310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:20.367310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:20.367310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:20.367310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:42:20.367310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:20.367310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:42:20.367310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:42:20.367310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:20.367310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:42:20.367310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:20.367310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:42:20.367310 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:42:20.367690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:20.367690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:42:20.367690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:20.367690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:42:20.368301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:20.368301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:42:20.368301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:42:20.368301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:42:20.368301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:42:20.368301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:42:20.368301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:20.368301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:42:20.368301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:42:20.368301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:42:20.368301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:42:20.368301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:42:20.368301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:20.368301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:20.368301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:42:20.368301 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:42:20.368853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:42:20.368853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:42:20.368853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:42:20.368853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:42:20.368853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:42:20.368853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:20.368853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:20.368853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:20.368853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:42:20.368853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:20.368853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:20.368853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:20.368853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:42:20.368853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:20.368853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:42:20.368853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:42:20.368853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:20.368853 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:42:20.369360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:20.369360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:42:20.369360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:42:20.369360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:20.369360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:42:20.369360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:20.369360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:42:20.369360 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:42:20.369360 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:42:20.369360 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:42:20.369713 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:42:20.369713 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:42:20.369713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:20.370352 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:42:20.370352 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:20.370352 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:42:20.370352 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:42:20.370352 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:20.370352 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:42:20.370352 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:20.370352 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:42:20.370949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:20.370949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:42:20.370949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:42:20.370949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:42:20.370949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:20.370949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:42:20.370949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:20.370949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:42:20.370949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:20.370949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:42:20.370949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:20.370949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:42:20.370949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:20.370949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:42:20.370949 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:42:20.371348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:20.371348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:42:20.371348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:20.371348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:42:20.371348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:42:20.371348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:20.371348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:42:20.371348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:20.371348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:42:20.371775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:20.371775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:42:20.371775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:20.371775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:42:20.371775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:42:20.371775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:42:20.371775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:42:20.371775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:42:20.371775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:42:20.371775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:20.371775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:20.372124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:20.372124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:42:20.372124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:20.372124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:20.372124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:20.372124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:42:20.372124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:20.372124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:42:20.372124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:42:20.372124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:20.372124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:42:20.372124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:20.372124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:42:20.372124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:42:20.372568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:20.372568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:42:20.372568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:20.372568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:42:20.372833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:20.372833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:42:20.372833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:42:20.372833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:42:20.372833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:42:20.372833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:42:20.372833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:20.372833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:42:20.372833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:42:20.372833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:42:20.372833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:42:20.372833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:42:20.372833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:20.372833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:20.372833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:42:20.372833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:42:20.372833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:42:20.373369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:42:20.373369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:42:20.373369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:42:20.373369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:20.373369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:20.373369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:20.373369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:42:20.373369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:20.373369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:20.373369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:20.373369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:42:20.373369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:20.373369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:42:20.373369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:42:20.373369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:20.373369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:42:20.373369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:20.373369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:42:20.373369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:42:20.373947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:20.373947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:42:20.373947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:20.373947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:42:20.374742 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:42:20.374959 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:42:20.375052 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:42:20.375189 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:42:20.375512 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:42:20.375722 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:42:20.375858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:20.376300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:42:20.376300 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:20.376300 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:42:20.376300 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:42:20.376300 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:20.376300 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:42:20.376300 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:20.376300 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:42:20.377083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:20.377083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:42:20.377083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:20.377083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:42:20.377083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:42:20.377083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:42:20.377083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:42:20.377083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:20.377083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:20.377083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:20.377083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:42:20.377083 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:20.377457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:20.377457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:20.377457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:42:20.377457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:20.377457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:42:20.377457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:42:20.377457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:20.377457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:42:20.377457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:20.377457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:42:20.377457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:42:20.377457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:20.377457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:42:20.377793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:20.377793 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:42:20.378278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:20.378278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:42:20.378278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:42:20.378278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:42:20.378278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:42:20.378278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:42:20.378278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:20.378278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:42:20.378278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:42:20.378278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:42:20.378278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:42:20.378278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:42:20.378278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:20.378278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:20.378278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:42:20.378278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:42:20.378724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:42:20.378724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:42:20.378724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:20.378724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:20.378724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:20.378724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:42:20.378724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:20.378724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:20.378724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:20.378724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:42:20.378724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:20.378724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:42:20.378724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:42:20.378724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:20.378724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:42:20.378724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:20.378724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:42:20.378724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:42:20.378724 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:20.379259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:42:20.379259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:20.379259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:42:25.123708 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:42:25.124468 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:42:25.124468 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:42:25.125338 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:42:25.132642 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG dbcounter [-] [105581] Writing DB stats neutron:SELECT=240 {{(pid=105581) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 14:42:25.167475 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:42:25.167754 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:25.168528 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:42:25.168528 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:25.168528 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:42:25.168528 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:42:25.168528 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:25.168528 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:42:25.168528 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:25.168528 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:42:25.169689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:25.169689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:42:25.169689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:25.169689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:42:25.169689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:42:25.169689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:42:25.169689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:42:25.169689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:25.169689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:25.169689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:25.169689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:42:25.169689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:25.170275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:25.170275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:25.170275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:42:25.170275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:25.170275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:42:25.170275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:42:25.170275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:25.170275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:42:25.170275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:25.170275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:42:25.170275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:42:25.170275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:25.170275 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:42:25.171002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:25.171002 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:42:25.171208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:25.171208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:42:25.171208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:42:25.171208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:42:25.171208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:42:25.171208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:42:25.171208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:25.171208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:42:25.171208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:42:25.171208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:42:25.171208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:42:25.171208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:42:25.171208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:25.171208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:25.171208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:42:25.171208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:42:25.171812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:42:25.171812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:42:25.171812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:25.171812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:25.171812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:25.171812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:42:25.171812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:25.171812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:25.171812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:25.171812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:42:25.171812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:25.171812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:42:25.171812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:42:25.171812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:25.171812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:42:25.171812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:25.171812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:42:25.171812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:42:25.171812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:25.172437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:42:25.172437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:25.172437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:42:25.355598 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:42:25.356085 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:42:25.356223 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:42:25.357035 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:42:25.357392 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:42:25.357656 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:25.358615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:42:25.358615 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:25.358615 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:42:25.358615 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:42:25.358615 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:25.358615 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:42:25.358615 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:25.358615 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:42:25.360535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:25.360535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:42:25.360535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:25.360535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:42:25.360535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:42:25.360535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:42:25.360535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:42:25.360535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:25.360535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:25.360535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:25.360535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:42:25.360535 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:25.361361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:25.361361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:25.361361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:42:25.361361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:25.361361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:42:25.361361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:42:25.361361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:25.361361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:42:25.361361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:25.361361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:42:25.361361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:42:25.361361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:25.361361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:42:25.362124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:25.362124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:42:25.362525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:25.362525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:42:25.362525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:42:25.362525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:42:25.362525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:42:25.362525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:42:25.362525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:25.362525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:42:25.362525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:42:25.362525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:42:25.362525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:42:25.362525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:42:25.362525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:25.362525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:25.362525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:42:25.362525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:42:25.363306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:42:25.363306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:42:25.363306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:25.363306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:25.363306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:25.363306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:42:25.363306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:25.363306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:25.363306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:25.363306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:42:25.363306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:25.363306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:42:25.363306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:42:25.363306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:25.363306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:42:25.363306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:25.363306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:42:25.363306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:42:25.363306 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:25.364278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:42:25.364278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:25.364278 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:42:25.364278 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:42:25.364278 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:42:25.364278 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:42:25.364278 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:42:25.364278 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:42:25.364818 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:25.365104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:42:25.365104 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:25.365104 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:42:25.365104 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:42:25.365104 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:25.365104 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:42:25.365104 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:25.365104 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:42:25.366347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:25.366347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:42:25.366347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:42:25.366347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:42:25.366347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:25.366347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:42:25.366347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:25.366347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:42:25.366347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:25.366347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:42:25.366347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:25.366347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:42:25.366347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:25.366347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:42:25.366347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:42:25.367289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:25.367289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:42:25.367289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:25.367289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:42:25.367289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:42:25.367289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:25.367289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:42:25.367289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:25.367289 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:42:25.367844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:25.367844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:42:25.367844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:25.367844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:42:25.367844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:42:25.367844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:42:25.367844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:42:25.367844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:42:25.367844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:42:25.367844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:42:25.367844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:25.367844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:25.368375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:25.368375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:42:25.368375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:25.368375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:25.368375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:25.368375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:42:25.368375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:25.368375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:42:25.368375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:42:25.368375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:25.368375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:42:25.368375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:25.368375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:42:25.368375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:42:25.368849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:25.368849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:42:25.368849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:25.368849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:42:25.368849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:25.368849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:42:25.368849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:42:25.368849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:42:25.368849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:42:25.368849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:42:25.368849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:25.368849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:42:25.368849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:42:25.368849 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:42:25.369263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:42:25.369263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:42:25.369263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:25.369263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:25.369263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:42:25.369263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:42:25.369263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:42:25.369263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:42:25.369263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:42:25.369263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:42:25.369263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:42:25.369263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:25.369263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:25.369263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:25.369263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:42:25.369263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:25.369263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:25.369263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:25.369263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:42:25.369861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:25.369861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:42:25.369861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:42:25.369861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:25.369861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:42:25.369861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:25.369861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:42:25.369861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:42:25.369861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:25.369861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:42:25.369861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:25.369861 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:42:25.369861 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:42:25.369861 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:42:25.370322 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:42:25.370322 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:42:25.370322 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:42:25.370322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:25.370759 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:42:25.370759 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:25.370759 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:42:25.370759 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:42:25.370759 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:25.370759 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:42:25.370759 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:25.370759 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:42:25.371508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:25.371508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:42:25.371508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:42:25.371508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:42:25.371508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:25.371508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:42:25.371508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:25.371508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:42:25.371508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:25.371508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:42:25.371508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:25.371508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:42:25.371508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:25.371508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:42:25.371508 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:42:25.371984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:25.371984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:42:25.371984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:25.371984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:42:25.371984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:42:25.371984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:25.371984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:42:25.371984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:25.371984 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:42:25.372341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:25.372341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:42:25.372341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:25.372341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:42:25.372341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:42:25.372341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:42:25.372341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:42:25.372341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:42:25.372341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:42:25.372341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:25.372341 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:25.372774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:25.372774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:42:25.372774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:25.372774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:25.372774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:25.372774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:42:25.372774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:25.372774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:42:25.372774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:42:25.372774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:25.372774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:42:25.372774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:25.372774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:42:25.372774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:42:25.373239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:25.373239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:42:25.373239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:25.373239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:42:25.373239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:25.373239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:42:25.373239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:42:25.373239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:42:25.373239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:42:25.373239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:42:25.373239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:25.373239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:42:25.373239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:42:25.373239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:42:25.373827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:42:25.373827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:42:25.373827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:25.373827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:25.373827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:42:25.373827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:42:25.373827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:42:25.373827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:42:25.373827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:42:25.373827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:42:25.373827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:25.373827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:25.373827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:25.373827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:42:25.373827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:25.373827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:25.373827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:25.373827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:42:25.373827 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:25.374387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:42:25.374387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:42:25.374387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:25.374387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:42:25.374387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:25.374387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:42:25.374387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:42:25.374387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:25.374387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:42:25.374387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:25.374387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:42:25.375184 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:42:25.375577 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:42:25.375737 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:42:25.375954 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:42:25.376353 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:42:25.376762 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:42:25.376990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:25.377702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:42:25.377702 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:25.377702 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:42:25.377702 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:42:25.377702 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:25.377702 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:42:25.377702 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:25.377702 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:42:25.378401 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:25.378401 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:42:25.378401 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:25.378401 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:42:25.378401 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:42:25.378401 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:42:25.378401 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:42:25.378401 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:25.378401 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:25.378401 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:25.378401 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:42:25.378401 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:25.378821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:25.378821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:25.378821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:42:25.378821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:25.378821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:42:25.378821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:42:25.378821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:25.378821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:42:25.378821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:25.378821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:42:25.378821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:42:25.378821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:25.378821 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:42:25.379292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:25.379292 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:42:25.379450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:25.379450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:42:25.379450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:42:25.379450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:42:25.379450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:42:25.379450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:42:25.379450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:25.379450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:42:25.379450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:42:25.379450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:42:25.379450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:42:25.379450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:42:25.379450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:25.379450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:25.379450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:42:25.379450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:42:25.379881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:42:25.379881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:42:25.379881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:25.379881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:25.379881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:25.379881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:42:25.379881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:25.379881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:25.379881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:25.379881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:42:25.379881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:25.379881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:42:25.379881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:42:25.379881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:25.379881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:42:25.379881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:25.379881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:42:25.379881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:42:25.379881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:25.380263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:42:25.380263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:25.380263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:42:30.124097 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:42:30.125077 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:42:30.125077 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:42:30.126280 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:42:30.164549 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:42:30.164549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:30.165058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:42:30.165058 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:30.165058 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:42:30.165058 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:42:30.165058 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:30.165058 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:42:30.165058 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:30.165058 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:42:30.165995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:30.165995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:42:30.165995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:30.165995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:42:30.165995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:42:30.165995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:42:30.165995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:42:30.165995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:30.165995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:30.165995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:30.165995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:42:30.165995 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:30.166978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:30.166978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:30.166978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:42:30.166978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:30.166978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:42:30.166978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:42:30.166978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:30.166978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:42:30.166978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:30.166978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:42:30.166978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:42:30.166978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:30.166978 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:42:30.168107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:30.168107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:42:30.168107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:30.168107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:42:30.168107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:42:30.168107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:42:30.168107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:42:30.168107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:42:30.168107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:30.168107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:42:30.168107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:42:30.168107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:42:30.168107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:42:30.168107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:42:30.168107 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:30.168690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:30.168690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:42:30.168690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:42:30.168690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:42:30.168690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:42:30.168690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:30.168690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:30.168690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:30.168690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:42:30.168690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:30.168690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:30.168690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:30.168690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:42:30.168690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:30.168690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:42:30.168690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:42:30.168690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:30.168690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:42:30.168690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:30.169287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:42:30.169287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:42:30.169287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:30.169287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:42:30.169287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:30.169287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:42:30.356123 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:42:30.356566 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:42:30.356688 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:42:30.357374 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:42:30.357655 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:42:30.357834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:30.358449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:42:30.358449 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:30.358449 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:42:30.358449 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:42:30.358449 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:30.358449 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:42:30.358449 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:30.358449 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:42:30.359393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:30.359393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:42:30.359393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:30.359393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:42:30.359393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:42:30.359393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:42:30.359393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:42:30.359393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:30.359393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:30.359393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:30.359393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:42:30.359393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:30.359752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:30.359752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:30.359752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:42:30.359752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:30.359752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:42:30.359752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:42:30.359752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:30.359752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:42:30.359752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:30.359752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:42:30.359752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:42:30.359752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:30.359752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:42:30.360080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:30.360080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:42:30.360399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:30.360399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:42:30.360399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:42:30.360399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:42:30.360399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:42:30.360399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:42:30.360399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:30.360399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:42:30.360399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:42:30.360399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:42:30.360399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:42:30.360399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:42:30.360399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:30.360399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:30.360399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:42:30.360399 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:42:30.360876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:42:30.360876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:42:30.360876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:30.360876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:30.360876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:30.360876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:42:30.360876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:30.360876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:30.360876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:30.360876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:42:30.360876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:30.360876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:42:30.360876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:42:30.360876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:30.360876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:42:30.360876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:30.360876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:42:30.360876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:42:30.360876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:30.361387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:42:30.361387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:30.361387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:42:30.363558 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:42:30.363904 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:42:30.364088 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:42:30.364287 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:42:30.364774 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:42:30.365112 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:30.365888 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:42:30.365888 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:30.365888 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:42:30.365888 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:42:30.365888 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:30.365888 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:42:30.365888 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:30.365888 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:42:30.366639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:30.366639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:42:30.366639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:42:30.366639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:42:30.366639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:30.366639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:42:30.366639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:30.366639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:42:30.366639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:30.366639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:42:30.366639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:30.366639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:42:30.366639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:30.366639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:42:30.366639 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:42:30.367109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:30.367109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:42:30.367109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:30.367109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:42:30.367109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:42:30.367109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:30.367109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:42:30.367109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:30.367109 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:42:30.367450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:30.367450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:42:30.367450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:30.367450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:42:30.367450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:42:30.367450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:42:30.367450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:42:30.367450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:42:30.367450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:42:30.367450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:42:30.367450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:30.367450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:30.367813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:30.367813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:42:30.367813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:30.367813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:30.367813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:30.367813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:42:30.367813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:30.367813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:42:30.367813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:42:30.367813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:30.367813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:42:30.367813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:30.367813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:42:30.367813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:42:30.368203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:30.368203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:42:30.368203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:30.368203 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:42:30.368418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:30.368418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:42:30.368418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:42:30.368418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:42:30.368418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:42:30.368418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:42:30.368418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:30.368418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:42:30.368418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:42:30.368418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:42:30.368418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:42:30.368418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:42:30.368418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:30.368418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:30.368418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:42:30.368418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:42:30.368857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:42:30.368857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:42:30.368857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:42:30.368857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:42:30.368857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:42:30.368857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:30.368857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:30.368857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:30.368857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:42:30.368857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:30.368857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:30.368857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:30.368857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:42:30.368857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:30.368857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:42:30.368857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:42:30.368857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:30.368857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:42:30.369369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:30.369369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:42:30.369369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:42:30.369369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:30.369369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:42:30.369369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:30.369369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:42:30.369369 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:42:30.369667 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:42:30.369727 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:42:30.369853 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:42:30.370296 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:42:30.370713 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:30.371416 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:42:30.371416 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:30.371416 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:42:30.371416 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:42:30.371416 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:30.371416 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:42:30.371416 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:30.371416 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:42:30.372140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:30.372140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:42:30.372140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:42:30.372140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:42:30.372140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:30.372140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:42:30.372140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:30.372140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:42:30.372140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:30.372140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:42:30.372140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:30.372140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:42:30.372140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:30.372140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:42:30.372140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:42:30.372550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:30.372550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:42:30.372550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:30.372550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:42:30.372550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:42:30.372550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:30.372550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:42:30.372550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:30.372550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:42:30.372989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:30.372989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:42:30.372989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:30.372989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:42:30.372989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:42:30.372989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:42:30.372989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:42:30.372989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:42:30.372989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:42:30.372989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:30.372989 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:30.373305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:30.373305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:42:30.373305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:30.373305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:30.373305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:30.373305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:42:30.373305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:30.373305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:42:30.373305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:42:30.373305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:30.373305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:42:30.373305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:30.373305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:42:30.373305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:42:30.373729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:30.373729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:42:30.373729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:30.373729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:42:30.374057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:30.374057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:42:30.374057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:42:30.374057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:42:30.374057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:42:30.374057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:42:30.374057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:30.374057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:42:30.374057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:42:30.374057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:42:30.374057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:42:30.374057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:42:30.374057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:30.374057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:30.374057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:42:30.374057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:42:30.374057 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:42:30.374590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:42:30.374590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:42:30.374590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:42:30.374590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:30.374590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:30.374590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:30.374590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:42:30.374590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:30.374590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:30.374590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:30.374590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:42:30.374590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:30.374590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:42:30.374590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:42:30.374590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:30.374590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:42:30.374590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:30.374590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:42:30.374590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:42:30.375075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:30.375075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:42:30.375075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:30.375075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:42:30.375571 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:42:30.375795 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:42:30.375901 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:42:30.376044 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:42:30.376347 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:42:30.376586 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:42:30.376727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:30.377189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:42:30.377189 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:30.377189 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:42:30.377189 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:42:30.377189 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:30.377189 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:42:30.377189 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:30.377189 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:42:30.378009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:30.378009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:42:30.378009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:30.378009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:42:30.378009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:42:30.378009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:42:30.378009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:42:30.378009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:30.378009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:30.378009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:30.378009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:42:30.378009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:30.378361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:30.378361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:30.378361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:42:30.378361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:30.378361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:42:30.378361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:42:30.378361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:30.378361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:42:30.378361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:30.378361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:42:30.378361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:42:30.378361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:30.378361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:42:30.378733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:30.378733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:42:30.378939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:30.378939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:42:30.378939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:42:30.378939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:42:30.378939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:42:30.378939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:42:30.378939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:30.378939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:42:30.378939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:42:30.378939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:42:30.378939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:42:30.378939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:42:30.378939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:30.378939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:30.378939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:42:30.378939 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:42:30.379360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:42:30.379360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:42:30.379360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:30.379360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:30.379360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:30.379360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:42:30.379360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:30.379360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:30.379360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:30.379360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:42:30.379360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:30.379360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:42:30.379360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:42:30.379360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:30.379360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:42:30.379360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:30.379360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:42:30.379360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:42:30.379360 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:30.379893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:42:30.379893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:30.379893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:42:35.124983 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:42:35.125767 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:42:35.125767 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:42:35.126835 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:42:35.160384 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:42:35.160684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:35.161335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:42:35.161335 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:35.161335 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:42:35.161335 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:42:35.161335 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:35.161335 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:42:35.161335 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:35.161335 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:42:35.162355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:35.162355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:42:35.162355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:35.162355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:42:35.162355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:42:35.162355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:42:35.162355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:42:35.162355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:35.162355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:35.162355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:35.162355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:42:35.162355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:35.162854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:35.162854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:35.162854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:42:35.162854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:35.162854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:42:35.162854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:42:35.162854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:35.162854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:42:35.162854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:35.162854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:42:35.162854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:42:35.162854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:35.162854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:42:35.163356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:35.163356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:42:35.163474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:35.163474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:42:35.163474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:42:35.163474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:42:35.163474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:42:35.163474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:42:35.163474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:35.163474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:42:35.163474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:42:35.163474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:42:35.163474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:42:35.163474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:42:35.163474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:35.163474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:35.163474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:42:35.163474 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:42:35.164100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:42:35.164100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:42:35.164100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:35.164100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:35.164100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:35.164100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:42:35.164100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:35.164100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:35.164100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:35.164100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:42:35.164100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:35.164100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:42:35.164100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:42:35.164100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:35.164100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:42:35.164100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:35.164100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:42:35.164100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:42:35.164100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:35.164829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:42:35.164829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:35.164829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:42:35.357143 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:42:35.357715 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:42:35.357854 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:42:35.358919 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:42:35.359270 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:42:35.359543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:35.360452 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:42:35.360452 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:35.360452 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:42:35.360452 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:42:35.360452 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:35.360452 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:42:35.360452 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:35.360452 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:42:35.362016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:35.362016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:42:35.362016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:35.362016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:42:35.362016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:42:35.362016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:42:35.362016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:42:35.362016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:35.362016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:35.362016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:35.362016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:42:35.362016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:35.362516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:35.362516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:35.362516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:42:35.362516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:35.362516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:42:35.362516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:42:35.362516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:35.362516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:42:35.362516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:35.362516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:42:35.362516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:42:35.362516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:35.362516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:42:35.363207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:35.363207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:42:35.363654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:35.363654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:42:35.363654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:42:35.363654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:42:35.363654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:42:35.363654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:42:35.363654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:35.363654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:42:35.363654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:42:35.363654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:42:35.363654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:42:35.363654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:42:35.363654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:35.363654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:35.363654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:42:35.363654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:42:35.364162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:42:35.364162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:42:35.364162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:35.364162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:35.364162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:35.364162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:42:35.364162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:35.364162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:35.364162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:35.364162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:42:35.364162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:35.364162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:42:35.364162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:42:35.364162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:35.364162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:42:35.364162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:35.364162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:42:35.364162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:42:35.364162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:35.364772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:42:35.364772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:35.364772 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:42:35.364772 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:42:35.364772 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:42:35.364772 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:42:35.364772 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:42:35.365186 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:42:35.365386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:35.366050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:42:35.366050 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:35.366050 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:42:35.366050 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:42:35.366050 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:35.366050 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:42:35.366050 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:35.366050 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:42:35.367221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:35.367221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:42:35.367221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:42:35.367221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:42:35.367221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:35.367221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:42:35.367221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:35.367221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:42:35.367221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:35.367221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:42:35.367221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:35.367221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:42:35.367221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:35.367221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:42:35.367221 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:42:35.367697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:35.367697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:42:35.367697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:35.367697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:42:35.367697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:42:35.367697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:35.367697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:42:35.367697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:35.367697 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:42:35.368541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:35.368541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:42:35.368541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:35.368541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:42:35.368541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:42:35.368541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:42:35.368541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:42:35.368541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:42:35.368541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:42:35.368541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:42:35.368541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:35.368541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:35.368926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:35.368926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:42:35.368926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:35.368926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:35.368926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:35.368926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:42:35.368926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:35.368926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:42:35.368926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:42:35.368926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:35.368926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:42:35.368926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:35.368926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:42:35.368926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:42:35.369321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:35.369321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:42:35.369321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:35.369321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:42:35.369813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:35.369813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:42:35.369813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:42:35.369813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:42:35.369813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:42:35.369813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:42:35.369813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:35.369813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:42:35.369813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:42:35.369813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:42:35.369813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:42:35.369813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:42:35.369813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:35.369813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:35.369813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:42:35.369813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:42:35.370349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:42:35.370349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:42:35.370349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:42:35.370349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:42:35.370349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:42:35.370349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:35.370349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:35.370349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:35.370349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:42:35.370349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:35.370349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:35.370349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:35.370349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:42:35.370349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:35.370349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:42:35.370349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:42:35.370349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:35.370349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:42:35.370915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:35.370915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:42:35.370915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:42:35.370915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:35.370915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:42:35.370915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:35.370915 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:42:35.370915 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:42:35.370915 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:42:35.370915 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:42:35.371352 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:42:35.371352 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:42:35.372127 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:35.372814 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:42:35.372814 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:35.372814 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:42:35.372814 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:42:35.372814 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:35.372814 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:42:35.372814 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:35.372814 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:42:35.373321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:35.373321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:42:35.373321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:42:35.373321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:42:35.373321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:35.373321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:42:35.373321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:35.373321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:42:35.373321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:35.373321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:42:35.373321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:35.373321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:42:35.373321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:35.373321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:42:35.373321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:42:35.373755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:35.373755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:42:35.373755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:35.373755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:42:35.373755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:42:35.373755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:35.373755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:42:35.373755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:35.373755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:42:35.374058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:35.374058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:42:35.374058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:35.374058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:42:35.374058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:42:35.374058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:42:35.374058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:42:35.374058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:42:35.374058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:42:35.374058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:35.374058 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:35.374449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:35.374449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:42:35.374449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:35.374449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:35.374449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:35.374449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:42:35.374449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:35.374449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:42:35.374449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:42:35.374449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:35.374449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:42:35.374449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:35.374449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:42:35.374449 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:42:35.375053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:35.375053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:42:35.375053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:35.375053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:42:35.375053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:35.375053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:42:35.375053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:42:35.375053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:42:35.375053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:42:35.375053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:42:35.375053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:35.375053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:42:35.375053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:42:35.375053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:42:35.375655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:42:35.375655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:42:35.375655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:35.375655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:35.375655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:42:35.375655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:42:35.375655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:42:35.375655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:42:35.375655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:42:35.375655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:42:35.375655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:35.375655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:35.375655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:35.375655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:42:35.375655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:35.375655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:35.375655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:35.375655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:42:35.375655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:35.376369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:42:35.376369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:42:35.376369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:35.376369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:42:35.376369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:35.376369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:42:35.376369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:42:35.376369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:35.376369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:42:35.376369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:35.376369 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:42:35.376369 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:42:35.376369 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:42:35.376821 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:42:35.376821 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:42:35.376821 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:42:35.377534 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:42:35.377702 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:35.378319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:42:35.378319 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:35.378319 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:42:35.378319 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:42:35.378319 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:35.378319 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:42:35.378319 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:35.378319 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:42:35.379135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:35.379135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:42:35.379135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:35.379135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:42:35.379135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:42:35.379135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:42:35.379135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:42:35.379135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:35.379135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:35.379135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:35.379135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:42:35.379135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:35.379551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:35.379551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:35.379551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:42:35.379551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:35.379551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:42:35.379551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:42:35.379551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:35.379551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:42:35.379551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:35.379551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:42:35.379551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:42:35.379551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:35.379551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:42:35.379905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:35.379905 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:42:35.380056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:35.380056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:42:35.380056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:42:35.380056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:42:35.380056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:42:35.380056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:42:35.380056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:35.380056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:42:35.380056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:42:35.380056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:42:35.380056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:42:35.380056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:42:35.380056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:35.380056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:35.380056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:42:35.380056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:42:35.380597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:42:35.380597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:42:35.380597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:35.380597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:35.380597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:35.380597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:42:35.380597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:35.380597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:35.380597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:35.380597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:42:35.380597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:35.380597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:42:35.380597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:42:35.380597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:35.380597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:42:35.380597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:35.380597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:42:35.380597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:42:35.380597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:35.381338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:42:35.381338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:35.381338 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:42:40.125644 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:42:40.126560 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:42:40.126560 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:42:40.127691 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:42:40.159993 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:42:40.160400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:40.161228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:42:40.161228 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:40.161228 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:42:40.161228 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:42:40.161228 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:40.161228 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:42:40.161228 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:40.161228 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:42:40.162166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:40.162166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:42:40.162166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:40.162166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:42:40.162166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:42:40.162166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:42:40.162166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:42:40.162166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:40.162166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:40.162166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:40.162166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:42:40.162166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:40.162564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:40.162564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:40.162564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:42:40.162564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:40.162564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:42:40.162564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:42:40.162564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:40.162564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:42:40.162564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:40.162564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:42:40.162564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:42:40.162564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:40.162564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:42:40.162963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:40.162963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:42:40.163192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:40.163192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:42:40.163192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:42:40.163192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:42:40.163192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:42:40.163192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:42:40.163192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:40.163192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:42:40.163192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:42:40.163192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:42:40.163192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:42:40.163192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:42:40.163192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:40.163192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:40.163192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:42:40.163192 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:42:40.163780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:42:40.163780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:42:40.163780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:40.163780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:40.163780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:40.163780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:42:40.163780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:40.163780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:40.163780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:40.163780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:42:40.163780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:40.163780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:42:40.163780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:42:40.163780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:40.163780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:42:40.163780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:40.163780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:42:40.163780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:42:40.163780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:40.164356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:42:40.164356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:40.164356 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:42:40.357827 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:42:40.358694 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:42:40.358884 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:42:40.360235 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:42:40.360599 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:42:40.360836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:40.361737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:42:40.361737 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:40.361737 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:42:40.361737 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:42:40.361737 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:40.361737 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:42:40.361737 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:40.361737 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:42:40.363322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:40.363322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:42:40.363322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:40.363322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:42:40.363322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:42:40.363322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:42:40.363322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:42:40.363322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:40.363322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:40.363322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:40.363322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:42:40.363322 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:40.363977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:40.363977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:40.363977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:42:40.363977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:40.363977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:42:40.363977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:42:40.363977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:40.363977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:42:40.363977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:40.363977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:42:40.363977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:42:40.363977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:40.363977 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:42:40.364626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:40.364626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:42:40.364626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:40.364626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:42:40.364626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:42:40.364626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:42:40.364626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:42:40.364626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:42:40.364626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:40.364626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:42:40.364626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:42:40.364626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:42:40.364626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:42:40.364626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:42:40.364626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:40.365315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:40.365315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:42:40.365315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:42:40.365315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:42:40.365315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:42:40.365315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:40.365315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:40.365315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:40.365315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:42:40.365315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:40.365315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:40.365315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:40.365315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:42:40.365315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:40.365315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:42:40.365315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:42:40.365315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:40.365315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:42:40.365315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:40.366393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:42:40.366393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:42:40.366393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:40.366393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:42:40.366393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:40.366393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:42:40.366393 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:42:40.366393 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:42:40.366393 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:42:40.366393 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:42:40.367249 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:42:40.367249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:40.367249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:42:40.367249 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:40.367249 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:42:40.367249 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:42:40.367249 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:40.367249 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:42:40.367249 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:40.367249 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:42:40.367801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:40.367801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:42:40.367801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:42:40.367801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:42:40.367801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:40.367801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:42:40.367801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:40.367801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:42:40.367801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:40.367801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:42:40.367801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:40.367801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:42:40.367801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:40.367801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:42:40.367801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:42:40.368691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:40.368691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:42:40.368691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:40.368691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:42:40.368691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:42:40.368691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:40.368691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:42:40.368691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:40.368691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:42:40.368691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:40.368691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:42:40.368691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:40.368691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:42:40.368691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:42:40.370202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:42:40.370202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:42:40.370202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:42:40.370202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:42:40.370202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:42:40.370202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:40.370202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:40.370202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:40.370202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:42:40.370202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:40.370202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:40.370202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:40.370202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:42:40.371086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:40.371086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:42:40.371086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:42:40.371086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:40.371086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:42:40.371086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:40.371086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:42:40.371086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:42:40.371086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:40.371086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:42:40.371086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:40.371086 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:42:40.371603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:40.371603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:42:40.371603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:42:40.371603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:42:40.371603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:42:40.371603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:42:40.371603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:40.371603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:42:40.371603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:42:40.371603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:42:40.371603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:42:40.371603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:42:40.371603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:40.371603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:40.371603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:42:40.371603 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:42:40.372022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:42:40.372022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:42:40.372022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:42:40.372022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:42:40.372022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:42:40.372022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:40.372022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:40.372022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:40.372022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:42:40.372022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:40.372022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:40.372022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:40.372022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:42:40.372022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:40.372022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:42:40.372022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:42:40.372022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:40.372022 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:42:40.372524 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:40.372524 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:42:40.372524 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:42:40.372524 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:40.372524 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:42:40.372524 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:40.372524 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:42:40.372524 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:42:40.372524 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:42:40.372524 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:42:40.372824 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:42:40.372824 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:42:40.372824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:40.372824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:42:40.372824 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:40.372824 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:42:40.372824 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:42:40.372824 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:40.372824 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:42:40.372824 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:40.372824 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:42:40.372824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:40.373155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:42:40.373155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:42:40.373155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:42:40.373155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:40.373155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:42:40.373155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:40.373155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:42:40.373155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:40.373155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:42:40.373155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:40.373155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:42:40.373155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:40.373155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:42:40.373155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:42:40.373155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:40.373155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:42:40.373155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:40.373600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:42:40.373600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:42:40.373600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:40.373600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:42:40.373600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:40.373600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:42:40.373600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:40.373600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:42:40.373600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:40.373600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:42:40.373600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:42:40.373600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:42:40.373600 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:42:40.373967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:42:40.373967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:42:40.373967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:40.373967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:40.373967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:40.373967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:42:40.373967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:40.373967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:40.373967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:40.373967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:42:40.373967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:40.373967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:42:40.373967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:42:40.374635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:40.374635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:42:40.374635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:40.374635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:42:40.374635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:42:40.374635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:40.374635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:42:40.374635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:40.374635 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:42:40.374909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:40.374909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:42:40.374909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:42:40.374909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:42:40.374909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:42:40.374909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:42:40.374909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:40.374909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:42:40.374909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:42:40.374909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:42:40.374909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:42:40.374909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:42:40.374909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:40.374909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:40.374909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:42:40.374909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:42:40.374909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:42:40.375418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:42:40.375418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:42:40.375418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:42:40.375418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:40.375418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:40.375418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:40.375418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:42:40.375418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:40.375418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:40.375418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:40.375418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:42:40.375418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:40.375418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:42:40.375418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:42:40.375418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:40.375418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:42:40.375418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:40.375418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:42:40.375418 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:42:40.376009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:40.376009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:42:40.376009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:40.376009 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:42:40.376384 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:42:40.376645 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:42:40.376750 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:42:40.376925 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:42:40.377343 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:42:40.377965 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:42:40.378200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:40.379020 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:42:40.379020 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:40.379020 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:42:40.379020 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:42:40.379020 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:40.379020 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:42:40.379020 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:40.379020 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:42:40.379722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:40.379722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:42:40.379722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:40.379722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:42:40.379722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:42:40.379722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:42:40.379722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:42:40.379722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:40.379722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:40.379722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:40.379722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:42:40.379722 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:40.380424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:40.380424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:40.380424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:42:40.380424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:40.380424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:42:40.380424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:42:40.380424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:40.380424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:42:40.380424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:40.380424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:42:40.380424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:42:40.380424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:40.380424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:42:40.380810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:40.380810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:42:40.380810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:40.380810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:42:40.380810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:42:40.380810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:42:40.380810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:42:40.380810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:42:40.380810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:40.380810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:42:40.380810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:42:40.380810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:42:40.380810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:42:40.380810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:42:40.380810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:40.381243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:40.381243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:42:40.381243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:42:40.381243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:42:40.381243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:42:40.381243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:40.381243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:40.381243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:40.381243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:42:40.381243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:40.381243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:40.381243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:40.381243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:42:40.381243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:40.381243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:42:40.381243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:42:40.381243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:40.381243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:42:40.381243 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:40.381747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:42:40.381747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:42:40.381747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:40.381747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:42:40.381747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:40.381747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:42:45.126222 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:42:45.127092 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:42:45.127092 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:42:45.127347 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:42:45.155916 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:42:45.156196 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:45.156931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:42:45.156931 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:45.156931 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:42:45.156931 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:42:45.156931 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:45.156931 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:42:45.156931 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:45.156931 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:42:45.157751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.030 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:45.157751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:42:45.157751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:45.157751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:42:45.157751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:42:45.157751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:42:45.157751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:42:45.157751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:45.157751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:45.157751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:45.157751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:42:45.157751 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:45.158355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:45.158355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:45.158355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:42:45.158355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:45.158355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:42:45.158355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:42:45.158355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:45.158355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:42:45.158355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:45.158355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:42:45.158355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:42:45.158355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:45.158355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:42:45.158825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:45.158825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:42:45.158825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:45.158825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:42:45.158825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:42:45.158825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:42:45.158825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:42:45.158825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:42:45.158825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:45.158825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:42:45.158825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:42:45.158825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:42:45.158825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:42:45.158825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:42:45.158825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:45.159295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:45.159295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:42:45.159295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:42:45.159295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:42:45.159295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:42:45.159295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:45.159295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:45.159295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:45.159295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:42:45.159295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:45.159295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:45.159295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:45.159295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:42:45.159295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:45.159295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:42:45.159295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:42:45.159295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:45.159295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:42:45.159295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:45.159971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:42:45.159971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:42:45.159971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:45.159971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:42:45.159971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:45.159971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:42:45.359028 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:42:45.359337 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:42:45.359479 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:42:45.360436 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:42:45.360872 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:42:45.361154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:45.362101 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:42:45.362101 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:45.362101 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:42:45.362101 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:42:45.362101 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:45.362101 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:42:45.362101 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:45.362101 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:42:45.362901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:45.362901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:42:45.362901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:45.362901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:42:45.362901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:42:45.362901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:42:45.362901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:42:45.362901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:45.362901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:45.362901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:45.362901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:42:45.362901 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:45.363538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:45.363538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:45.363538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:42:45.363538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:45.363538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:42:45.363538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:42:45.363538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:45.363538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:42:45.363538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:45.363538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:42:45.363538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:42:45.363538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:45.363538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:42:45.364168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:45.364168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:42:45.364168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:45.364168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:42:45.364168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:42:45.364168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:42:45.364168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:42:45.364168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:42:45.364168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:45.364168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:42:45.364168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:42:45.364168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:42:45.364168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:42:45.364168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:42:45.364168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:45.364906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:45.364906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:42:45.364906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:42:45.364906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:42:45.364906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:42:45.364906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:45.364906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:45.364906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:45.364906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:42:45.364906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:45.364906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:45.364906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:45.364906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:42:45.364906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:45.364906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:42:45.364906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:42:45.364906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:45.364906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:42:45.364906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:45.365737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:42:45.365737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:42:45.365737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:45.365737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:42:45.365737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:45.365737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:42:45.365737 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:42:45.365737 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:42:45.365737 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:42:45.365737 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:42:45.366183 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:42:45.366183 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:45.366752 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:42:45.366752 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:45.366752 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:42:45.366752 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:42:45.366752 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:45.366752 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:42:45.366752 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:45.366752 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:42:45.367281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:45.367281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:42:45.367281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:42:45.367281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:42:45.367281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:45.367281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:42:45.367281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:45.367281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:42:45.367281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:45.367281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:42:45.367281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:45.367281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:42:45.367281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:45.367281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:42:45.367281 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:42:45.368021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:45.368021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:42:45.368021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:45.368021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:42:45.368021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:42:45.368021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:45.368021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:42:45.368021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:45.368021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:42:45.368021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:45.368021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:42:45.368021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:45.368021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:42:45.368021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:42:45.368644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:42:45.368644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:42:45.368644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:42:45.368644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:42:45.368644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:42:45.368644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:45.368644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:45.368644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:45.368644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:42:45.368644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:45.368644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:45.368644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:45.368644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:42:45.369226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:45.369226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:42:45.369226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:42:45.369226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:45.369226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:42:45.369226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:45.369226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:42:45.369226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:42:45.369226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:45.369226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:42:45.369226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:45.369226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:42:45.369780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:45.369780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:42:45.369780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:42:45.369780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:42:45.369780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:42:45.369780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:42:45.369780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:45.369780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:42:45.369780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:42:45.369780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:42:45.369780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:42:45.369780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:42:45.369780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:45.369780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:45.369780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:42:45.369780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:42:45.370564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:42:45.370564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:42:45.370564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:42:45.370564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:42:45.370564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:42:45.370564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:45.370564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:45.370564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:45.370564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:42:45.370564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:45.370564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:45.370564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:45.370564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:42:45.370564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:45.370564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:42:45.370564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:42:45.370564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:45.370564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:42:45.371439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:45.371439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:42:45.371439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:42:45.371439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:45.371439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:42:45.371439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:45.371439 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:42:45.371439 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:42:45.371439 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:42:45.371439 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:42:45.371976 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:42:45.371976 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:42:45.371976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:45.372377 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:42:45.372377 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:45.372377 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:42:45.372377 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:42:45.372377 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:45.372377 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:42:45.372377 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:45.372377 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:42:45.373016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:45.373016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:42:45.373016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:42:45.373016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:42:45.373016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:45.373016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:42:45.373016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:45.373016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:42:45.373016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:45.373016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:42:45.373016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:45.373016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:42:45.373016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:45.373016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:42:45.373016 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:42:45.373313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:45.373313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:42:45.373313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:45.373313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:42:45.373313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:42:45.373313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:45.373313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:42:45.373313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:45.373313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:42:45.373651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:45.373651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:42:45.373651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:45.373651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:42:45.373651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:42:45.373651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:42:45.373651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:42:45.373651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:42:45.373651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:42:45.373651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:45.373651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:45.373881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:45.373881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:42:45.373881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:45.373881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:45.373881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:45.373881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:42:45.373881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:45.373881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:42:45.373881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:42:45.373881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:45.373881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:42:45.373881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:45.373881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:42:45.373881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:42:45.374151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:45.374151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:42:45.374151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:45.374151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:42:45.374638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:45.374638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:42:45.374638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:42:45.374638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:42:45.374638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:42:45.374638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:42:45.374638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:45.374638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:42:45.374638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:42:45.374638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:42:45.374638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:42:45.374638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:42:45.374638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:45.374638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:45.374638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:42:45.374638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:42:45.374638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:42:45.374991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:42:45.374991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:42:45.374991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:42:45.374991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:45.374991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:45.374991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:45.374991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:42:45.374991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:45.374991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:45.374991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:45.374991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:42:45.374991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:45.374991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:42:45.374991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:42:45.374991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:45.374991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:42:45.374991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:45.374991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:42:45.374991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:42:45.375361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:45.375361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:42:45.375361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:45.375361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:42:45.376710 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:42:45.376880 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:42:45.376957 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:42:45.377065 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:42:45.377316 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:42:45.377559 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:42:45.377694 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:45.378072 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:42:45.378072 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:45.378072 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:42:45.378072 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:42:45.378072 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:45.378072 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:42:45.378072 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:45.378072 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:42:45.378670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:45.378670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:42:45.378670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:45.378670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:42:45.378670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:42:45.378670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:42:45.378670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:42:45.378670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:45.378670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:45.378670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:45.378670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:42:45.378670 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:45.378931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:45.378931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:45.378931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:42:45.378931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:45.378931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:42:45.378931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:42:45.378931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:45.378931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:42:45.378931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:45.378931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:42:45.378931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:42:45.378931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:45.378931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:42:45.379186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:45.379186 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:42:45.379335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:45.379335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:42:45.379335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:42:45.379335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:42:45.379335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:42:45.379335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:42:45.379335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:45.379335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:42:45.379335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:42:45.379335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:42:45.379335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:42:45.379335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:42:45.379335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:45.379335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:45.379335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:42:45.379335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:42:45.379676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:42:45.379676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:42:45.379676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:45.379676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:45.379676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:45.379676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:42:45.379676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:45.379676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:45.379676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:45.379676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:42:45.379676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:45.379676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:42:45.379676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:42:45.379676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:45.379676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:42:45.379676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:45.379676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:42:45.379676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:42:45.379676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:45.380144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:42:45.380144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:45.380144 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:42:50.126863 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:42:50.127948 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:42:50.127948 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:42:50.131652 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:42:50.167804 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:42:50.168148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:50.168963 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:42:50.168963 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:50.168963 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:42:50.168963 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:42:50.168963 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:50.168963 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:42:50.168963 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:50.168963 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:42:50.170195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:50.170195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:42:50.170195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:50.170195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:42:50.170195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:42:50.170195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:42:50.170195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:42:50.170195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:50.170195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:50.170195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:50.170195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:42:50.170195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:50.171197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:50.171197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:50.171197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:42:50.171197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:50.171197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:42:50.171197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:42:50.171197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:50.171197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:42:50.171197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:50.171197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:42:50.171197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:42:50.171197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:50.171197 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:42:50.172102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:50.172102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:42:50.172102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:50.172102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:42:50.172102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:42:50.172102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:42:50.172102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:42:50.172102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:42:50.172102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:50.172102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:42:50.172102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:42:50.172102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:42:50.172102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:42:50.172102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:42:50.172102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:50.174137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:50.174137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:42:50.174137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:42:50.174137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:42:50.174137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:42:50.174137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:50.174137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:50.174137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:50.174137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:42:50.174137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:50.174137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:50.174137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:50.174137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:42:50.174137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:50.174137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:42:50.174137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:42:50.174137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:50.174137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:42:50.174137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:50.175545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:42:50.175545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:42:50.175545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:50.175545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:42:50.175545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:50.175545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:42:50.359544 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:42:50.360169 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:42:50.360484 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:42:50.361596 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:42:50.362096 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:42:50.362266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:50.363074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:42:50.363074 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:50.363074 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:42:50.363074 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:42:50.363074 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:50.363074 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:42:50.363074 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:50.363074 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:42:50.365045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:50.365045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:42:50.365045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:50.365045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:42:50.365045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:42:50.365045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:42:50.365045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:42:50.365045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:50.365045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:50.365045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:50.365045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:42:50.365045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:50.366049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:50.366049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:50.366049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:42:50.366049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:50.366049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:42:50.366049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:42:50.366049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:50.366049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:42:50.366049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:50.366049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:42:50.366049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:42:50.366049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:50.366049 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:42:50.366739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:50.366739 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:42:50.367198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:50.367198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:42:50.367198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:42:50.367198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:42:50.367198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:42:50.367198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:42:50.367198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:50.367198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:42:50.367198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:42:50.367198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:42:50.367198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:42:50.367198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:42:50.367198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:50.367198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:50.367198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:42:50.367198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:42:50.369176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:42:50.369176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:42:50.369176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:50.369176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:50.369176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:50.369176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:42:50.369176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:50.369176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:50.369176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:50.369176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:42:50.369176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:50.369176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:42:50.369176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:42:50.369176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:50.369176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:42:50.369176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:50.369176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:42:50.369176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:42:50.369176 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:50.370590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:42:50.370590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:50.370590 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:42:50.370590 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:42:50.370590 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:42:50.370590 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:42:50.370590 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:42:50.370590 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:42:50.371202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:50.371202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:42:50.371202 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:50.371202 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:42:50.371202 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:42:50.371202 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:50.371202 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:42:50.371202 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:50.371202 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:42:50.371202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:50.371202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:42:50.371202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:42:50.371202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:42:50.371202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:50.371202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:42:50.371202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:50.371202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:42:50.372386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:50.372386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:42:50.372386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:50.372386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:42:50.372386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:50.372386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:42:50.372386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:42:50.372386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:50.372386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:42:50.372386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:50.372386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:42:50.372386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:42:50.372386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:50.372386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:42:50.372386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:50.372386 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:42:50.373684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:50.373684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:42:50.373684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:50.373684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:42:50.373684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:42:50.373684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:42:50.373684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:42:50.373684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:42:50.373684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:42:50.373684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:42:50.373684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:50.373684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:50.374126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:50.374126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:42:50.374126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:50.374126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:50.374126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:50.374126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:42:50.374126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:50.374126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:42:50.374126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:42:50.374126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:50.374126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:42:50.374126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:50.374126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:42:50.374126 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:42:50.374506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:50.374506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:42:50.374506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:50.374506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:42:50.374506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:50.374506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:42:50.374506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:42:50.374506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:42:50.374506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:42:50.374506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:42:50.374506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:50.374506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:42:50.374506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:42:50.374506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:42:50.374895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:42:50.374895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:42:50.374895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:50.374895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:50.374895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:42:50.374895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:42:50.374895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:42:50.374895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:42:50.374895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:42:50.374895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:42:50.374895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:42:50.374895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:50.374895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:50.374895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:50.374895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:42:50.374895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:50.374895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:50.374895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:50.374895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:42:50.375429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:50.375429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:42:50.375429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:42:50.375429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:50.375429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:42:50.375429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:50.375429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:42:50.375429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:42:50.375429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:50.375429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:42:50.375429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:50.375429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:42:50.375429 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:42:50.375429 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:42:50.375829 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:42:50.375829 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:42:50.375829 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:42:50.375829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:50.375829 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:42:50.375829 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:50.375829 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:42:50.375829 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:42:50.375829 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:50.375829 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:42:50.375829 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:50.375829 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:42:50.376732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:50.376732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:42:50.376732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:42:50.376732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:42:50.376732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:50.376732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:42:50.376732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:50.376732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:42:50.376732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:50.376732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:42:50.376732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:50.376732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:42:50.376732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:50.376732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:42:50.376732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:42:50.377251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:50.377251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:42:50.377251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:50.377251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:42:50.377251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:42:50.377251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:50.377251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:42:50.377251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:50.377251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:42:50.378549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:50.378549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:42:50.378549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:50.378549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:42:50.378549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:42:50.378549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:42:50.378549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:42:50.378549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:42:50.378549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:42:50.378549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:50.378549 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:50.378899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:50.378899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:42:50.378899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:50.378899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:50.378899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:50.378899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:42:50.378899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:50.378899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:42:50.378899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:42:50.378899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:50.378899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:42:50.378899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:50.378899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:42:50.378899 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:42:50.379255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:50.379255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:42:50.379255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:50.379255 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:42:50.379255 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:42:50.381715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:50.381715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:42:50.381715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:42:50.381715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:42:50.381715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:42:50.381715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:42:50.381715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:50.381715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:42:50.381715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:42:50.381715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:42:50.381715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:42:50.381715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:42:50.381715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:50.381715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:50.381715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:42:50.381715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:42:50.381715 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:42:50.382207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:42:50.382207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:42:50.382207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:42:50.382207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:50.382207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:50.382207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:50.382207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:42:50.382207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:50.382207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:50.382207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:50.382207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:42:50.382207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:50.382207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:42:50.382207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:42:50.382207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:50.382207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:42:50.382207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:50.382207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:42:50.382207 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:42:50.382726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:50.382726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:42:50.382726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:50.382726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:42:50.382726 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:42:50.382726 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:42:50.382908 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:42:50.383120 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:42:50.383330 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:42:50.383500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:50.384179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:42:50.384179 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:50.384179 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:42:50.384179 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:42:50.384179 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:50.384179 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:42:50.384179 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:50.384179 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:42:50.385566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:50.385566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:42:50.385566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:50.385566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:42:50.385566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:42:50.385566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:42:50.385566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:42:50.385566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:50.385566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:50.385566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:50.385566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:42:50.385566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:50.385993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:50.385993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:50.385993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:42:50.385993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:50.385993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:42:50.385993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:42:50.385993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:50.385993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:42:50.385993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:50.385993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:42:50.385993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:42:50.385993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:50.385993 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:42:50.386501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:50.386501 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:42:50.387131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:50.387131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:42:50.387131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:42:50.387131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:42:50.387131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:42:50.387131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:42:50.387131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:50.387131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:42:50.387131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:42:50.387131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:42:50.387131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:42:50.387131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:42:50.387131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:50.387131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:50.387131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:42:50.387131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:42:50.387820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:42:50.387820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:42:50.387820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:50.387820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:50.387820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:50.387820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:42:50.387820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:50.387820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:50.387820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:50.387820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:42:50.387820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:50.387820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:42:50.387820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:42:50.387820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:50.387820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:42:50.387820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:50.387820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:42:50.387820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:42:50.387820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:50.388553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:42:50.388553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:50.388553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:42:55.127448 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:42:55.128176 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:42:55.128176 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:42:55.128729 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:42:55.167538 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:42:55.167538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:55.168168 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:42:55.168168 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:55.168168 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:42:55.168168 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:42:55.168168 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:55.168168 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:42:55.168168 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:55.168168 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:42:55.169305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:55.169305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:42:55.169305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:55.169305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:42:55.169305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:42:55.169305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:42:55.169305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:42:55.169305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:55.169305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:55.169305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:55.169305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:42:55.169305 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:55.170816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:55.170816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:55.170816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:42:55.170816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:55.170816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:42:55.170816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:42:55.170816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:55.170816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:42:55.170816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:55.170816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:42:55.170816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:42:55.170816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:55.170816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:42:55.171458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:55.171458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:42:55.171458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:55.171458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:42:55.171458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:42:55.171458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:42:55.171458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:42:55.171458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:42:55.171458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:55.171458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:42:55.171458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:42:55.171458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:42:55.171458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:42:55.171458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:42:55.171458 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:55.172153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:55.172153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:42:55.172153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:42:55.172153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:42:55.172153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:42:55.172153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:55.172153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:55.172153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:55.172153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:42:55.172153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:55.172153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:55.172153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:55.172153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:42:55.172153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:55.172153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:42:55.172153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:42:55.172153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:55.172153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:42:55.172153 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:55.173028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:42:55.173028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:42:55.173028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:55.173028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:42:55.173028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:55.173028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:42:55.360250 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:42:55.360755 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:42:55.360993 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:42:55.361945 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:42:55.362235 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:42:55.362519 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:55.363335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:42:55.363335 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:55.363335 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:42:55.363335 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:42:55.363335 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:55.363335 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:42:55.363335 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:55.363335 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:42:55.365219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:55.365219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:42:55.365219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:55.365219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:42:55.365219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:42:55.365219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:42:55.365219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:42:55.365219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:55.365219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:55.365219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:55.365219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:42:55.365219 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:55.365945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:55.365945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:55.365945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:42:55.365945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:55.365945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:42:55.365945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:42:55.365945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:55.365945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:42:55.365945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:55.365945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:42:55.365945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:42:55.365945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:55.365945 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:42:55.366741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:55.366741 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:42:55.367027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:55.367027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:42:55.367027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:42:55.367027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:42:55.367027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:42:55.367027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:42:55.367027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:55.367027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:42:55.367027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:42:55.367027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:42:55.367027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:42:55.367027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:42:55.367027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:55.367027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:55.367027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:42:55.367027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:42:55.367937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:42:55.367937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:42:55.367937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:55.367937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:55.367937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:55.367937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:42:55.367937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:55.367937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:55.367937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:55.367937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:42:55.367937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:55.367937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:42:55.367937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:42:55.367937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:55.367937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:42:55.367937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:55.367937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:42:55.367937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:42:55.367937 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:55.369140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:42:55.369140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:55.369140 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:42:55.369140 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:42:55.369140 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:42:55.369140 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:42:55.369140 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:42:55.369140 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:42:55.369683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:55.369683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:42:55.369683 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:55.369683 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:42:55.369683 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:42:55.369683 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:55.369683 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:42:55.369683 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:55.369683 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:42:55.370233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:55.370233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:42:55.370233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:42:55.370233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:42:55.370233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:55.370233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:42:55.370233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:55.370233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:42:55.370233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:55.370233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:42:55.370233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:55.370233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:42:55.370233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:55.370233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:42:55.370233 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:42:55.371163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:55.371163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:42:55.371163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:55.371163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:42:55.371163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:42:55.371163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:55.371163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:42:55.371163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:55.371163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:42:55.371163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:55.371163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:42:55.371163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:55.371163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:42:55.371163 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:42:55.372177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:42:55.372177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:42:55.372177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:42:55.372177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:42:55.372177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:42:55.372177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:55.372177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:55.372177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:55.372177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:42:55.372177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:55.372177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:55.372177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:55.372177 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:42:55.373239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:55.373239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:42:55.373239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:42:55.373239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:55.373239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:42:55.373239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:55.373239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:42:55.373239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:42:55.373239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:55.373239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:42:55.373239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:55.373239 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:42:55.374298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:55.374298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:42:55.374298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:42:55.374298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:42:55.374298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:42:55.374298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:42:55.374298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:55.374298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:42:55.374298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:42:55.374298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:42:55.374298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:42:55.374298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:42:55.374298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:55.374298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:55.374298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:42:55.374298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:42:55.375645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:42:55.375645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:42:55.375645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:42:55.375645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:42:55.375645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:42:55.375645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:55.375645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:55.375645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:55.375645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:42:55.375645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:55.375645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:55.375645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:55.375645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:42:55.375645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:55.375645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:42:55.375645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:42:55.375645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:55.375645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:42:55.376602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:55.376602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:42:55.376602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:42:55.376602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:55.376602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:42:55.376602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:55.376602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:42:55.376602 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:42:55.376602 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:42:55.376602 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:42:55.377075 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:42:55.377075 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:42:55.377075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:55.377075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:42:55.377075 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:55.377075 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:42:55.377075 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:42:55.377075 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:55.377075 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:42:55.377075 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:55.377075 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:42:55.377075 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:55.377551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:42:55.377551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:42:55.377551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:42:55.377551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:55.377551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:42:55.377551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:55.377551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:42:55.377551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:55.377551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:42:55.377551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:55.377551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:42:55.377551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:55.377551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:42:55.377551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:42:55.377551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:55.377551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:42:55.377551 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:55.378133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:42:55.378133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:42:55.378133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:55.378133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:42:55.378133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:55.378133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:42:55.378133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:55.378133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:42:55.378133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:55.378133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:42:55.378133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:42:55.378133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:42:55.378133 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:42:55.378780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:42:55.378780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:42:55.378780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:55.378780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:55.378780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:55.378780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:42:55.378780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:55.378780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:55.378780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:55.378780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:42:55.378780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:55.378780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:42:55.378780 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:42:55.379366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:55.379366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:42:55.379366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:55.379366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:42:55.379366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:42:55.379366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:55.379366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:42:55.379366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:55.379366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:42:55.379366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:55.379366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:42:55.379366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:42:55.379912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:42:55.379912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:42:55.379912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:42:55.379912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:55.379912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:42:55.379912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:42:55.379912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:42:55.379912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:42:55.379912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:42:55.379912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:55.379912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:55.379912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:42:55.379912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:42:55.379912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:42:55.379912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:42:55.379912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:42:55.379912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:42:55.379912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:55.379912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:55.380617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:55.380617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:42:55.380617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:55.380617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:55.380617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:55.380617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:42:55.380617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:55.380617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:42:55.380617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:42:55.380617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:55.380617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:42:55.380617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:55.380617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:42:55.380617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:42:55.380617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:55.380617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:42:55.380617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:55.380617 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:42:55.382580 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:42:55.382580 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:42:55.382580 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:42:55.382580 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:42:55.382859 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:42:55.382958 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:42:55.383076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:55.383554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:42:55.383554 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:55.383554 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:42:55.383554 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:42:55.383554 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:55.383554 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:42:55.383554 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:55.383554 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:42:55.384424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:55.384424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:42:55.384424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:55.384424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:42:55.384424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:42:55.384424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:42:55.384424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:42:55.384424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:55.384424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:55.384424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:55.384424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:42:55.384424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:55.384834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:42:55.384834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:55.384834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:42:55.384834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:55.384834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:42:55.384834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:42:55.384834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:55.384834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:42:55.384834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:55.384834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:42:55.384834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:42:55.384834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:55.384834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:42:55.385195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:55.385195 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:42:55.385425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:55.385425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:42:55.385425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:42:55.385425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:42:55.385425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:42:55.385425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:42:55.385425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:55.385425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:42:55.385425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:42:55.385425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:42:55.385425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:42:55.385425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:42:55.385425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:42:55.385425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:42:55.385425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:42:55.385425 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:42:55.385926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:42:55.385926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:42:55.385926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:55.385926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:55.385926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:42:55.385926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:42:55.385926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:42:55.385926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:42:55.385926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:42:55.385926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:42:55.385926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:42:55.385926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:42:55.385926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:42:55.385926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:42:55.385926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:42:55.385926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:42:55.385926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:42:55.385926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:42:55.385926 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:42:55.386529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:42:55.386529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:42:55.386529 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:43:00.128283 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:43:00.129092 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:43:00.129092 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:43:00.130312 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:43:00.163657 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:43:00.163931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:00.164453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:43:00.164453 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:00.164453 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:43:00.164453 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:43:00.164453 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:00.164453 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:43:00.164453 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:00.164453 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:43:00.165373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:00.165373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:43:00.165373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:00.165373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:43:00.165373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:43:00.165373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:43:00.165373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:43:00.165373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:00.165373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:00.165373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:00.165373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:43:00.165373 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:00.165956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:00.165956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:00.165956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:43:00.165956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:00.165956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:43:00.165956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:43:00.165956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:00.165956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:43:00.165956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:00.165956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:43:00.165956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:43:00.165956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:00.165956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:43:00.166488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:00.166488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:43:00.166488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:00.166488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:43:00.166488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:43:00.166488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:43:00.166488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:43:00.166488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:43:00.166488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:00.166488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:43:00.166488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:43:00.166488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:43:00.166488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:43:00.166488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:43:00.166488 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:00.167066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:00.167066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:43:00.167066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:43:00.167066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:43:00.167066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:43:00.167066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:00.167066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:00.167066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:00.167066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:43:00.167066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:00.167066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:00.167066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:00.167066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:43:00.167066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:00.167066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:43:00.167066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:43:00.167066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:00.167066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:43:00.167066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:00.167750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:43:00.167750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:43:00.167750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:00.167750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:43:00.167750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:00.167750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:43:00.360965 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:43:00.362159 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:43:00.362159 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:43:00.362794 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:43:00.363040 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:43:00.363391 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:00.364733 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:43:00.364733 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:00.364733 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:43:00.364733 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:43:00.364733 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:00.364733 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:43:00.364733 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:00.364733 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:43:00.366095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:00.366095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:43:00.366095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:00.366095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:43:00.366095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:43:00.366095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:43:00.366095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:43:00.366095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:00.366095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:00.366095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:00.366095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:43:00.366095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:00.366503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:00.366503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:00.366503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:43:00.366503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:00.366503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:43:00.366503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:43:00.366503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:00.366503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:43:00.366503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:00.366503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:43:00.366503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:43:00.366503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:00.366503 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:43:00.367190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:00.367190 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:43:00.368124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:00.368124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:43:00.368124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:43:00.368124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:43:00.368124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:43:00.368124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:43:00.368124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:00.368124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:43:00.368124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:43:00.368124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:43:00.368124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:43:00.368124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:43:00.368124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:00.368124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:00.368124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:43:00.368124 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:43:00.368615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:43:00.368615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:43:00.368615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:00.368615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:00.368615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:00.368615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:43:00.368615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:00.368615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:00.368615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:00.368615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:43:00.368615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:00.368615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:43:00.368615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:43:00.368615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:00.368615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:43:00.368615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:00.368615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:43:00.368615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:43:00.368615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:00.369135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:43:00.369135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:00.369135 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:43:00.369135 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:43:00.369135 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:43:00.369135 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:43:00.369135 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:43:00.369559 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:43:00.369904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:00.370806 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:43:00.370806 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:00.370806 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:43:00.370806 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:43:00.370806 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:00.370806 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:43:00.370806 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:00.370806 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:43:00.371934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:00.371934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:43:00.371934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:43:00.371934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:43:00.371934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:00.371934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:43:00.371934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:00.371934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:43:00.371934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:00.371934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:43:00.371934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:00.371934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:43:00.371934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:00.371934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:43:00.371934 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:43:00.372394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:00.372394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:43:00.372394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:00.372394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:43:00.372394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:43:00.372394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:00.372394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:43:00.372394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:00.372394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:43:00.372679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:00.372679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:43:00.372679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:00.372679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:43:00.372679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:43:00.372679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:43:00.372679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:43:00.372679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:43:00.372679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:43:00.372679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:43:00.372679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:00.372679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:00.373037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:00.373037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:43:00.373037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:00.373037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:00.373037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:00.373037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:43:00.373037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:00.373037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:43:00.373037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:43:00.373037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:00.373037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:43:00.373037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:00.373037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:43:00.373037 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:43:00.373490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:00.373490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:43:00.373490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:00.373490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:43:00.373490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:00.373490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:43:00.373490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:43:00.373490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:43:00.373490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:43:00.373490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:43:00.373490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:00.373490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:43:00.373490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:43:00.373490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:43:00.373893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:43:00.373893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:43:00.373893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:00.373893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:00.373893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:43:00.373893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:43:00.373893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:43:00.373893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:43:00.373893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:43:00.373893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:43:00.373893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:43:00.373893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:00.373893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:00.373893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:00.373893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:43:00.373893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:00.373893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:00.373893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:00.373893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:43:00.374451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:00.374451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:43:00.374451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:43:00.374451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:00.374451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:43:00.374451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:00.374451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:43:00.374451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:43:00.374451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:00.374451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:43:00.374451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:00.374451 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:43:00.374451 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:43:00.374828 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:43:00.374828 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:43:00.374828 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:43:00.374953 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:43:00.374994 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:00.375645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:43:00.375645 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:00.375645 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:43:00.375645 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:43:00.375645 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:00.375645 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:43:00.375645 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:00.375645 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:43:00.376026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:00.376026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:43:00.376026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:43:00.376026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:43:00.376026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:00.376026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:43:00.376026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:00.376026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:43:00.376026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:00.376026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:43:00.376026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:00.376026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:43:00.376026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:00.376026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:43:00.376026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:43:00.376500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:00.376500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:43:00.376500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:00.376500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:43:00.376500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:43:00.376500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:00.376500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:43:00.376500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:00.376500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:43:00.376775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:00.376775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:43:00.376775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:00.376775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:43:00.376775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:43:00.376775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:43:00.376775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:43:00.376775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:43:00.376775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:43:00.376775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:00.376775 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:00.378085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:00.378085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:43:00.378085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:00.378085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:00.378085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:00.378085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:43:00.378085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:00.378085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:43:00.378085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:43:00.378085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:00.378085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:43:00.378085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:00.378085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:43:00.378085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:43:00.378531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:00.378531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:43:00.378531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:00.378531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:43:00.378531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:00.378531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:43:00.378531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:43:00.378531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:43:00.378531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:43:00.378531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:43:00.378531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:00.378531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:43:00.378531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:43:00.378531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:43:00.379311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:43:00.379311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:43:00.379311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:00.379311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:00.379311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:43:00.379311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:43:00.379311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:43:00.379311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:43:00.379311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:43:00.379311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:43:00.379311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:00.379311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:00.379311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:00.379311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:43:00.379311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:00.379311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:00.379311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:00.379311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:43:00.379311 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:00.379927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:43:00.379927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:43:00.379927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:00.379927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:43:00.379927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:00.379927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:43:00.379927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:43:00.379927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:00.379927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:43:00.379927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:00.379927 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:43:00.382463 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:43:00.382768 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:43:00.382878 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:43:00.383023 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:43:00.383229 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:43:00.383484 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:43:00.383619 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:00.384147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:43:00.384147 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:00.384147 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:43:00.384147 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:43:00.384147 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:00.384147 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:43:00.384147 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:00.384147 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:43:00.384955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:00.384955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:43:00.384955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:00.384955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:43:00.384955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:43:00.384955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:43:00.384955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:43:00.384955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:00.384955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:00.384955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:00.384955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:43:00.384955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:00.385225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:00.385225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:00.385225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:43:00.385225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:00.385225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:43:00.385225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:43:00.385225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:00.385225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:43:00.385225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:00.385225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:43:00.385225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:43:00.385225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:00.385225 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:43:00.385496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:00.385496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:43:00.385931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:00.385931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:43:00.385931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:43:00.385931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:43:00.385931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:43:00.385931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:43:00.385931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:00.385931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:43:00.385931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:43:00.385931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:43:00.385931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:43:00.385931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:43:00.385931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:00.385931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:00.385931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:43:00.385931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:43:00.386277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:43:00.386277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:43:00.386277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:00.386277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:00.386277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:00.386277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:43:00.386277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:00.386277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:00.386277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:00.386277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:43:00.386277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:00.386277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:43:00.386277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:43:00.386277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:00.386277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:43:00.386277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:00.386277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:43:00.386277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:43:00.386277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:00.386668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:43:00.386668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:00.386668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:43:05.128982 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:43:05.129787 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:43:05.129787 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:43:05.131062 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:43:05.164636 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:43:05.164788 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:05.165227 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:43:05.165227 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:05.165227 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:43:05.165227 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:43:05.165227 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:05.165227 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:43:05.165227 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:05.165227 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:43:05.165884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:05.165884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:43:05.165884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:05.165884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:43:05.165884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:43:05.165884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:43:05.165884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:43:05.165884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:05.165884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:05.165884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:05.165884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:43:05.165884 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:05.166411 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:05.166411 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:05.166411 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:43:05.166411 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:05.166411 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:43:05.166411 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:43:05.166411 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:05.166411 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:43:05.166411 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:05.166411 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:43:05.166411 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:43:05.166411 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:05.166411 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:43:05.167006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:05.167006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:43:05.167006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:05.167006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:43:05.167006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:43:05.167006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:43:05.167006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:43:05.167006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:43:05.167006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:05.167006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:43:05.167006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:43:05.167006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:43:05.167006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:43:05.167006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:43:05.167006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:05.167774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:05.167774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:43:05.167774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:43:05.167774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:43:05.167774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:43:05.167774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:05.167774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:05.167774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:05.167774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:43:05.167774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:05.167774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:05.167774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:05.167774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:43:05.167774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:05.167774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:43:05.167774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:43:05.167774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:05.167774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:43:05.167774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:05.168502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:43:05.168502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:43:05.168502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:05.168502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:43:05.168502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:05.168502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:43:05.361634 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:43:05.362144 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:43:05.362266 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:43:05.363212 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:43:05.363390 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:43:05.363570 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:05.364079 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:43:05.364079 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:05.364079 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:43:05.364079 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:43:05.364079 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:05.364079 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:43:05.364079 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:05.364079 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:43:05.364957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:05.364957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:43:05.364957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:05.364957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:43:05.364957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:43:05.364957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:43:05.364957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:43:05.364957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:05.364957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:05.364957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:05.364957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:43:05.364957 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:05.365376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:05.365376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:05.365376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:43:05.365376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:05.365376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:43:05.365376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:43:05.365376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:05.365376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:43:05.365376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:05.365376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:43:05.365376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:43:05.365376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:05.365376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:43:05.366060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:05.366060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:43:05.366182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:05.366182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:43:05.366182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:43:05.366182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:43:05.366182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:43:05.366182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:43:05.366182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:05.366182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:43:05.366182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:43:05.366182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:43:05.366182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:43:05.366182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:43:05.366182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:05.366182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:05.366182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:43:05.366182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:43:05.366948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:43:05.366948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:43:05.366948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:05.366948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:05.366948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:05.366948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:43:05.366948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:05.366948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:05.366948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:05.366948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:43:05.366948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:05.366948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:43:05.366948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:43:05.366948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:05.366948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:43:05.366948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:05.366948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:43:05.366948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:43:05.366948 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:05.367880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:43:05.367880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:05.367880 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:43:05.368960 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:43:05.369230 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:43:05.369326 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:43:05.369476 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:43:05.369828 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:43:05.370006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:05.370487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:43:05.370487 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:05.370487 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:43:05.370487 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:43:05.370487 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:05.370487 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:43:05.370487 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:05.370487 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:43:05.371180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:05.371180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:43:05.371180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:43:05.371180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:43:05.371180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:05.371180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:43:05.371180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:05.371180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:43:05.371180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:05.371180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:43:05.371180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:05.371180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:43:05.371180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:05.371180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:43:05.371180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:43:05.371867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:05.371867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:43:05.371867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:05.371867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:43:05.371867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:43:05.371867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:05.371867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:43:05.371867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:05.371867 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:43:05.372284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:05.372284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:43:05.372284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:05.372284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:43:05.372284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:43:05.372284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:43:05.372284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:43:05.372284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:43:05.372284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:43:05.372284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:43:05.372284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:05.372284 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:05.372847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:05.372847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:43:05.372847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:05.372847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:05.372847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:05.372847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:43:05.372847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:05.372847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:43:05.372847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:43:05.372847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:05.372847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:43:05.372847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:05.372847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:43:05.372847 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:43:05.373490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:05.373490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:43:05.373490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:05.373490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:43:05.373490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:05.373490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:43:05.373490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:43:05.373490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:43:05.373490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:43:05.373490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:43:05.373490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:05.373490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:43:05.373490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:43:05.373490 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:43:05.374151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:43:05.374151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:43:05.374151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:05.374151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:05.374151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:43:05.374151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:43:05.374151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:43:05.374151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:43:05.374151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:43:05.374151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:43:05.374151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:43:05.374151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:05.374151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:05.374151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:05.374151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:43:05.374151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:05.374151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:05.374151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:05.374151 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:43:05.374686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:05.374686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:43:05.374686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:43:05.374686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:05.374686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:43:05.374686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:05.374686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:43:05.374686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:43:05.374686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:05.374686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:43:05.374686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:05.374686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:43:05.374686 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:43:05.375018 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:43:05.375018 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:43:05.375083 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:43:05.375313 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:43:05.375550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:05.375952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:43:05.375952 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:05.375952 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:43:05.375952 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:43:05.375952 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:05.375952 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:43:05.375952 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:05.375952 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:43:05.376648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:05.376648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:43:05.376648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:43:05.376648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:43:05.376648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:05.376648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:43:05.376648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:05.376648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:43:05.376648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:05.376648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:43:05.376648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:05.376648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:43:05.376648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:05.376648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:43:05.376648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:43:05.377047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:05.377047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:43:05.377047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:05.377047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:43:05.377047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:43:05.377047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:05.377047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:43:05.377047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:05.377047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:43:05.377466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:05.377466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:43:05.377466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:05.377466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:43:05.377466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:43:05.377466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:43:05.377466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:43:05.377466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:43:05.377466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:43:05.377466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:05.377466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:05.377796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:05.377796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:43:05.377796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:05.377796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:05.377796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:05.377796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:43:05.377796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:05.377796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:43:05.377796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:43:05.377796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:05.377796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:43:05.377796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:05.377796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:43:05.377796 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:43:05.378204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:05.378204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:43:05.378204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:05.378204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:43:05.378554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:05.378554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:43:05.378554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:43:05.378554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:43:05.378554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:43:05.378554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:43:05.378554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:05.378554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:43:05.378554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:43:05.378554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:43:05.378554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:43:05.378554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:43:05.378554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:05.378554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:05.378554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:43:05.378554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:43:05.378554 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:43:05.379068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:43:05.379068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:43:05.379068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:43:05.379068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:05.379068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:05.379068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:05.379068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:43:05.379068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:05.379068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:05.379068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:05.379068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:43:05.379068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:05.379068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:43:05.379068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:43:05.379068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:05.379068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:43:05.379068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:05.379068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:43:05.379068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:43:05.379581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:05.379581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:43:05.379581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:05.379581 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:43:05.382939 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:43:05.383196 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:43:05.383305 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:43:05.383461 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:43:05.383675 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:43:05.383884 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:43:05.384062 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:05.384424 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:43:05.384424 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:05.384424 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:43:05.384424 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:43:05.384424 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:05.384424 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:43:05.384424 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:05.384424 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:43:05.385182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:05.385182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:43:05.385182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:05.385182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:43:05.385182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:43:05.385182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:43:05.385182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:43:05.385182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:05.385182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:05.385182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:05.385182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:43:05.385182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:05.385541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:05.385541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:05.385541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:43:05.385541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:05.385541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:43:05.385541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:43:05.385541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:05.385541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:43:05.385541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:05.385541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:43:05.385541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:43:05.385541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:05.385541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:43:05.385897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:05.385897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:43:05.386131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:05.386131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:43:05.386131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:43:05.386131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:43:05.386131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:43:05.386131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:43:05.386131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:05.386131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:43:05.386131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:43:05.386131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:43:05.386131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:43:05.386131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:43:05.386131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:05.386131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:05.386131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:43:05.386131 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:43:05.386610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:43:05.386610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:43:05.386610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:05.386610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:05.386610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:05.386610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:43:05.386610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:05.386610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:05.386610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:05.386610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:43:05.386610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:05.386610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:43:05.386610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:43:05.386610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:05.386610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:43:05.386610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:05.386610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:43:05.386610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:43:05.386610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:05.387119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:43:05.387119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:05.387119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:43:10.129944 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:43:10.130859 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:43:10.130859 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:43:10.131369 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:43:10.165723 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:43:10.166226 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:10.167393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:43:10.167393 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:10.167393 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:43:10.167393 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:43:10.167393 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:10.167393 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:43:10.167393 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:10.167393 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:43:10.168417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:10.168417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:43:10.168417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:10.168417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:43:10.168417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:43:10.168417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:43:10.168417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:43:10.168417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:10.168417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:10.168417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:10.168417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:43:10.168417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:10.168931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:10.168931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:10.168931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:43:10.168931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:10.168931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:43:10.168931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:43:10.168931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:10.168931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:43:10.168931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:10.168931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:43:10.168931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:43:10.168931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:10.168931 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:43:10.169357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:10.169357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:43:10.169500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:10.169500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:43:10.169500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:43:10.169500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:43:10.169500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:43:10.169500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:43:10.169500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:10.169500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:43:10.169500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:43:10.169500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:43:10.169500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:43:10.169500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:43:10.169500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:10.169500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:10.169500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:43:10.169500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:43:10.170030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:43:10.170030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:43:10.170030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:10.170030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:10.170030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:10.170030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:43:10.170030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:10.170030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:10.170030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:10.170030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:43:10.170030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:10.170030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:43:10.170030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:43:10.170030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:10.170030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:43:10.170030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:10.170030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:43:10.170030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:43:10.170030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:10.170683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:43:10.170683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:10.170683 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:43:10.362278 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:43:10.362910 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:43:10.363103 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:43:10.364338 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:43:10.364749 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:43:10.365003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:10.365924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:43:10.365924 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:10.365924 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:43:10.365924 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:43:10.365924 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:10.365924 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:43:10.365924 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:10.365924 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:43:10.367459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:10.367459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:43:10.367459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:10.367459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:43:10.367459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:43:10.367459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:43:10.367459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:43:10.367459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:10.367459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:10.367459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:10.367459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:43:10.367459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:10.368060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:10.368060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:10.368060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:43:10.368060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:10.368060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:43:10.368060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:43:10.368060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:10.368060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:43:10.368060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:10.368060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:43:10.368060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:43:10.368060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:10.368060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:43:10.368640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:10.368640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:43:10.368981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:10.368981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:43:10.368981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:43:10.368981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:43:10.368981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:43:10.368981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:43:10.368981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:10.368981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:43:10.368981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:43:10.368981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:43:10.368981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:43:10.368981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:43:10.368981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:10.368981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:10.368981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:43:10.368981 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:43:10.369682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:43:10.369682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:43:10.369682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:10.369682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:10.369682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:10.369682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:43:10.369682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:10.369682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:10.369682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:10.369682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:43:10.369682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:10.369682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:43:10.369682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:43:10.369682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:10.369682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:43:10.369682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:10.369682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:43:10.369682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:43:10.369682 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:10.370536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:43:10.370536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:10.370536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:43:10.370536 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:43:10.370536 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:43:10.370536 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:43:10.370536 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:43:10.370536 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:43:10.370990 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:10.371365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:43:10.371365 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:10.371365 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:43:10.371365 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:43:10.371365 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:10.371365 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:43:10.371365 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:10.371365 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:43:10.372141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:10.372141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:43:10.372141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:43:10.372141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:43:10.372141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:10.372141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:43:10.372141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:10.372141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:43:10.372141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:10.372141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:43:10.372141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:10.372141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:43:10.372141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:10.372141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:43:10.372141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:43:10.372774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:10.372774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:43:10.372774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:10.372774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:43:10.372774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:43:10.372774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:10.372774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:43:10.372774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:10.372774 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:43:10.373055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:10.373055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:43:10.373055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:10.373055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:43:10.373055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:43:10.373055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:43:10.373055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:43:10.373055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:43:10.373055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:43:10.373055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:43:10.373055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:10.373055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:10.373422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:10.373422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:43:10.373422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:10.373422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:10.373422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:10.373422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:43:10.373422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:10.373422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:43:10.373422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:43:10.373422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:10.373422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:43:10.373422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:10.373422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:43:10.373422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:43:10.373802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:10.373802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:43:10.373802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:10.373802 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:43:10.374089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:10.374089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:43:10.374089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:43:10.374089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:43:10.374089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:43:10.374089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:43:10.374089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:10.374089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:43:10.374089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:43:10.374089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:43:10.374089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:43:10.374089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:43:10.374089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:10.374089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:10.374089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:43:10.374089 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:43:10.374565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:43:10.374565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:43:10.374565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:43:10.374565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:43:10.374565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:43:10.374565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:10.374565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:10.374565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:10.374565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:43:10.374565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:10.374565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:10.374565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:10.374565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:43:10.374565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:10.374565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:43:10.374565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:43:10.374565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:10.374565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:43:10.375076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:10.375076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:43:10.375076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:43:10.375076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:10.375076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:43:10.375076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:10.375076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:43:10.375076 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:43:10.375324 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:43:10.375436 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:43:10.375600 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:43:10.375931 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:43:10.376145 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:10.376811 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:43:10.376811 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:10.376811 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:43:10.376811 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:43:10.376811 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:10.376811 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:43:10.376811 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:10.376811 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:43:10.377914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:10.377914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:43:10.377914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:43:10.377914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:43:10.377914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:10.377914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:43:10.377914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:10.377914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:43:10.377914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:10.377914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:43:10.377914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:10.377914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:43:10.377914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:10.377914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:43:10.377914 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:43:10.378342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:10.378342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:43:10.378342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:10.378342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:43:10.378342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:43:10.378342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:10.378342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:43:10.378342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:10.378342 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:43:10.378801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:10.378801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:43:10.378801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:10.378801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:43:10.378801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:43:10.378801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:43:10.378801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:43:10.378801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:43:10.378801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:43:10.378801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:10.378801 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:10.379130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:10.379130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:43:10.379130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:10.379130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:10.379130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:10.379130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:43:10.379130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:10.379130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:43:10.379130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:43:10.379130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:10.379130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:43:10.379130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:10.379130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:43:10.379130 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:43:10.379531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:10.379531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:43:10.379531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:10.379531 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:43:10.380011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:10.380011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:43:10.380011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:43:10.380011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:43:10.380011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:43:10.380011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:43:10.380011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:10.380011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:43:10.380011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:43:10.380011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:43:10.380011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:43:10.380011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:43:10.380011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:10.380011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:10.380011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:43:10.380011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:43:10.380011 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:43:10.380577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:43:10.380577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:43:10.380577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:43:10.380577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:10.380577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:10.380577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:10.380577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:43:10.380577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:10.380577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:10.380577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:10.380577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:43:10.380577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:10.380577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:43:10.380577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:43:10.380577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:10.380577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:43:10.380577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:10.380577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:43:10.380577 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:43:10.381105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:10.381105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:43:10.381105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:10.381105 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:43:10.383437 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:43:10.383769 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:43:10.383894 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:43:10.384077 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:43:10.384554 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:43:10.384979 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:43:10.385208 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:10.385928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:43:10.385928 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:10.385928 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:43:10.385928 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:43:10.385928 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:10.385928 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:43:10.385928 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:10.385928 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:43:10.386750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:10.386750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:43:10.386750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:10.386750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:43:10.386750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:43:10.386750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:43:10.386750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:43:10.386750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:10.386750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:10.386750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:10.386750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:43:10.386750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:10.387100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:10.387100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:10.387100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:43:10.387100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:10.387100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:43:10.387100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:43:10.387100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:10.387100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:43:10.387100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:10.387100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:43:10.387100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:43:10.387100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:10.387100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:43:10.387478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:10.387478 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:43:10.387791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:10.387791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:43:10.387791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:43:10.387791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:43:10.387791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:43:10.387791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:43:10.387791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:10.387791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:43:10.387791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:43:10.387791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:43:10.387791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:43:10.387791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:43:10.387791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:10.387791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:10.387791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:43:10.387791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:43:10.388249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:43:10.388249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:43:10.388249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:10.388249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:10.388249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:10.388249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:43:10.388249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:10.388249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:10.388249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:10.388249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:43:10.388249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:10.388249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:43:10.388249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:43:10.388249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:10.388249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:43:10.388249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:10.388249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:43:10.388249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:43:10.388249 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:10.390202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:43:10.390202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:10.390202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:43:15.130564 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:43:15.131210 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:43:15.131210 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:43:15.131819 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:43:15.166978 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:43:15.167257 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:15.169824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:43:15.169824 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:15.169824 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:43:15.169824 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:43:15.169824 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:15.169824 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:43:15.169824 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:15.169824 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:43:15.169824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:15.169824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:43:15.169824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:15.169824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:43:15.169824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:43:15.169824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:43:15.169824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:43:15.170368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:15.170368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:15.170368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:15.170368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:43:15.170368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:15.170368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:15.170368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:15.170368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:43:15.170368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:15.170368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:43:15.170368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:43:15.170368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:15.170368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:43:15.170368 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:15.170965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:43:15.170965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:43:15.170965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:15.170965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:43:15.170965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:15.170965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:43:15.170965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:15.170965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:43:15.170965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:43:15.170965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:43:15.170965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:43:15.170965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:43:15.170965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:15.170965 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:43:15.171434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:43:15.171434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:43:15.171434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:43:15.171434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:43:15.171434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:15.171434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:15.171434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:43:15.171434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:43:15.171434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:43:15.171434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:43:15.171434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:15.171434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:15.171434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:15.171434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:43:15.171434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:15.171434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:15.171434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:15.171434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:43:15.171434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:15.171434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:43:15.171434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:43:15.172092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:15.172092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:43:15.172092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:15.172092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:43:15.172092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:43:15.172092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:15.172092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:43:15.172092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:15.172092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:43:15.363049 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:43:15.364551 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:43:15.364551 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:43:15.365159 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:43:15.365558 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:43:15.365807 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:15.366712 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:43:15.366712 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:15.366712 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:43:15.366712 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:43:15.366712 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:15.366712 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:43:15.366712 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:15.366712 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:43:15.368240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:15.368240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:43:15.368240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:15.368240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:43:15.368240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:43:15.368240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:43:15.368240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:43:15.368240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:15.368240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:15.368240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:15.368240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:43:15.368240 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:15.368518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:15.368518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:15.368518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:43:15.368518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:15.368518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:43:15.368518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:43:15.368518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:15.368518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:43:15.368518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:15.368518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:43:15.368518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:43:15.368518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:15.368518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:43:15.368761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:15.368761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:43:15.369903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:15.369903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:43:15.369903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:43:15.369903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:43:15.369903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:43:15.369903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:43:15.369903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:15.369903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:43:15.369903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:43:15.369903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:43:15.369903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:43:15.369903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:43:15.369903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:15.369903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:15.369903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:43:15.369903 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:43:15.370220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:43:15.370220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:43:15.370220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:15.370220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:15.370220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:15.370220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:43:15.370220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:15.370220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:15.370220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:15.370220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:43:15.370220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:15.370220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:43:15.370220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:43:15.370220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:15.370220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:43:15.370220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:15.370220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:43:15.370220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:43:15.370220 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:15.370582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:43:15.370582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:15.370582 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:43:15.370582 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:43:15.370876 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:43:15.371119 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:43:15.371306 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:43:15.371813 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:43:15.372073 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:15.372756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:43:15.372756 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:15.372756 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:43:15.372756 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:43:15.372756 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:15.372756 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:43:15.372756 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:15.372756 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:43:15.373498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:15.373498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:43:15.373498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:43:15.373498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:43:15.373498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:15.373498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:43:15.373498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:15.373498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:43:15.373498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:15.373498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:43:15.373498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:15.373498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:43:15.373498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:15.373498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:43:15.373498 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:43:15.373825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:15.373825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:43:15.373825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:15.373825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:43:15.373825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:43:15.373825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:15.373825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:43:15.373825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:15.373825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:43:15.374290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:15.374290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:43:15.374290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:15.374290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:43:15.374290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:43:15.374290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:43:15.374290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:43:15.374290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:43:15.374290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:43:15.374290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:43:15.374290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:15.374290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:15.374578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:15.374578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:43:15.374578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:15.374578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:15.374578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:15.374578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:43:15.374578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:15.374578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:43:15.374578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:43:15.374578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:15.374578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:43:15.374578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:15.374578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:43:15.374578 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:43:15.374882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:15.374882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:43:15.374882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:15.374882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:43:15.375312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:15.375312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:43:15.375312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:43:15.375312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:43:15.375312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:43:15.375312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:43:15.375312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:15.375312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:43:15.375312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:43:15.375312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:43:15.375312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:43:15.375312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:43:15.375312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:15.375312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:15.375312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:43:15.375312 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:43:15.375657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:43:15.375657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:43:15.375657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:43:15.375657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:43:15.375657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:43:15.375657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:15.375657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:15.375657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:15.375657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:43:15.375657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:15.375657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:15.375657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:15.375657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:43:15.375657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:15.375657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:43:15.375657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:43:15.375657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:15.375657 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:43:15.375992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:15.375992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:43:15.375992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:43:15.375992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:15.375992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:43:15.375992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:15.375992 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:43:15.375992 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:43:15.375992 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:43:15.375992 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:43:15.376202 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:43:15.376336 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:43:15.376568 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:15.377015 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:43:15.377015 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:15.377015 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:43:15.377015 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:43:15.377015 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:15.377015 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:43:15.377015 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:15.377015 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:43:15.377765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:15.377765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:43:15.377765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:43:15.377765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:43:15.377765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:15.377765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:43:15.377765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:15.377765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:43:15.377765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:15.377765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:43:15.377765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:15.377765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:43:15.377765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:15.377765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:43:15.377765 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:43:15.378040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:15.378040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:43:15.378040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:15.378040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:43:15.378040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:43:15.378040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:15.378040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:43:15.378040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:15.378040 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:43:15.378624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:15.378624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:43:15.378624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:15.378624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:43:15.378624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:43:15.378624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:43:15.378624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:43:15.378624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:43:15.378624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:43:15.378624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:15.378624 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:15.378868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:15.378868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:43:15.378868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:15.378868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:15.378868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:15.378868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:43:15.378868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:15.378868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:43:15.378868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:43:15.378868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:15.378868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:43:15.378868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:15.378868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:43:15.378868 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:43:15.379162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:15.379162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:43:15.379162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:15.379162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:43:15.379708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:15.379708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:43:15.379708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:43:15.379708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:43:15.379708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:43:15.379708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:43:15.379708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:15.379708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:43:15.379708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:43:15.379708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:43:15.379708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:43:15.379708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:43:15.379708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:15.379708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:15.379708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:43:15.379708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:43:15.379708 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:43:15.380053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:43:15.380053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:43:15.380053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:43:15.380053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:15.380053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:15.380053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:15.380053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:43:15.380053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:15.380053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:15.380053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:15.380053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:43:15.380053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:15.380053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:43:15.380053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:43:15.380053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:15.380053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:43:15.380053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:15.380053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:43:15.380053 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:43:15.380436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:15.380436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:43:15.380436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:15.380436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:43:15.383847 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:43:15.384043 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:43:15.384151 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:43:15.384295 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:43:15.384736 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:43:15.385073 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:43:15.385286 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:15.385953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:43:15.385953 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:15.385953 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:43:15.385953 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:43:15.385953 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:15.385953 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:43:15.385953 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:15.385953 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:43:15.386776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:15.386776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:43:15.386776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:15.386776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:43:15.386776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:43:15.386776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:43:15.386776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:43:15.386776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:15.386776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:15.386776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:15.386776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:43:15.386776 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:15.387117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:15.387117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:15.387117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:43:15.387117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:15.387117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:43:15.387117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:43:15.387117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:15.387117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:43:15.387117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:15.387117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:43:15.387117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:43:15.387117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:15.387117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:43:15.387395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:15.387395 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:43:15.387893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:15.387893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:43:15.387893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:43:15.387893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:43:15.387893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:43:15.387893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:43:15.387893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:15.387893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:43:15.387893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:43:15.387893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:43:15.387893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:43:15.387893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:43:15.387893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:15.387893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:15.387893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:43:15.387893 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:43:15.388209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:43:15.388209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:43:15.388209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:15.388209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:15.388209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:15.388209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:43:15.388209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:15.388209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:15.388209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:15.388209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:43:15.388209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:15.388209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:43:15.388209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:43:15.388209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:15.388209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:43:15.388209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:15.388209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:43:15.388209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:43:15.388209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:15.388587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:43:15.388587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:15.388587 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:43:20.131682 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:43:20.132515 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:43:20.132515 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:43:20.133606 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:43:20.170955 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:43:20.171445 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:20.172679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:43:20.172679 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:20.172679 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:43:20.172679 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:43:20.172679 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:20.172679 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:43:20.172679 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:20.172679 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:43:20.175615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:20.175615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:43:20.175615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:20.175615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:43:20.175615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:43:20.175615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:43:20.175615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:43:20.175615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:20.175615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:20.175615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:20.175615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:43:20.175615 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:20.176397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:20.176397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:20.176397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:43:20.176397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:20.176397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:43:20.176397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:43:20.176397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:20.176397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:43:20.176397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:20.176397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:43:20.176397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:43:20.176397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:20.176397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:43:20.177119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:20.177119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:43:20.177119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:20.177119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:43:20.177119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:43:20.177119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:43:20.177119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:43:20.177119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:43:20.177119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:20.177119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:43:20.177119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:43:20.177119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:43:20.177119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:43:20.177119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:43:20.177119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:20.178148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:20.178148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:43:20.178148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:43:20.178148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:43:20.178148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:43:20.178148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:20.178148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:20.178148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:20.178148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:43:20.178148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:20.178148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:20.178148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:20.178148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:43:20.178148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:20.178148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:43:20.178148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:43:20.178148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:20.178148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:43:20.178148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:20.179100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:43:20.179100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:43:20.179100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:20.179100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:43:20.179100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:20.179100 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:43:20.363930 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:43:20.364603 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:43:20.364807 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:43:20.365907 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:43:20.366149 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:43:20.366314 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:20.367030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:43:20.367030 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:20.367030 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:43:20.367030 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:43:20.367030 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:20.367030 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:43:20.367030 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:20.367030 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:43:20.368645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:20.368645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:43:20.368645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:20.368645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:43:20.368645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:43:20.368645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:43:20.368645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:43:20.368645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:20.368645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:20.368645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:20.368645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:43:20.368645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:20.369021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:20.369021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:20.369021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:43:20.369021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:20.369021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:43:20.369021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:43:20.369021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:20.369021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:43:20.369021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:20.369021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:43:20.369021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:43:20.369021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:20.369021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:43:20.369402 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:20.369402 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:43:20.370376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:20.370376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:43:20.370376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:43:20.370376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:43:20.370376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:43:20.370376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:43:20.370376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:20.370376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:43:20.370376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:43:20.370376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:43:20.370376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:43:20.370376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:43:20.370376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:20.370376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:20.370376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:43:20.370376 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:43:20.370874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:43:20.370874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:43:20.370874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:20.370874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:20.370874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:20.370874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:43:20.370874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:20.370874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:20.370874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:20.370874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:43:20.370874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:20.370874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:43:20.370874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:43:20.370874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:20.370874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:43:20.370874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:20.370874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:43:20.370874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:43:20.370874 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:20.371432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:43:20.371432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:20.371432 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:43:20.371432 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:43:20.371432 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:43:20.371630 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:43:20.371729 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:43:20.372117 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:43:20.372313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:20.372909 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:43:20.372909 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:20.372909 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:43:20.372909 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:43:20.372909 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:20.372909 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:43:20.372909 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:20.372909 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:43:20.374182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:20.374182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:43:20.374182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:43:20.374182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:43:20.374182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:20.374182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:43:20.374182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:20.374182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:43:20.374182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:20.374182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:43:20.374182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:20.374182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:43:20.374182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:20.374182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:43:20.374182 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:43:20.374637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:20.374637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:43:20.374637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:20.374637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:43:20.374637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:43:20.374637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:20.374637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:43:20.374637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:20.374637 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:43:20.375595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:20.375595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:43:20.375595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:20.375595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:43:20.375595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:43:20.375595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:43:20.375595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:43:20.375595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:43:20.375595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:43:20.375595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:43:20.375595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:20.375595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:20.375975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:20.375975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:43:20.375975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:20.375975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:20.375975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:20.375975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:43:20.375975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:20.375975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:43:20.375975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:43:20.375975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:20.375975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:43:20.375975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:20.375975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:43:20.375975 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:43:20.376372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:20.376372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:43:20.376372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:20.376372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:43:20.377051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:20.377051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:43:20.377051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:43:20.377051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:43:20.377051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:43:20.377051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:43:20.377051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:20.377051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:43:20.377051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:43:20.377051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:43:20.377051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:43:20.377051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:43:20.377051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:20.377051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:20.377051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:43:20.377051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:43:20.377532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:43:20.377532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:43:20.377532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:43:20.377532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:43:20.377532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:43:20.377532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:20.377532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:20.377532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:20.377532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:43:20.377532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:20.377532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:20.377532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:20.377532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:43:20.377532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:20.377532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:43:20.377532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:43:20.377532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:20.377532 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:43:20.378066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:20.378066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:43:20.378066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:43:20.378066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:20.378066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:43:20.378066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:20.378066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:43:20.378066 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:43:20.378066 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:43:20.378066 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:43:20.378371 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:43:20.378371 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:43:20.378459 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:20.379006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:43:20.379006 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:20.379006 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:43:20.379006 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:43:20.379006 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:20.379006 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:43:20.379006 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:20.379006 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:43:20.379545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:20.379545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:43:20.379545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:43:20.379545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:43:20.379545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:20.379545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:43:20.379545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:20.379545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:43:20.379545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:20.379545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:43:20.379545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:20.379545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:43:20.379545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:20.379545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:43:20.379545 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:43:20.379971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:20.379971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:43:20.379971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:20.379971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:43:20.379971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:43:20.379971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:20.379971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:43:20.379971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:20.379971 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:43:20.380223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:20.380223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:43:20.380223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:20.380223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:43:20.380223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:43:20.380223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:43:20.380223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:43:20.380223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:43:20.380223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:43:20.380223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:20.380223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:20.380560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:20.380560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:43:20.380560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:20.380560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:20.380560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:20.380560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:43:20.380560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:20.380560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:43:20.380560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:43:20.380560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:20.380560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:43:20.380560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:20.380560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:43:20.380560 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:43:20.380976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:20.380976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:43:20.380976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:20.380976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:43:20.380976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:20.380976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:43:20.380976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:43:20.380976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:43:20.380976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:43:20.380976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:43:20.380976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:20.380976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:43:20.380976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:43:20.380976 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:43:20.381361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:43:20.381361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:43:20.381361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:20.381361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:20.381361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:43:20.381361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:43:20.381361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:43:20.381361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:43:20.381361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:43:20.381361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:43:20.381361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:20.381361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:20.381361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:20.381361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:43:20.381361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:20.381361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:20.381361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:20.381361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:43:20.381361 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:20.381922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:43:20.381922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:43:20.381922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:20.381922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:43:20.381922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:20.381922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:43:20.381922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:43:20.381922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:20.381922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:43:20.381922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:20.381922 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:43:20.384150 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:43:20.384346 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:43:20.384420 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:43:20.384561 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:43:20.384819 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:43:20.385014 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:43:20.385092 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:20.385473 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:43:20.385473 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:20.385473 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:43:20.385473 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:43:20.385473 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:20.385473 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:43:20.385473 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:20.385473 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:43:20.386188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:20.386188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:43:20.386188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:20.386188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:43:20.386188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:43:20.386188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:43:20.386188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:43:20.386188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:20.386188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:20.386188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:20.386188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:43:20.386188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:20.386580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:20.386580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:20.386580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:43:20.386580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:20.386580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:43:20.386580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:43:20.386580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:20.386580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:43:20.386580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:20.386580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:43:20.386580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:43:20.386580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:20.386580 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:43:20.387076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:20.387076 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:43:20.387143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:20.387143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:43:20.387143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:43:20.387143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:43:20.387143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:43:20.387143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:43:20.387143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:20.387143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:43:20.387143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:43:20.387143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:43:20.387143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:43:20.387143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:43:20.387143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:20.387143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:20.387143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:43:20.387143 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:43:20.387626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:43:20.387626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:43:20.387626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:20.387626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:20.387626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:20.387626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:43:20.387626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:20.387626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:20.387626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:20.387626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:43:20.387626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:20.387626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:43:20.387626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:43:20.387626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:20.387626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:43:20.387626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:20.387626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:43:20.387626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:43:20.387626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:20.388166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:43:20.388166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:20.388166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:43:25.132195 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:43:25.132849 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:43:25.133054 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:43:25.134601 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:43:25.141562 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG dbcounter [-] [105581] Writing DB stats neutron:SELECT=240 {{(pid=105581) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 14:43:25.172161 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:43:25.172564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:25.173483 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:43:25.173483 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:25.173483 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:43:25.173483 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:43:25.173483 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:25.173483 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:43:25.173483 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:25.173483 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:43:25.174883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:25.174883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:43:25.174883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:25.174883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:43:25.174883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:43:25.174883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:43:25.174883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:43:25.174883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:25.174883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:25.174883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:25.174883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:43:25.174883 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:25.175198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:25.175198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:25.175198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:43:25.175198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:25.175198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:43:25.175198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:43:25.175198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:25.175198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:43:25.175198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:25.175198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:43:25.175198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:43:25.175198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:25.175198 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:43:25.175496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:25.175496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:43:25.176365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:25.176365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:43:25.176365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:43:25.176365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:43:25.176365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:43:25.176365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:43:25.176365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:25.176365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:43:25.176365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:43:25.176365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:43:25.176365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:43:25.176365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:43:25.176365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:25.176365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:25.176365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:43:25.176365 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:43:25.176761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:43:25.176761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:43:25.176761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:25.176761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:25.176761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:25.176761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:43:25.176761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:25.176761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:25.176761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:25.176761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:43:25.176761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:25.176761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:43:25.176761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:43:25.176761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:25.176761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:43:25.176761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:25.176761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:43:25.176761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:43:25.176761 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:25.177200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:43:25.177200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:25.177200 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:43:25.364562 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:43:25.365104 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:43:25.365359 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:43:25.366727 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:43:25.367127 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:43:25.367437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:25.368450 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:43:25.368450 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:25.368450 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:43:25.368450 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:43:25.368450 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:25.368450 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:43:25.368450 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:25.368450 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:43:25.370446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:25.370446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:43:25.370446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:25.370446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:43:25.370446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:43:25.370446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:43:25.370446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:43:25.370446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:25.370446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:25.370446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:25.370446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:43:25.370446 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:25.372106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:25.372106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:25.372106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:43:25.372106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:25.372106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:43:25.372106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:43:25.372106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:25.372106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:43:25.372106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:25.372106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:43:25.372106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:43:25.372106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:25.372106 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:43:25.373055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:25.373055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:43:25.373055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:25.373055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:43:25.373055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:43:25.373055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:43:25.373055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:43:25.373055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:43:25.373055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:25.373055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:43:25.373055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:43:25.373055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:43:25.373055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:43:25.373055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:43:25.373055 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:25.373644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:25.373644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:43:25.373644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:43:25.373644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:43:25.373644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:43:25.373644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:25.373644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:25.373644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:25.373644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:43:25.373644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:25.373644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:25.373644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:25.373644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:43:25.373644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:25.373644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:43:25.373644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:43:25.373644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:25.373644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:43:25.373644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:25.374244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:43:25.374244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:43:25.374244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:25.374244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:43:25.374244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:25.374244 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:43:25.374244 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:43:25.374244 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:43:25.374244 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:43:25.374244 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:43:25.375209 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:43:25.375209 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:25.375730 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:43:25.375730 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:25.375730 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:43:25.375730 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:43:25.375730 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:25.375730 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:43:25.375730 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:25.375730 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:43:25.376736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:25.376736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:43:25.376736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:43:25.376736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:43:25.376736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:25.376736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:43:25.376736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:25.376736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:43:25.376736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:25.376736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:43:25.376736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:25.376736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:43:25.376736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:25.376736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:43:25.376736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:43:25.377357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:25.377357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:43:25.377357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:25.377357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:43:25.377357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:43:25.377357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:25.377357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:43:25.377357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:25.377357 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:43:25.378147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:25.378147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:43:25.378147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:25.378147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:43:25.378147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:43:25.378147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:43:25.378147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:43:25.378147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:43:25.378147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:43:25.378147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:43:25.378147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:25.378147 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:25.378690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:25.378690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:43:25.378690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:25.378690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:25.378690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:25.378690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:43:25.378690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:25.378690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:43:25.378690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:43:25.378690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:25.378690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:43:25.378690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:25.378690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:43:25.378690 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:43:25.379173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:25.379173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:43:25.379173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:25.379173 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:43:25.379923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:25.379923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:43:25.379923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:43:25.379923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:43:25.379923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:43:25.379923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:43:25.379923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:25.379923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:43:25.379923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:43:25.379923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:43:25.379923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:43:25.379923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:43:25.379923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:25.379923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:25.379923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:43:25.379923 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:43:25.380337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:43:25.380337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:43:25.380337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:43:25.380337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:43:25.380337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:43:25.380337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:25.380337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:25.380337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:25.380337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:43:25.380337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:25.380337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:25.380337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:25.380337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:43:25.380337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:25.380337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:43:25.380337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:43:25.380337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:25.380337 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:43:25.380828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:25.380828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:43:25.380828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:43:25.380828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:25.380828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:43:25.380828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:25.380828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:43:25.380828 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:43:25.380828 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:43:25.380828 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:43:25.381027 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:43:25.381330 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:43:25.381475 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:25.382167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:43:25.382167 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:25.382167 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:43:25.382167 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:43:25.382167 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:25.382167 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:43:25.382167 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:25.382167 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:43:25.382950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:25.382950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:43:25.382950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:43:25.382950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:43:25.382950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:25.382950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:43:25.382950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:25.382950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:43:25.382950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:25.382950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:43:25.382950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:25.382950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:43:25.382950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:25.382950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:43:25.382950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:43:25.384035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:25.384035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:43:25.384035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:25.384035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:43:25.384035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:43:25.384035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:25.384035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:43:25.384035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:25.384035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:43:25.384035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:25.384035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:43:25.384035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:25.384035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:43:25.384035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:43:25.384469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:43:25.384469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:43:25.384469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:43:25.384469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:43:25.384469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:25.384469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:25.384469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:25.384469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:43:25.384469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:25.384469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:25.384469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:25.384469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:43:25.384469 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:25.384808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:43:25.384808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:43:25.384808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:25.384808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:43:25.384808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:25.384808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:43:25.384808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:43:25.384808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:25.384808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:43:25.384808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:25.384808 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:43:25.385394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:25.385394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:43:25.385394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:43:25.385394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:43:25.385394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:43:25.385394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:43:25.385394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:25.385394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:43:25.385394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:43:25.385394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:43:25.385394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:43:25.385394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:43:25.385394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:25.385394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:25.385394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:43:25.385394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:43:25.385394 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:43:25.386616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:43:25.386616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:43:25.386616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:43:25.386616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:25.386616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:25.386616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:25.386616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:43:25.386616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:25.386616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:25.386616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:25.386616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:43:25.386616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:25.386616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:43:25.386616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:43:25.386616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:25.386616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:43:25.386616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:25.386616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:43:25.386616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:43:25.387379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:25.387379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:43:25.387379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:25.387379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:43:25.387379 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:43:25.387379 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:43:25.387379 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:43:25.387379 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:43:25.387628 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:43:25.387628 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:43:25.387628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:25.387950 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:43:25.387950 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:25.387950 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:43:25.387950 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:43:25.387950 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:25.387950 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:43:25.387950 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:25.387950 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:43:25.388737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:25.388737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:43:25.388737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:25.388737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:43:25.388737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:43:25.388737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:43:25.388737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:43:25.388737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:25.388737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:25.388737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:25.388737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:43:25.388737 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:25.389575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:25.389575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:25.389575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:43:25.389575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:25.389575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:43:25.389575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:43:25.389575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:25.389575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:43:25.389575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:25.389575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:43:25.389575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:43:25.389575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:25.389575 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:43:25.389871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:25.389871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:43:25.389871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:25.389871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:43:25.389871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:43:25.389871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:43:25.389871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:43:25.389871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:43:25.389871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:25.389871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:43:25.389871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:43:25.389871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:43:25.389871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:43:25.389871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:43:25.389871 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:25.390148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:25.390148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:43:25.390148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:43:25.390148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:43:25.390148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:43:25.390148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:25.390148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:25.390148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:25.390148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:43:25.390148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:25.390148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:25.390148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:25.390148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:43:25.390148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:25.390148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:43:25.390148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:43:25.390148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:25.390148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:43:25.390148 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:25.390504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:43:25.390504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:43:25.390504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:25.390504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:43:25.390504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:25.390504 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:43:30.132946 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:43:30.133836 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:43:30.133993 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:43:30.135335 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:43:30.183395 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:43:30.183726 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:30.185890 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:43:30.185890 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:30.185890 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:43:30.185890 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:43:30.185890 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:30.185890 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:43:30.185890 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:30.185890 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:43:30.186738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.052 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:30.186738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:43:30.186738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:30.186738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:43:30.186738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:43:30.186738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:43:30.186738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:43:30.186738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:30.186738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:30.186738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:30.186738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:43:30.186738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:30.187294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:30.187294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:30.187294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:43:30.187294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:30.187294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:43:30.187294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:43:30.187294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:30.187294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:43:30.187294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:30.187294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:43:30.187294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:43:30.187294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:30.187294 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:43:30.188021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:30.188021 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:43:30.188755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:30.188755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:43:30.188755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:43:30.188755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:43:30.188755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:43:30.188755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:43:30.188755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:30.188755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:43:30.188755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:43:30.188755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:43:30.188755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:43:30.188755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:43:30.188755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:30.188755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:30.188755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:43:30.188755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:43:30.189081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:43:30.189081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:43:30.189081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:30.189081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:30.189081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:30.189081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:43:30.189081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:30.189081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:30.189081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:30.189081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:43:30.189081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:30.189081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:43:30.189081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:43:30.189081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:30.189081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:43:30.189081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:30.189081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:43:30.189081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:43:30.189081 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:30.189471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:43:30.189471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:30.189471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:43:30.365131 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:43:30.365487 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:43:30.365667 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:43:30.366454 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:43:30.366708 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:43:30.366897 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:30.367667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:43:30.367667 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:30.367667 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:43:30.367667 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:43:30.367667 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:30.367667 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:43:30.367667 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:30.367667 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:43:30.368592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:30.368592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:43:30.368592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:30.368592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:43:30.368592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:43:30.368592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:43:30.368592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:43:30.368592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:30.368592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:30.368592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:30.368592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:43:30.368592 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:30.369216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:30.369216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:30.369216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:43:30.369216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:30.369216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:43:30.369216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:43:30.369216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:30.369216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:43:30.369216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:30.369216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:43:30.369216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:43:30.369216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:30.369216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:43:30.369651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:30.369651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:43:30.369651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:30.369651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:43:30.369651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:43:30.369651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:43:30.369651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:43:30.369651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:43:30.369651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:30.369651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:43:30.369651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:43:30.369651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:43:30.369651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:43:30.369651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:43:30.369651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:30.370077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:30.370077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:43:30.370077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:43:30.370077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:43:30.370077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:43:30.370077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:30.370077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:30.370077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:30.370077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:43:30.370077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:30.370077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:30.370077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:30.370077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:43:30.370077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:30.370077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:43:30.370077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:43:30.370077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:30.370077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:43:30.370077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:30.370599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:43:30.370599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:43:30.370599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:30.370599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:43:30.370599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:30.370599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:43:30.373529 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:43:30.373746 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:43:30.373884 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:43:30.373953 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:43:30.374312 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:43:30.374563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:30.375524 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:43:30.375524 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:30.375524 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:43:30.375524 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:43:30.375524 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:30.375524 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:43:30.375524 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:30.375524 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:43:30.376250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:30.376250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:43:30.376250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:43:30.376250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:43:30.376250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:30.376250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:43:30.376250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:30.376250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:43:30.376250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:30.376250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:43:30.376250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:30.376250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:43:30.376250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:30.376250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:43:30.376250 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:43:30.376825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:30.376825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:43:30.376825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:30.376825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:43:30.376825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:43:30.376825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:30.376825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:43:30.376825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:30.376825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:43:30.376825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:30.376825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:43:30.376825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:30.376825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:43:30.376825 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:43:30.377308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:43:30.377308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:43:30.377308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:43:30.377308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:43:30.377308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:43:30.377308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:30.377308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:30.377308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:30.377308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:43:30.377308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:30.377308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:30.377308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:30.377308 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:43:30.377848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:30.377848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:43:30.377848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:43:30.377848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:30.377848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:43:30.377848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:30.377848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:43:30.377848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:43:30.377848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:30.377848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:43:30.377848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:30.377848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:43:30.378362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:30.378362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:43:30.378362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:43:30.378362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:43:30.378362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:43:30.378362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:43:30.378362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:30.378362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:43:30.378362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:43:30.378362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:43:30.378362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:43:30.378362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:43:30.378362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:30.378362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:30.378362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:43:30.378362 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:43:30.378932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:43:30.378932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:43:30.378932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:43:30.378932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:43:30.378932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:43:30.378932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:30.378932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:30.378932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:30.378932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:43:30.378932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:30.378932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:30.378932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:30.378932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:43:30.378932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:30.378932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:43:30.378932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:43:30.378932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:30.378932 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:43:30.379729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:30.379729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:43:30.379729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:43:30.379729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:30.379729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:43:30.379729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:30.379729 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:43:30.380303 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:43:30.380459 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:43:30.380546 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:43:30.380619 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:43:30.380998 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:43:30.381272 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:30.381859 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:43:30.381859 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:30.381859 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:43:30.381859 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:43:30.381859 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:30.381859 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:43:30.381859 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:30.381859 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:43:30.382422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:30.382422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:43:30.382422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:43:30.382422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:43:30.382422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:30.382422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:43:30.382422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:30.382422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:43:30.382422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:30.382422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:43:30.382422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:30.382422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:43:30.382422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:30.382422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:43:30.382422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:43:30.382953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:30.382953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:43:30.382953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:30.382953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:43:30.382953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:43:30.382953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:30.382953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:43:30.382953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:30.382953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:43:30.383202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:30.383202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:43:30.383202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:30.383202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:43:30.383202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:43:30.383202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:43:30.383202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:43:30.383202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:43:30.383202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:43:30.383202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:30.383202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:30.383583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:30.383583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:43:30.383583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:30.383583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:30.383583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:30.383583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:43:30.383583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:30.383583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:43:30.383583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:43:30.383583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:30.383583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:43:30.383583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:30.383583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:43:30.383583 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:43:30.384069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:30.384069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:43:30.384069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:30.384069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:43:30.384069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:30.384069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:43:30.384069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:43:30.384069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:43:30.384069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:43:30.384069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:43:30.384069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:30.384069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:43:30.384069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:43:30.384069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:43:30.384653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:43:30.384653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:43:30.384653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:30.384653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:30.384653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:43:30.384653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:43:30.384653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:43:30.384653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:43:30.384653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:43:30.384653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:43:30.384653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:30.384653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:30.384653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:30.384653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:43:30.384653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:30.384653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:30.384653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:30.384653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:43:30.384653 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:30.385383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:43:30.385383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:43:30.385383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:30.385383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:43:30.385383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:30.385383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:43:30.385383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:43:30.385383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:30.385383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:43:30.385383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:30.385383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:43:30.386030 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:43:30.386030 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:43:30.386030 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:43:30.386166 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:43:30.386514 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:43:30.386732 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:43:30.386881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:30.387438 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:43:30.387438 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:30.387438 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:43:30.387438 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:43:30.387438 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:30.387438 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:43:30.387438 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:30.387438 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:43:30.388188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:30.388188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:43:30.388188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:30.388188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:43:30.388188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:43:30.388188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:43:30.388188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:43:30.388188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:30.388188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:30.388188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:30.388188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:43:30.388188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:30.388830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:30.388830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:30.388830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:43:30.388830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:30.388830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:43:30.388830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:43:30.388830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:30.388830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:43:30.388830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:30.388830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:43:30.388830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:43:30.388830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:30.388830 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:43:30.389355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:30.389355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:43:30.389355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:30.389355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:43:30.389355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:43:30.389355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:43:30.389355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:43:30.389355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:43:30.389355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:30.389355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:43:30.389355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:43:30.389355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:43:30.389355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:43:30.389355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:43:30.389355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:30.389970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:30.389970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:43:30.389970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:43:30.389970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:43:30.389970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:43:30.389970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:30.389970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:30.389970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:30.389970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:43:30.389970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:30.389970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:30.389970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:30.389970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:43:30.389970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:30.389970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:43:30.389970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:43:30.389970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:30.389970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:43:30.389970 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:30.390732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:43:30.390732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:43:30.390732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:30.390732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:43:30.390732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:30.390732 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:43:35.133997 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:43:35.134925 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:43:35.135065 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:43:35.136229 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:43:35.173871 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:43:35.174277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:35.175162 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:43:35.175162 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:35.175162 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:43:35.175162 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:43:35.175162 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:35.175162 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:43:35.175162 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:35.175162 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:43:35.176228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:35.176228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:43:35.176228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:35.176228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:43:35.176228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:43:35.176228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:43:35.176228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:43:35.176228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:35.176228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:35.176228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:35.176228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:43:35.176228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:35.177030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:35.177030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:35.177030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:43:35.177030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:35.177030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:43:35.177030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:43:35.177030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:35.177030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:43:35.177030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:35.177030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:43:35.177030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:43:35.177030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:35.177030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:43:35.177744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:35.177744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:43:35.177744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:35.177744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:43:35.177744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:43:35.177744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:43:35.177744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:43:35.177744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:43:35.177744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:35.177744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:43:35.177744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:43:35.177744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:43:35.177744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:43:35.177744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:43:35.177744 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:35.178566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:35.178566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:43:35.178566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:43:35.178566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:43:35.178566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:43:35.178566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:35.178566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:35.178566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:35.178566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:43:35.178566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:35.178566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:35.178566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:35.178566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:43:35.178566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:35.178566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:43:35.178566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:43:35.178566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:35.178566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:43:35.178566 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:35.179544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:43:35.179544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:43:35.179544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:35.179544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:43:35.179544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:35.179544 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:43:35.365860 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:43:35.367592 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:43:35.367826 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:43:35.369042 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:43:35.369385 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:43:35.369655 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:35.371747 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:43:35.371747 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:35.371747 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:43:35.371747 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:43:35.371747 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:35.371747 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:43:35.371747 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:35.371747 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:43:35.373295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:35.373295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:43:35.373295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:35.373295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:43:35.373295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:43:35.373295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:43:35.373295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:43:35.373295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:35.373295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:35.373295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:35.373295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:43:35.373295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:35.373911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:35.373911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:35.373911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:43:35.373911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:35.373911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:43:35.373911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:43:35.373911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:35.373911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:43:35.373911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:35.373911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:43:35.373911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:43:35.373911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:35.373911 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:43:35.375114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:35.375114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:43:35.375114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:35.375114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:43:35.375114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:43:35.375114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:43:35.375114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:43:35.375114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:43:35.375114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:35.375114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:43:35.375114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:43:35.375114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:43:35.375114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:43:35.375114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:43:35.375114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:35.376085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:35.376085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:43:35.376085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:43:35.376085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:43:35.376085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:43:35.376085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:35.376085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:35.376085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:35.376085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:43:35.376085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:35.376085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:35.376085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:35.376085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:43:35.376085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:35.376085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:43:35.376085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:43:35.376085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:35.376085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:43:35.376085 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:35.376929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:43:35.376929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:43:35.376929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:35.376929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:43:35.376929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:35.376929 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:43:35.376929 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:43:35.376929 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:43:35.376929 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:43:35.376929 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:43:35.378436 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:43:35.378436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:35.378436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:43:35.378436 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:35.378436 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:43:35.378436 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:43:35.378436 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:35.378436 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:43:35.378436 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:35.378436 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:43:35.378436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:35.378436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:43:35.378436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:43:35.378436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:43:35.380111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:35.380111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:43:35.380111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:35.380111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:43:35.380111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:35.380111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:43:35.380111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:35.380111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:43:35.380111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:35.380111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:43:35.380111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:43:35.380111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:35.380111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:43:35.380111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:35.380111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:43:35.380111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:43:35.380111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:35.380111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:43:35.382235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:35.382235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:43:35.382235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:35.382235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:43:35.382235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:35.382235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:43:35.382235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:43:35.382235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:43:35.382235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:43:35.382235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:43:35.382235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:43:35.382235 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:43:35.382755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:35.382755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:35.382755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:35.382755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:43:35.382755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:35.382755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:35.382755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:35.382755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:43:35.382755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:35.382755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:43:35.382755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:43:35.382755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:35.382755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:43:35.382755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:35.383304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:43:35.383304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:43:35.383304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:35.383304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:43:35.383304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:35.383304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:43:35.383304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:35.383304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:43:35.383304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:43:35.383304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:43:35.383304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:43:35.383304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:43:35.383304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:35.383304 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:43:35.383638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:43:35.383638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:43:35.383638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:43:35.383638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:43:35.383638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:35.383638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:35.383638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:43:35.383638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:43:35.383638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:43:35.383638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:43:35.383638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:43:35.383638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:43:35.383638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:43:35.383638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:35.383638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:35.383638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:35.383638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:43:35.383638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:35.383638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:35.383991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:35.383991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:43:35.383991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:35.383991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:43:35.383991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:43:35.383991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:35.383991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:43:35.383991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:35.383991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:43:35.383991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:43:35.383991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:35.383991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:43:35.383991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:35.383991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:43:35.383991 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:43:35.384370 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:43:35.384370 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:43:35.384370 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:43:35.384370 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:43:35.384370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:35.384370 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:43:35.384370 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:35.384370 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:43:35.384370 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:43:35.384649 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:35.384649 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:43:35.384649 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:35.384649 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:43:35.384649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:35.384649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:43:35.384649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:43:35.384649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:43:35.384649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:35.384649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:43:35.384649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:35.384649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:43:35.384649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:35.384649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:43:35.384649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:35.384649 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:43:35.385003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:35.385003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:43:35.385003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:43:35.385003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:35.385003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:43:35.385003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:35.385003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:43:35.385003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:43:35.385003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:35.385003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:43:35.385003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:35.385003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:43:35.385003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:35.385003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:43:35.385287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:35.385287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:43:35.385287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:43:35.385287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:43:35.385287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:43:35.385287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:43:35.385287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:43:35.385287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:35.385287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:35.385287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:35.385287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:43:35.385287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:35.385287 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:35.385667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:35.385667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:43:35.385667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:35.385667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:43:35.385667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:43:35.385667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:35.385667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:43:35.385667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:35.385667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:43:35.385667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:43:35.385667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:35.385667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:43:35.385667 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:35.385887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:43:35.385887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:35.385887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:43:35.385887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:43:35.385887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:43:35.385887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:43:35.385887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:43:35.385887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:35.385887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:43:35.385887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:43:35.385887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:43:35.385887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:43:35.385887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:43:35.385887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:35.385887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:35.385887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:43:35.385887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:43:35.386187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:43:35.386187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:43:35.386187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:43:35.386187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:43:35.386187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:35.386187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:35.386187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:35.386187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:43:35.386187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:35.386187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:35.386187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:35.386187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:43:35.386187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:35.386187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:43:35.386187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:43:35.386187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:35.386187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:43:35.386187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:35.386187 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:43:35.386530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:43:35.386530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:35.386530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:43:35.386530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:35.386530 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:43:35.386530 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:43:35.386530 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:43:35.386530 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:43:35.386702 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:43:35.386886 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:43:35.387088 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:43:35.387222 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:35.387686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:43:35.387686 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:35.387686 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:43:35.387686 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:43:35.387686 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:35.387686 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:43:35.387686 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:35.387686 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:43:35.388412 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:35.388412 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:43:35.388412 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:35.388412 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:43:35.388412 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:43:35.388412 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:43:35.388412 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:43:35.388412 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:35.388412 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:35.388412 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:35.388412 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:43:35.388412 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:35.388935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:35.388935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:35.388935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:43:35.388935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:35.388935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:43:35.388935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:43:35.388935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:35.388935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:43:35.388935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:35.388935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:43:35.388935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:43:35.388935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:35.388935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:43:35.389559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:35.389559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:43:35.389559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:35.389559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:43:35.389559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:43:35.389559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:43:35.389559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:43:35.389559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:43:35.389559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:35.389559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:43:35.389559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:43:35.389559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:43:35.389559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:43:35.389559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:43:35.389559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:35.390234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:35.390234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:43:35.390234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:43:35.390234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:43:35.390234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:43:35.390234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:35.390234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:35.390234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:35.390234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:43:35.390234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:35.390234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:35.390234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:35.390234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:43:35.390234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:35.390234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:43:35.390234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:43:35.390234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:35.390234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:43:35.390234 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:35.391029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:43:35.391029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:43:35.391029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:35.391029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:43:35.391029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:35.391029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:43:40.134795 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:43:40.136450 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:43:40.136450 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:43:40.139725 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:43:40.176029 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:43:40.176216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:40.176782 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:43:40.176782 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:40.176782 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:43:40.176782 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:43:40.176782 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:40.176782 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:43:40.176782 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:40.176782 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:43:40.177687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:40.177687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:43:40.177687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:40.177687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:43:40.177687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:43:40.177687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:43:40.177687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:43:40.177687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:40.177687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:40.177687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:40.177687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:43:40.177687 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:40.178334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:40.178334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:40.178334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:43:40.178334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:40.178334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:43:40.178334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:43:40.178334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:40.178334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:43:40.178334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:40.178334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:43:40.178334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:43:40.178334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:40.178334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:43:40.179042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:40.179042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:43:40.179042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:40.179042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:43:40.179042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:43:40.179042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:43:40.179042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:43:40.179042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:43:40.179042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:40.179042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:43:40.179042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:43:40.179042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:43:40.179042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:43:40.179042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:43:40.179042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:40.180028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:40.180028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:43:40.180028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:43:40.180028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:43:40.180028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:43:40.180028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:40.180028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:40.180028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:40.180028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:43:40.180028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:40.180028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:40.180028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:40.180028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:43:40.180028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:40.180028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:43:40.180028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:43:40.180028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:40.180028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:43:40.180028 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:40.181066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:43:40.181066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:43:40.181066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:40.181066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:43:40.181066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:40.181066 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:43:40.367805 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:43:40.368424 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:43:40.368673 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:43:40.369720 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:43:40.370012 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:43:40.370285 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:40.371149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:43:40.371149 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:40.371149 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:43:40.371149 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:43:40.371149 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:40.371149 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:43:40.371149 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:40.371149 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:43:40.372613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:40.372613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:43:40.372613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:40.372613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:43:40.372613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:43:40.372613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:43:40.372613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:43:40.372613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:40.372613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:40.372613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:40.372613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:43:40.372613 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:40.373262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:40.373262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:40.373262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:43:40.373262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:40.373262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:43:40.373262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:43:40.373262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:40.373262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:43:40.373262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:40.373262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:43:40.373262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:43:40.373262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:40.373262 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:43:40.373820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:40.373820 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:43:40.374019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:40.374019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:43:40.374019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:43:40.374019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:43:40.374019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:43:40.374019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:43:40.374019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:40.374019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:43:40.374019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:43:40.374019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:43:40.374019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:43:40.374019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:43:40.374019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:40.374019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:40.374019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:43:40.374019 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:43:40.374748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:43:40.374748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:43:40.374748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:40.374748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:40.374748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:40.374748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:43:40.374748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:40.374748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:40.374748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:40.374748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:43:40.374748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:40.374748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:43:40.374748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:43:40.374748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:40.374748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:43:40.374748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:40.374748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:43:40.374748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:43:40.374748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:40.375586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:43:40.375586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:40.375586 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:43:40.375586 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:43:40.375586 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:43:40.375586 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:43:40.375904 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:43:40.375904 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:43:40.376119 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:40.376618 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:43:40.376618 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:40.376618 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:43:40.376618 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:43:40.376618 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:40.376618 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:43:40.376618 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:40.376618 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:43:40.377379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:40.377379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:43:40.377379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:43:40.377379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:43:40.377379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:40.377379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:43:40.377379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:40.377379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:43:40.377379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:40.377379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:43:40.377379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:40.377379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:43:40.377379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:40.377379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:43:40.377379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:43:40.377886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:40.377886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:43:40.377886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:40.377886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:43:40.377886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:43:40.377886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:40.377886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:43:40.377886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:40.377886 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:43:40.378202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:40.378202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:43:40.378202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:40.378202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:43:40.378202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:43:40.378202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:43:40.378202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:43:40.378202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:43:40.378202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:43:40.378202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:43:40.378202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:40.378202 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:40.378595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:40.378595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:43:40.378595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:40.378595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:40.378595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:40.378595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:43:40.378595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:40.378595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:43:40.378595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:43:40.378595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:40.378595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:43:40.378595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:40.378595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:43:40.378595 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:43:40.378955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:40.378955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:43:40.378955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:40.378955 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:43:40.379218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:40.379218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:43:40.379218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:43:40.379218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:43:40.379218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:43:40.379218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:43:40.379218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:40.379218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:43:40.379218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:43:40.379218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:43:40.379218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:43:40.379218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:43:40.379218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:40.379218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:40.379218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:43:40.379218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:43:40.379701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:43:40.379701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:43:40.379701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:43:40.379701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:43:40.379701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:43:40.379701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:40.379701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:40.379701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:40.379701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:43:40.379701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:40.379701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:40.379701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:40.379701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:43:40.379701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:40.379701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:43:40.379701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:43:40.379701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:40.379701 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:43:40.380201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:40.380201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:43:40.380201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:43:40.380201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:40.380201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:43:40.380201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:40.380201 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:43:40.380979 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:43:40.381319 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:43:40.381463 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:43:40.381591 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:43:40.381832 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:43:40.382054 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:40.382506 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:43:40.382506 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:40.382506 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:43:40.382506 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:43:40.382506 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:40.382506 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:43:40.382506 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:40.382506 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:43:40.383193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:40.383193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:43:40.383193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:43:40.383193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:43:40.383193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:40.383193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:43:40.383193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:40.383193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:43:40.383193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:40.383193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:43:40.383193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:40.383193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:43:40.383193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:40.383193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:43:40.383193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:43:40.383644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:40.383644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:43:40.383644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:40.383644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:43:40.383644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:43:40.383644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:40.383644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:43:40.383644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:40.383644 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:43:40.384077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:40.384077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:43:40.384077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:40.384077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:43:40.384077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:43:40.384077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:43:40.384077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:43:40.384077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:43:40.384077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:43:40.384077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:40.384077 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:40.384426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:40.384426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:43:40.384426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:40.384426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:40.384426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:40.384426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:43:40.384426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:40.384426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:43:40.384426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:43:40.384426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:40.384426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:43:40.384426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:40.384426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:43:40.384426 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:43:40.384860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:40.384860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:43:40.384860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:40.384860 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:43:40.385179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:40.385179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:43:40.385179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:43:40.385179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:43:40.385179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:43:40.385179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:43:40.385179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:40.385179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:43:40.385179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:43:40.385179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:43:40.385179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:43:40.385179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:43:40.385179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:40.385179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:40.385179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:43:40.385179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:43:40.385179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:43:40.385689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:43:40.385689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:43:40.385689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:43:40.385689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:40.385689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:40.385689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:40.385689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:43:40.385689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:40.385689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:40.385689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:40.385689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:43:40.385689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:40.385689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:43:40.385689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:43:40.385689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:40.385689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:43:40.385689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:40.385689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:43:40.385689 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:43:40.386188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:40.386188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:43:40.386188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:40.386188 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:43:40.386601 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:43:40.386897 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:43:40.387011 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:43:40.387155 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:43:40.387372 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:43:40.387623 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:43:40.387750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:40.388185 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:43:40.388185 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:40.388185 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:43:40.388185 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:43:40.388185 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:40.388185 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:43:40.388185 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:40.388185 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:43:40.389030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:40.389030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:43:40.389030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:40.389030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:43:40.389030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:43:40.389030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:43:40.389030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:43:40.389030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:40.389030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:40.389030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:40.389030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:43:40.389030 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:40.389436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:40.389436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:40.389436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:43:40.389436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:40.389436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:43:40.389436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:43:40.389436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:40.389436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:43:40.389436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:40.389436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:43:40.389436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:43:40.389436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:40.389436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:43:40.389824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:40.389824 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:43:40.390029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:40.390029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:43:40.390029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:43:40.390029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:43:40.390029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:43:40.390029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:43:40.390029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:40.390029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:43:40.390029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:43:40.390029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:43:40.390029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:43:40.390029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:43:40.390029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:40.390029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:40.390029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:43:40.390029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:43:40.390496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:43:40.390496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:43:40.390496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:40.390496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:40.390496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:40.390496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:43:40.390496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:40.390496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:40.390496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:40.390496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:43:40.390496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:40.390496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:43:40.390496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:43:40.390496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:40.390496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:43:40.390496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:40.390496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:43:40.390496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:43:40.390496 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:40.391026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:43:40.391026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:40.391026 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:43:45.136680 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:43:45.136680 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:43:45.136680 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:43:45.137702 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:43:45.170128 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:43:45.170387 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:45.170959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:43:45.170959 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:45.170959 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:43:45.170959 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:43:45.170959 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:45.170959 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:43:45.170959 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:45.170959 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:43:45.171691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:45.171691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:43:45.171691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:45.171691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:43:45.171691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:43:45.171691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:43:45.171691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:43:45.171691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:45.171691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:45.171691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:45.171691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:43:45.171691 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:45.172364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:45.172364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:45.172364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:43:45.172364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:45.172364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:43:45.172364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:43:45.172364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:45.172364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:43:45.172364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:45.172364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:43:45.172364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:43:45.172364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:45.172364 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:43:45.173094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:45.173094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:43:45.173094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:45.173094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:43:45.173094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:43:45.173094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:43:45.173094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:43:45.173094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:43:45.173094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:45.173094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:43:45.173094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:43:45.173094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:43:45.173094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:43:45.173094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:43:45.173094 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:45.173943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:45.173943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:43:45.173943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:43:45.173943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:43:45.173943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:43:45.173943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:45.173943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:45.173943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:45.173943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:43:45.173943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:45.173943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:45.173943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:45.173943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:43:45.173943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:45.173943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:43:45.173943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:43:45.173943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:45.173943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:43:45.173943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:45.175563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:43:45.175563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:43:45.175563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:45.175563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:43:45.175563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:45.175563 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:43:45.368479 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:43:45.369162 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:43:45.369421 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:43:45.370744 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:43:45.371172 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:43:45.371319 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:45.372334 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:43:45.372334 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:45.372334 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:43:45.372334 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:43:45.372334 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:45.372334 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:43:45.372334 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:45.372334 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:43:45.373959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:45.373959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:43:45.373959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:45.373959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:43:45.373959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:43:45.373959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:43:45.373959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:43:45.373959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:45.373959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:45.373959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:45.373959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:43:45.373959 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:45.374605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:45.374605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:45.374605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:43:45.374605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:45.374605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:43:45.374605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:43:45.374605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:45.374605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:43:45.374605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:45.374605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:43:45.374605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:43:45.374605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:45.374605 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:43:45.375155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:45.375155 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:43:45.376422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:45.376422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:43:45.376422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:43:45.376422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:43:45.376422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:43:45.376422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:43:45.376422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:45.376422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:43:45.376422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:43:45.376422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:43:45.376422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:43:45.376422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:43:45.376422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:45.376422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:45.376422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:43:45.376422 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:43:45.376956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:43:45.376956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:43:45.376956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:45.376956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:45.376956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:45.376956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:43:45.376956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:45.376956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:45.376956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:45.376956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:43:45.376956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:45.376956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:43:45.376956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:43:45.376956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:45.376956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:43:45.376956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:45.376956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:43:45.376956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:43:45.376956 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:45.377500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:43:45.377500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:45.377500 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:43:45.378128 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:43:45.378630 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:43:45.379039 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:43:45.379140 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:43:45.379528 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:43:45.379758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:45.380573 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:43:45.380573 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:45.380573 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:43:45.380573 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:43:45.380573 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:45.380573 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:43:45.380573 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:45.380573 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:43:45.381232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:45.381232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:43:45.381232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:43:45.381232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:43:45.381232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:45.381232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:43:45.381232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:45.381232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:43:45.381232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:45.381232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:43:45.381232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:45.381232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:43:45.381232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:45.381232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:43:45.381232 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:43:45.381756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:45.381756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:43:45.381756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:45.381756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:43:45.381756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:43:45.381756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:45.381756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:43:45.381756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:45.381756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:43:45.381756 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:43:45.382419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:45.382419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:43:45.382419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:45.382419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:43:45.382419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:43:45.382419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:43:45.382419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:43:45.382419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:43:45.382419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:43:45.382419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:43:45.382419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:45.382419 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:45.383005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:45.383005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:43:45.383005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:45.383005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:45.383005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:45.383005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:43:45.383005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:45.383005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:43:45.383005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:43:45.383005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:45.383005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:43:45.383005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:45.383005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:43:45.383005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:43:45.383393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:45.383393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:43:45.383393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:45.383393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:43:45.383393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:45.383393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:43:45.383393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:43:45.383393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:43:45.383393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:43:45.383393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:43:45.383393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:45.383393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:43:45.383393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:43:45.383393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:43:45.384035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:43:45.384035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:43:45.384035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:45.384035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:45.384035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:43:45.384035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:43:45.384035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:43:45.384035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:43:45.384035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:43:45.384035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:43:45.384035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:43:45.384035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:45.384035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:45.384035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:45.384035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:43:45.384035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:45.384035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:45.384035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:45.384035 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:43:45.384594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:45.384594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:43:45.384594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:43:45.384594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:45.384594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:43:45.384594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:45.384594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:43:45.384594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:43:45.384594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:45.384594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:43:45.384594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:45.384594 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:43:45.384594 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:43:45.384594 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:43:45.385313 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:43:45.385313 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:43:45.385313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:45.385313 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:43:45.385313 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:45.385313 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:43:45.385313 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:43:45.385313 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:45.385313 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:43:45.385313 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:45.385313 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:43:45.386154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:45.386154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:43:45.386154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:43:45.386154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:43:45.386154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:45.386154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:43:45.386154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:45.386154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:43:45.386154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:45.386154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:43:45.386154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:45.386154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:43:45.386154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:45.386154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:43:45.386154 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:43:45.386972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:45.386972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:43:45.386972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:45.386972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:43:45.386972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:43:45.386972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:45.386972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:43:45.386972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:45.386972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:43:45.386972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:45.386972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:43:45.386972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:45.386972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:43:45.386972 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:43:45.387661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:43:45.387661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:43:45.387661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:43:45.387661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:43:45.387661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:45.387661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:45.387661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:45.387661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:43:45.387661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:45.387661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:45.387661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:45.387661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:43:45.387661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:45.388378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:43:45.388378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:43:45.388378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:45.388378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:43:45.388378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:45.388378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:43:45.388378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:43:45.388378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:45.388378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:43:45.388378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:45.388378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:43:45.388378 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:45.389050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:43:45.389050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:43:45.389050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:43:45.389050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:43:45.389050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:43:45.389050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:45.389050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:43:45.389050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:43:45.389050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:43:45.389050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:43:45.389050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:43:45.389050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:45.389050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:45.389050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:43:45.389050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:43:45.389050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:43:45.389050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:43:45.389050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:43:45.389050 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:43:45.390070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:45.390070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:45.390070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:45.390070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:43:45.390070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:45.390070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:45.390070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:45.390070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:43:45.390070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:45.390070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:43:45.390070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:43:45.390070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:45.390070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:43:45.390070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:45.390070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:43:45.390070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:43:45.390070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:45.390070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:43:45.390070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:45.390070 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:43:45.391135 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:43:45.391135 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:43:45.391135 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:43:45.391135 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:43:45.391135 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:43:45.391135 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:43:45.391571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:45.391571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:43:45.391571 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:45.391571 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:43:45.391571 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:43:45.391571 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:45.391571 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:43:45.391571 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:45.391571 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:43:45.391571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:45.391571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:43:45.391571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:45.391571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:43:45.391571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:43:45.391571 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:43:45.392327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:43:45.392327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:45.392327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:45.392327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:45.392327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:43:45.392327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:45.392327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:45.392327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:45.392327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:43:45.392327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:45.392327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:43:45.392327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:43:45.392327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:45.392327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:43:45.392836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:45.392836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:43:45.392836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:43:45.392836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:45.392836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:43:45.392836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:45.392836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:43:45.392836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:45.392836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:43:45.392836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:43:45.392836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:43:45.392836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:43:45.392836 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:43:45.393218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:45.393218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:43:45.393218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:43:45.393218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:43:45.393218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:43:45.393218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:43:45.393218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:45.393218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:45.393218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:43:45.393218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:43:45.393218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:43:45.393218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:43:45.393218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:45.393218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:45.393218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:45.393218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:43:45.393218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:45.393218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:45.393218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:45.393218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:43:45.393626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:45.393626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:43:45.393626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:43:45.393626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:45.393626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:43:45.393626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:45.393626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:43:45.393626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:43:45.393626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:45.393626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:43:45.393626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:45.393626 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:43:50.136784 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:43:50.138044 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:43:50.138270 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:43:50.139545 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:43:50.170937 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:43:50.171189 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:50.171891 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:43:50.171891 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:50.171891 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:43:50.171891 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:43:50.171891 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:50.171891 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:43:50.171891 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:50.171891 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:43:50.172638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:50.172638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:43:50.172638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:50.172638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:43:50.172638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:43:50.172638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:43:50.172638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:43:50.172638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:50.172638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:50.172638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:50.172638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:43:50.172638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:50.173321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:50.173321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:50.173321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:43:50.173321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:50.173321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:43:50.173321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:43:50.173321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:50.173321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:43:50.173321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:50.173321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:43:50.173321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:43:50.173321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:50.173321 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:43:50.174056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:50.174056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:43:50.174056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:50.174056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:43:50.174056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:43:50.174056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:43:50.174056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:43:50.174056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:43:50.174056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:50.174056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:43:50.174056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:43:50.174056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:43:50.174056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:43:50.174056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:43:50.174056 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:50.174928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:50.174928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:43:50.174928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:43:50.174928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:43:50.174928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:43:50.174928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:50.174928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:50.174928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:50.174928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:43:50.174928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:50.174928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:50.174928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:50.174928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:43:50.174928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:50.174928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:43:50.174928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:43:50.174928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:50.174928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:43:50.174928 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:50.175858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:43:50.175858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:43:50.175858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:50.175858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:43:50.175858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:50.175858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:43:50.369078 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:43:50.369712 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:43:50.369907 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:43:50.371148 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:43:50.371562 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:43:50.371865 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:50.372967 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:43:50.372967 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:50.372967 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:43:50.372967 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:43:50.372967 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:50.372967 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:43:50.372967 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:50.372967 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:43:50.374526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:50.374526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:43:50.374526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:50.374526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:43:50.374526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:43:50.374526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:43:50.374526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:43:50.374526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:50.374526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:50.374526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:50.374526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:43:50.374526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:50.374882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:50.374882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:50.374882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:43:50.374882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:50.374882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:43:50.374882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:43:50.374882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:50.374882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:43:50.374882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:50.374882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:43:50.374882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:43:50.374882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:50.374882 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:43:50.375216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:50.375216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:43:50.376179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:50.376179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:43:50.376179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:43:50.376179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:43:50.376179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:43:50.376179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:43:50.376179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:50.376179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:43:50.376179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:43:50.376179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:43:50.376179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:43:50.376179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:43:50.376179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:50.376179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:50.376179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:43:50.376179 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:43:50.376623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:43:50.376623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:43:50.376623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:50.376623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:50.376623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:50.376623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:43:50.376623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:50.376623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:50.376623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:50.376623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:43:50.376623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:50.376623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:43:50.376623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:43:50.376623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:50.376623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:43:50.376623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:50.376623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:43:50.376623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:43:50.376623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:50.377097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:43:50.377097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:50.377097 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:43:50.378625 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:43:50.378915 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:43:50.379027 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:43:50.379145 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:43:50.379487 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:43:50.379686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:50.380223 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:43:50.380223 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:50.380223 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:43:50.380223 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:43:50.380223 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:50.380223 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:43:50.380223 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:50.380223 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:43:50.381032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:50.381032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:43:50.381032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:43:50.381032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:43:50.381032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:50.381032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:43:50.381032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:50.381032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:43:50.381032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:50.381032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:43:50.381032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:50.381032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:43:50.381032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:50.381032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:43:50.381032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:43:50.381492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:50.381492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:43:50.381492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:50.381492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:43:50.381492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:43:50.381492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:50.381492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:43:50.381492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:50.381492 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:43:50.381837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:50.381837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:43:50.381837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:50.381837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:43:50.381837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:43:50.381837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:43:50.381837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:43:50.381837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:43:50.381837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:43:50.381837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:43:50.381837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:50.381837 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:50.382194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:50.382194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:43:50.382194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:50.382194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:50.382194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:50.382194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:43:50.382194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:50.382194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:43:50.382194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:43:50.382194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:50.382194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:43:50.382194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:50.382194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:43:50.382194 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:43:50.382559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:50.382559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:43:50.382559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:50.382559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:43:50.382854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:50.382854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:43:50.382854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:43:50.382854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:43:50.382854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:43:50.382854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:43:50.382854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:50.382854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:43:50.382854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:43:50.382854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:43:50.382854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:43:50.382854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:43:50.382854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:50.382854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:50.382854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:43:50.382854 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:43:50.383261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:43:50.383261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:43:50.383261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:43:50.383261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:43:50.383261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:43:50.383261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:50.383261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:50.383261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:50.383261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:43:50.383261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:50.383261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:50.383261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:50.383261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:43:50.383261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:50.383261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:43:50.383261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:43:50.383261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:50.383261 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:43:50.383746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:50.383746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:43:50.383746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:43:50.383746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:50.383746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:43:50.383746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:50.383746 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:43:50.383746 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:43:50.384026 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:43:50.384138 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:43:50.384267 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:43:50.384630 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:43:50.384947 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:50.385673 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:43:50.385673 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:50.385673 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:43:50.385673 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:43:50.385673 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:50.385673 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:43:50.385673 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:50.385673 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:43:50.386436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:50.386436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:43:50.386436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:43:50.386436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:43:50.386436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:50.386436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:43:50.386436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:50.386436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:43:50.386436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:50.386436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:43:50.386436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:50.386436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:43:50.386436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:50.386436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:43:50.386436 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:43:50.386858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:50.386858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:43:50.386858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:50.386858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:43:50.386858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:43:50.386858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:50.386858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:43:50.386858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:50.386858 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:43:50.387295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:50.387295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:43:50.387295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:50.387295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:43:50.387295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:43:50.387295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:43:50.387295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:43:50.387295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:43:50.387295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:43:50.387295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:50.387295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:50.387648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:50.387648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:43:50.387648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:50.387648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:50.387648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:50.387648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:43:50.387648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:50.387648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:43:50.387648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:43:50.387648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:50.387648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:43:50.387648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:50.387648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:43:50.387648 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:43:50.388013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:50.388013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:43:50.388013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:50.388013 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:43:50.388349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:50.388349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:43:50.388349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:43:50.388349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:43:50.388349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:43:50.388349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:43:50.388349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:50.388349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:43:50.388349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:43:50.388349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:43:50.388349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:43:50.388349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:43:50.388349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:50.388349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:50.388349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:43:50.388349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:43:50.388349 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:43:50.388826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:43:50.388826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:43:50.388826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:43:50.388826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:50.388826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:50.388826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:50.388826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:43:50.388826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:50.388826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:50.388826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:50.388826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:43:50.388826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:50.388826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:43:50.388826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:43:50.388826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:50.388826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:43:50.388826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:50.388826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:43:50.388826 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:43:50.389327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:50.389327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:43:50.389327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:50.389327 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:43:50.389327 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:43:50.389327 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:43:50.389327 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:43:50.389327 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:43:50.389590 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:43:50.389892 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:43:50.390111 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:50.390848 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:43:50.390848 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:50.390848 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:43:50.390848 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:43:50.390848 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:50.390848 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:43:50.390848 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:50.390848 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:43:50.391664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:50.391664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:43:50.391664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:50.391664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:43:50.391664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:43:50.391664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:43:50.391664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:43:50.391664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:50.391664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:50.391664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:50.391664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:43:50.391664 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:50.392027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:50.392027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:50.392027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:43:50.392027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:50.392027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:43:50.392027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:43:50.392027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:50.392027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:43:50.392027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:50.392027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:43:50.392027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:43:50.392027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:50.392027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:43:50.392366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:50.392366 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:43:50.392627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:50.392627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:43:50.392627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:43:50.392627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:43:50.392627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:43:50.392627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:43:50.392627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:50.392627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:43:50.392627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:43:50.392627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:43:50.392627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:43:50.392627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:43:50.392627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:50.392627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:50.392627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:43:50.392627 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:43:50.393051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:43:50.393051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:43:50.393051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:50.393051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:50.393051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:50.393051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:43:50.393051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:50.393051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:50.393051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:50.393051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:43:50.393051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:50.393051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:43:50.393051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:43:50.393051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:50.393051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:43:50.393051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:50.393051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:43:50.393051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:43:50.393051 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:50.393559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:43:50.393559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:50.393559 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:43:55.138108 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:43:55.138924 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:43:55.138924 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:43:55.139930 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:43:55.176320 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:43:55.176770 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:55.179259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:43:55.179259 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:55.179259 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:43:55.179259 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:43:55.179259 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:55.179259 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:43:55.179259 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:55.179259 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:43:55.179259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:55.179259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:43:55.179259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:55.179259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:43:55.179259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:43:55.179259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:43:55.179259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:43:55.179658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:55.179658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:55.179658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:55.179658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:43:55.179658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:55.179658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:55.179658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:55.179658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:43:55.179658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:55.179658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:43:55.179658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:43:55.179658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:55.179658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:43:55.179658 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:55.180003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:43:55.180003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:43:55.180003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:55.180003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:43:55.180003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:55.180003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:43:55.180003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:55.180003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:43:55.180003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:43:55.180003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:43:55.180003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:43:55.180003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:43:55.180003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:55.180003 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:43:55.180315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:43:55.180315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:43:55.180315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:43:55.180315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:43:55.180315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:55.180315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:55.180315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:43:55.180315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:43:55.180315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:43:55.180315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:43:55.180315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:55.180315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:55.180315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:55.180315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:43:55.180315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:55.180315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:55.180315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:55.180315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:43:55.180315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:55.180315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:43:55.180315 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:43:55.180834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:55.180834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:43:55.180834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:55.180834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:43:55.180834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:43:55.180834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:55.180834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:43:55.180834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:55.180834 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:43:55.369835 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:43:55.371623 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:43:55.371623 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:43:55.372256 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:43:55.372683 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:43:55.373045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:55.374540 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:43:55.374540 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:55.374540 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:43:55.374540 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:43:55.374540 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:55.374540 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:43:55.374540 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:55.374540 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:43:55.375745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:55.375745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:43:55.375745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:55.375745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:43:55.375745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:43:55.375745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:43:55.375745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:43:55.375745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:55.375745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:55.375745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:55.375745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:43:55.375745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:55.376141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:55.376141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:55.376141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:43:55.376141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:55.376141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:43:55.376141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:43:55.376141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:55.376141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:43:55.376141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:55.376141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:43:55.376141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:43:55.376141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:55.376141 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:43:55.376538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:55.376538 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:43:55.377677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:55.377677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:43:55.377677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:43:55.377677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:43:55.377677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:43:55.377677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:43:55.377677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:55.377677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:43:55.377677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:43:55.377677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:43:55.377677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:43:55.377677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:43:55.377677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:55.377677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:55.377677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:43:55.377677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:43:55.378123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:43:55.378123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:43:55.378123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:55.378123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:55.378123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:55.378123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:43:55.378123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:55.378123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:55.378123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:55.378123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:43:55.378123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:55.378123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:43:55.378123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:43:55.378123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:55.378123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:43:55.378123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:55.378123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:43:55.378123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:43:55.378123 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:55.378631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:43:55.378631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:55.378631 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:43:55.379018 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:43:55.379284 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:43:55.379414 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:43:55.379542 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:43:55.379985 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:43:55.380282 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:55.381024 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:43:55.381024 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:55.381024 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:43:55.381024 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:43:55.381024 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:55.381024 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:43:55.381024 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:55.381024 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:43:55.381812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:55.381812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:43:55.381812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:43:55.381812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:43:55.381812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:55.381812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:43:55.381812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:55.381812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:43:55.381812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:55.381812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:43:55.381812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:55.381812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:43:55.381812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:55.381812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:43:55.381812 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:43:55.382290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:55.382290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:43:55.382290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:55.382290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:43:55.382290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:43:55.382290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:55.382290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:43:55.382290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:55.382290 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:43:55.382640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:55.382640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:43:55.382640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:55.382640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:43:55.382640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:43:55.382640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:43:55.382640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:43:55.382640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:43:55.382640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:43:55.382640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:43:55.382640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:55.382640 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:55.383001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:55.383001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:43:55.383001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:55.383001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:55.383001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:55.383001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:43:55.383001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:55.383001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:43:55.383001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:43:55.383001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:55.383001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:43:55.383001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:55.383001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:43:55.383001 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:43:55.383388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:55.383388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:43:55.383388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:55.383388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:43:55.383707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:55.383707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:43:55.383707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:43:55.383707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:43:55.383707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:43:55.383707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:43:55.383707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:55.383707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:43:55.383707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:43:55.383707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:43:55.383707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:43:55.383707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:43:55.383707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:55.383707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:55.383707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:43:55.383707 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:43:55.384180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:43:55.384180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:43:55.384180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:43:55.384180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:43:55.384180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:43:55.384180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:55.384180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:55.384180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:55.384180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:43:55.384180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:55.384180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:55.384180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:55.384180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:43:55.384180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:55.384180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:43:55.384180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:43:55.384180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:55.384180 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:43:55.384740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:55.384740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:43:55.384740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:43:55.384740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:55.384740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:43:55.384740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:55.384740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:43:55.384740 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:43:55.384740 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:43:55.384740 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:43:55.385183 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:43:55.385183 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:43:55.385335 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:55.386048 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:43:55.386048 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:55.386048 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:43:55.386048 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:43:55.386048 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:55.386048 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:43:55.386048 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:55.386048 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:43:55.386953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:55.386953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:43:55.386953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:43:55.386953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:43:55.386953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:55.386953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:43:55.386953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:55.386953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:43:55.386953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:55.386953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:43:55.386953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:55.386953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:43:55.386953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:55.386953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:43:55.386953 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:43:55.387367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:55.387367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:43:55.387367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:55.387367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:43:55.387367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:43:55.387367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:55.387367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:43:55.387367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:55.387367 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:43:55.387974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:55.387974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:43:55.387974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:55.387974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:43:55.387974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:43:55.387974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:43:55.387974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:43:55.387974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:43:55.387974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:43:55.387974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:55.387974 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:55.388359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:55.388359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:43:55.388359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:55.388359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:55.388359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:55.388359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:43:55.388359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:55.388359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:43:55.388359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:43:55.388359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:55.388359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:43:55.388359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:55.388359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:43:55.388359 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:43:55.388768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:55.388768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:43:55.388768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:55.388768 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:43:55.389429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:55.389429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:43:55.389429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:43:55.389429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:43:55.389429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:43:55.389429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:43:55.389429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:55.389429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:43:55.389429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:43:55.389429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:43:55.389429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:43:55.389429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:43:55.389429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:55.389429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:55.389429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:43:55.389429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:43:55.389429 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:43:55.389943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:43:55.389943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:43:55.389943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:43:55.389943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:55.389943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:55.389943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:55.389943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:43:55.389943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:55.389943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:55.389943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:55.389943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:43:55.389943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:55.389943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:43:55.389943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:43:55.389943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:55.389943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:43:55.389943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:55.389943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:43:55.389943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:43:55.390539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:55.390539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:43:55.390539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:55.390539 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:43:55.390539 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:43:55.390539 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:43:55.390539 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:43:55.390539 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:43:55.390813 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:43:55.391075 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:43:55.391277 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:55.391985 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:43:55.391985 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:55.391985 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:43:55.391985 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:43:55.391985 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:55.391985 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:43:55.391985 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:55.391985 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:43:55.392828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:55.392828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:43:55.392828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:55.392828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:43:55.392828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:43:55.392828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:43:55.392828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:43:55.392828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:55.392828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:55.392828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:55.392828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:43:55.392828 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:55.393253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:43:55.393253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:55.393253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:43:55.393253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:55.393253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:43:55.393253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:43:55.393253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:55.393253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:43:55.393253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:55.393253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:43:55.393253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:43:55.393253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:55.393253 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:43:55.393654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:55.393654 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:43:55.394029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:55.394029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:43:55.394029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:43:55.394029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:43:55.394029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:43:55.394029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:43:55.394029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:55.394029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:43:55.394029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:43:55.394029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:43:55.394029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:43:55.394029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:43:55.394029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:43:55.394029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:43:55.394029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:43:55.394029 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:43:55.394543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:43:55.394543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:43:55.394543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:55.394543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:55.394543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:43:55.394543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:43:55.394543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:43:55.394543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:43:55.394543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:43:55.394543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:43:55.394543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:43:55.394543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:43:55.394543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:43:55.394543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:43:55.394543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:43:55.394543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:43:55.394543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:43:55.394543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:43:55.394543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:43:55.395068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:43:55.395068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:43:55.395068 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:44:00.138800 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:44:00.139547 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:44:00.139755 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:44:00.140870 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:44:00.175669 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:44:00.175930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:00.176588 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:44:00.176588 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:00.176588 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:44:00.176588 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:44:00.176588 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:00.176588 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:44:00.176588 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:00.176588 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:44:00.177574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:00.177574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:44:00.177574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:44:00.177574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:44:00.177574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:44:00.177574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:44:00.177574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:44:00.177574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:00.177574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:44:00.177574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:44:00.177574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:44:00.177574 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:00.178213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:44:00.178213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:44:00.178213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:44:00.178213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:44:00.178213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:44:00.178213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:44:00.178213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:44:00.178213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:44:00.178213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:00.178213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:44:00.178213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:44:00.178213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:00.178213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:44:00.178878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:00.178878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:44:00.178878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:00.178878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:44:00.178878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:44:00.178878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:44:00.178878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:44:00.178878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:44:00.178878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:44:00.178878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:44:00.178878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:44:00.178878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:44:00.178878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:44:00.178878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:44:00.178878 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:44:00.179676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:44:00.179676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:44:00.179676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:44:00.179676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:44:00.179676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:44:00.179676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:00.179676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:44:00.179676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:44:00.179676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:44:00.179676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:00.179676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:44:00.179676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:44:00.179676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:44:00.179676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:44:00.179676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:44:00.179676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:44:00.179676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:44:00.179676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:44:00.179676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:00.181908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:44:00.181908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:44:00.181908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:00.181908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:44:00.181908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:00.181908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:44:00.371039 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:44:00.371597 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:44:00.371786 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:44:00.372975 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:44:00.373226 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:44:00.373388 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:00.374548 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:44:00.374548 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:00.374548 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:44:00.374548 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:44:00.374548 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:00.374548 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:44:00.374548 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:00.374548 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:44:00.376082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:00.376082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:44:00.376082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:44:00.376082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:44:00.376082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:44:00.376082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:44:00.376082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:44:00.376082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:00.376082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:44:00.376082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:44:00.376082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:44:00.376082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:00.377347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:44:00.377347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:44:00.377347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:44:00.377347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:44:00.377347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:44:00.377347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:44:00.377347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:44:00.377347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:44:00.377347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:00.377347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:44:00.377347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:44:00.377347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:00.377347 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:44:00.377904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:00.377904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:44:00.377904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:00.377904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:44:00.377904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:44:00.377904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:44:00.377904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:44:00.377904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:44:00.377904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:44:00.377904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:44:00.377904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:44:00.377904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:44:00.377904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:44:00.377904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:44:00.377904 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:44:00.378553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:44:00.378553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:44:00.378553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:44:00.378553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:44:00.378553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:44:00.378553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:00.378553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:44:00.378553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:44:00.378553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:44:00.378553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:00.378553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:44:00.378553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:44:00.378553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:44:00.378553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:44:00.378553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:44:00.378553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:44:00.378553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:44:00.378553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:44:00.378553 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:00.379295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:44:00.379295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:44:00.379295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:00.379295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:44:00.379295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:00.379295 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:44:00.380678 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:44:00.380678 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:44:00.380678 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:44:00.380678 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:44:00.380678 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:44:00.381267 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:00.381628 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:44:00.381628 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:00.381628 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:44:00.381628 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:44:00.381628 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:00.381628 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:44:00.381628 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:00.381628 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:44:00.382564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:00.382564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:44:00.382564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:44:00.382564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:44:00.382564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:00.382564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:44:00.382564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:44:00.382564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:44:00.382564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:00.382564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:44:00.382564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:44:00.382564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:44:00.382564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:44:00.382564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:44:00.382564 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:44:00.384216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:44:00.384216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:44:00.384216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:00.384216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:44:00.384216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:44:00.384216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:00.384216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:44:00.384216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:00.384216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:44:00.384216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:00.384216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:44:00.384216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:44:00.384216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:44:00.384216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:44:00.384526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:44:00.384526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:44:00.384526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:44:00.384526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:44:00.384526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:44:00.384526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:00.384526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:44:00.384526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:44:00.384526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:44:00.384526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:00.384526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:44:00.384526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:44:00.384526 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:44:00.384844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:44:00.384844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:44:00.384844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:44:00.384844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:44:00.384844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:44:00.384844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:00.384844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:44:00.384844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:44:00.384844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:00.384844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:44:00.384844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:00.384844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:44:00.385938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:00.385938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:44:00.385938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:44:00.385938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:44:00.385938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:44:00.385938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:44:00.385938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:44:00.385938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:44:00.385938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:44:00.385938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:44:00.385938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:44:00.385938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:44:00.385938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:44:00.385938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:44:00.385938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:44:00.385938 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:44:00.386525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:44:00.386525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:44:00.386525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:44:00.386525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:44:00.386525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:44:00.386525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:00.386525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:44:00.386525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:44:00.386525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:44:00.386525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:00.386525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:44:00.386525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:44:00.386525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:44:00.386525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:44:00.386525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:44:00.386525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:44:00.386525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:44:00.386525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:44:00.386997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:00.386997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:44:00.386997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:44:00.386997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:00.386997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:44:00.386997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:00.386997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:44:00.386997 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:44:00.386997 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:44:00.386997 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:44:00.387199 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:44:00.387199 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:44:00.387199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:00.387199 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:44:00.387199 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:00.387199 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:44:00.387199 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:44:00.387199 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:00.387199 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:44:00.387199 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:00.387199 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:44:00.387679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:00.387679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:44:00.387679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:44:00.387679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:44:00.387679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:00.387679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:44:00.387679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:44:00.387679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:44:00.387679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:00.387679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:44:00.387679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:44:00.387679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:44:00.387679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:44:00.387679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:44:00.387679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:44:00.388397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:44:00.388397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:44:00.388397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:00.388397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:44:00.388397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:44:00.388397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:00.388397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:44:00.388397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:00.388397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:44:00.388397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:00.388397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:44:00.388397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:44:00.388397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:44:00.388397 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:44:00.388887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:44:00.388887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:44:00.388887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:44:00.388887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:44:00.388887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:00.388887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:44:00.388887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:44:00.388887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:44:00.388887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:00.388887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:44:00.388887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:44:00.388887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:44:00.388887 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:44:00.389437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:44:00.389437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:44:00.389437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:44:00.389437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:44:00.389437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:00.389437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:44:00.389437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:44:00.389437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:00.389437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:44:00.389437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:00.389437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:44:00.389437 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:00.389906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:44:00.389906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:44:00.389906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:44:00.389906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:44:00.389906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:44:00.389906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:44:00.389906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:44:00.389906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:44:00.389906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:44:00.389906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:44:00.389906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:44:00.389906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:44:00.389906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:44:00.389906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:44:00.389906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:44:00.389906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:44:00.389906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:44:00.389906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:44:00.389906 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:44:00.390616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:00.390616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:44:00.390616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:44:00.390616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:44:00.390616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:00.390616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:44:00.390616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:44:00.390616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:44:00.390616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:44:00.390616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:44:00.390616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:44:00.390616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:44:00.390616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:44:00.390616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:00.390616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:44:00.390616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:44:00.390616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:00.390616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:44:00.390616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:00.390616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:44:00.390965 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:44:00.390965 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:44:00.390965 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:44:00.390965 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:44:00.391256 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:44:00.391583 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:44:00.391815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:00.392781 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:44:00.392781 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:00.392781 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:44:00.392781 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:44:00.392781 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:00.392781 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:44:00.392781 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:00.392781 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:44:00.393593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:00.393593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:44:00.393593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:44:00.393593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:44:00.393593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:44:00.393593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:44:00.393593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:44:00.393593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:00.393593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:44:00.393593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:44:00.393593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:44:00.393593 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:00.394178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:44:00.394178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:44:00.394178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:44:00.394178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:44:00.394178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:44:00.394178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:44:00.394178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:44:00.394178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:44:00.394178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:00.394178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:44:00.394178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:44:00.394178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:00.394178 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:44:00.394420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:00.394420 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:44:00.394584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:00.394584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:44:00.394584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:44:00.394584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:44:00.394584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:44:00.394584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:44:00.394584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:44:00.394584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:44:00.394584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:44:00.394584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:44:00.394584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:44:00.394584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:44:00.394584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:44:00.394584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:44:00.394584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:44:00.394584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:44:00.395213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:44:00.395213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:44:00.395213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:00.395213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:44:00.395213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:44:00.395213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:44:00.395213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:00.395213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:44:00.395213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:44:00.395213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:44:00.395213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:44:00.395213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:44:00.395213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:44:00.395213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:44:00.395213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:44:00.395213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:00.395213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:44:00.395213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:44:00.395213 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:00.395646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:44:00.395646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:00.395646 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:44:05.139494 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:44:05.140151 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:44:05.140151 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:44:05.141244 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:44:05.169449 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:44:05.169832 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:05.170599 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:44:05.170599 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:05.170599 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:44:05.170599 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:44:05.170599 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:05.170599 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:44:05.170599 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:05.170599 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:44:05.171471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.031 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:05.171471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:44:05.171471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:44:05.171471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:44:05.171471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:44:05.171471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:44:05.171471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:44:05.171471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:05.171471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:44:05.171471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:44:05.171471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:44:05.171471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:05.172069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:44:05.172069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:44:05.172069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:44:05.172069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:44:05.172069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:44:05.172069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:44:05.172069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:44:05.172069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:44:05.172069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:05.172069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:44:05.172069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:44:05.172069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:05.172069 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:44:05.172541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:05.172541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:44:05.172541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:05.172541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:44:05.172541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:44:05.172541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:44:05.172541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:44:05.172541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:44:05.172541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:44:05.172541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:44:05.172541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:44:05.172541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:44:05.172541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:44:05.172541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:44:05.172541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:44:05.173060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:44:05.173060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:44:05.173060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:44:05.173060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:44:05.173060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:44:05.173060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:05.173060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:44:05.173060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:44:05.173060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:44:05.173060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:05.173060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:44:05.173060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:44:05.173060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:44:05.173060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:44:05.173060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:44:05.173060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:44:05.173060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:44:05.173060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:44:05.173060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:05.173745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:44:05.173745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:44:05.173745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:05.173745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:44:05.173745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:05.173745 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:44:05.371552 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:44:05.372134 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:44:05.372293 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:44:05.373360 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:44:05.373625 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:44:05.373791 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:05.374547 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:44:05.374547 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:05.374547 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:44:05.374547 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:44:05.374547 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:05.374547 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:44:05.374547 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:05.374547 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:44:05.376046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:05.376046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:44:05.376046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:44:05.376046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:44:05.376046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:44:05.376046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:44:05.376046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:44:05.376046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:05.376046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:44:05.376046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:44:05.376046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:44:05.376046 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:05.376785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:44:05.376785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:44:05.376785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:44:05.376785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:44:05.376785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:44:05.376785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:44:05.376785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:44:05.376785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:44:05.376785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:05.376785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:44:05.376785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:44:05.376785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:05.376785 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:44:05.377363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:05.377363 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:44:05.377924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:05.377924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:44:05.377924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:44:05.377924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:44:05.377924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:44:05.377924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:44:05.377924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:44:05.377924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:44:05.377924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:44:05.377924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:44:05.377924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:44:05.377924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:44:05.377924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:44:05.377924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:44:05.377924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:44:05.377924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:44:05.378750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:44:05.378750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:44:05.378750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:05.378750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:44:05.378750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:44:05.378750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:44:05.378750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:05.378750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:44:05.378750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:44:05.378750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:44:05.378750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:44:05.378750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:44:05.378750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:44:05.378750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:44:05.378750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:44:05.378750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:05.378750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:44:05.378750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:44:05.378750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:05.379738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:44:05.379738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:05.379738 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:44:05.380049 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:44:05.380293 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:44:05.380381 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:44:05.380526 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:44:05.381375 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:44:05.382238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:05.382969 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:44:05.382969 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:05.382969 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:44:05.382969 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:44:05.382969 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:05.382969 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:44:05.382969 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:05.382969 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:44:05.383895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:05.383895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:44:05.383895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:44:05.383895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:44:05.383895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:05.383895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:44:05.383895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:44:05.383895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:44:05.383895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:05.383895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:44:05.383895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:44:05.383895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:44:05.383895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:44:05.383895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:44:05.383895 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:44:05.384383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:44:05.384383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:44:05.384383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:05.384383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:44:05.384383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:44:05.384383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:05.384383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:44:05.384383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:05.384383 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:44:05.384727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:05.384727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:44:05.384727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:44:05.384727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:44:05.384727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:44:05.384727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:44:05.384727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:44:05.384727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:44:05.384727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:44:05.384727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:44:05.384727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:05.384727 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:44:05.385095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:44:05.385095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:44:05.385095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:05.385095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:44:05.385095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:44:05.385095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:44:05.385095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:44:05.385095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:44:05.385095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:44:05.385095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:44:05.385095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:44:05.385095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:05.385095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:44:05.385095 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:44:05.385457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:05.385457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:44:05.385457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:05.385457 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:44:05.385720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:05.385720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:44:05.385720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:44:05.385720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:44:05.385720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:44:05.385720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:44:05.385720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:44:05.385720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:44:05.385720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:44:05.385720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:44:05.385720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:44:05.385720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:44:05.385720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:44:05.385720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:44:05.385720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:44:05.385720 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:44:05.386174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:44:05.386174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:44:05.386174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:44:05.386174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:44:05.386174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:44:05.386174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:05.386174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:44:05.386174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:44:05.386174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:44:05.386174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:05.386174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:44:05.386174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:44:05.386174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:44:05.386174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:44:05.386174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:44:05.386174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:44:05.386174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:44:05.386174 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:44:05.386721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:05.386721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:44:05.386721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:44:05.386721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:05.386721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:44:05.386721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:05.386721 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:44:05.386721 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:44:05.386721 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:44:05.386721 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:44:05.387039 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:44:05.387272 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:44:05.387749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:05.388625 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:44:05.388625 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:05.388625 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:44:05.388625 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:44:05.388625 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:05.388625 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:44:05.388625 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:05.388625 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:44:05.389346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:05.389346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:44:05.389346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:44:05.389346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:44:05.389346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:05.389346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:44:05.389346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:44:05.389346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:44:05.389346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:05.389346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:44:05.389346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:44:05.389346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:44:05.389346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:44:05.389346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:44:05.389346 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:44:05.389833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:44:05.389833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:44:05.389833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:05.389833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:44:05.389833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:44:05.389833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:05.389833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:44:05.389833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:05.389833 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:44:05.390246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:05.390246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:44:05.390246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:44:05.390246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:44:05.390246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:44:05.390246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:44:05.390246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:44:05.390246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:44:05.390246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:44:05.390246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:05.390246 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:44:05.390610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:44:05.390610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:44:05.390610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:05.390610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:44:05.390610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:44:05.390610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:44:05.390610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:44:05.390610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:44:05.390610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:44:05.390610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:44:05.390610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:44:05.390610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:05.390610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:44:05.390610 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:44:05.390946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:05.390946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:44:05.390946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:05.390946 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:44:05.391297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:05.391297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:44:05.391297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:44:05.391297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:44:05.391297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:44:05.391297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:44:05.391297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:44:05.391297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:44:05.391297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:44:05.391297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:44:05.391297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:44:05.391297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:44:05.391297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:44:05.391297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:44:05.391297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:44:05.391297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:44:05.391297 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:44:05.391756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:44:05.391756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:44:05.391756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:44:05.391756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:05.391756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:44:05.391756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:44:05.391756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:44:05.391756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:05.391756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:44:05.391756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:44:05.391756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:44:05.391756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:44:05.391756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:44:05.391756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:44:05.391756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:44:05.391756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:44:05.391756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:05.391756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:44:05.391756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:44:05.392237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:05.392237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:44:05.392237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:05.392237 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:44:05.392237 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:44:05.392237 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:44:05.392237 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:44:05.392237 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:44:05.392519 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:44:05.392926 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:44:05.393193 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:05.394291 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:44:05.394291 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:05.394291 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:44:05.394291 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:44:05.394291 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:05.394291 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:44:05.394291 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:05.394291 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:44:05.395875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:05.395875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:44:05.395875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:44:05.395875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:44:05.395875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:44:05.395875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:44:05.395875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:44:05.395875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:05.395875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:44:05.395875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:44:05.395875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:44:05.395875 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:05.396245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:44:05.396245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:44:05.396245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:44:05.396245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:44:05.396245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:44:05.396245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:44:05.396245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:44:05.396245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:44:05.396245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:05.396245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:44:05.396245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:44:05.396245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:05.396245 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:44:05.396638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:05.396638 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:44:05.397679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:05.397679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:44:05.397679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:44:05.397679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:44:05.397679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:44:05.397679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:44:05.397679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:44:05.397679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:44:05.397679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:44:05.397679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:44:05.397679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:44:05.397679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:44:05.397679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:44:05.397679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:44:05.397679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:44:05.397679 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:44:05.398129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:44:05.398129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:44:05.398129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:05.398129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:44:05.398129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:44:05.398129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:44:05.398129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:05.398129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:44:05.398129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:44:05.398129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:44:05.398129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:44:05.398129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:44:05.398129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:44:05.398129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:44:05.398129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:44:05.398129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:05.398129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:44:05.398129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:44:05.398129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:05.398736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:44:05.398736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:05.398736 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:44:10.141004 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:44:10.142045 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:44:10.142045 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:44:10.143892 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:44:10.180976 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:44:10.181167 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:10.181659 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:44:10.181659 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:10.181659 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:44:10.181659 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:44:10.181659 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:10.181659 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:44:10.181659 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:10.181659 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:44:10.182434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:10.182434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:44:10.182434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:44:10.182434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:44:10.182434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:44:10.182434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:44:10.182434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:44:10.182434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:10.182434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:44:10.182434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:44:10.182434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:44:10.182434 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:10.183098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:44:10.183098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:44:10.183098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:44:10.183098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:44:10.183098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:44:10.183098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:44:10.183098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:44:10.183098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:44:10.183098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:10.183098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:44:10.183098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:44:10.183098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:10.183098 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:44:10.183876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:10.183876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:44:10.183876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:10.183876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:44:10.183876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:44:10.183876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:44:10.183876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:44:10.183876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:44:10.183876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:44:10.183876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:44:10.183876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:44:10.183876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:44:10.183876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:44:10.183876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:44:10.183876 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:44:10.184777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:44:10.184777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:44:10.184777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:44:10.184777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:44:10.184777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:44:10.184777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:10.184777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:44:10.184777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:44:10.184777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:44:10.184777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:10.184777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:44:10.184777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:44:10.184777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:44:10.184777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:44:10.184777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:44:10.184777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:44:10.184777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:44:10.184777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:44:10.184777 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:10.185645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:44:10.185645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:44:10.185645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:10.185645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:44:10.185645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:10.185645 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:44:10.372210 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:44:10.372690 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:44:10.372816 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:44:10.373641 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:44:10.373878 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:44:10.374045 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:10.374686 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:44:10.374686 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:10.374686 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:44:10.374686 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:44:10.374686 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:10.374686 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:44:10.374686 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:10.374686 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:44:10.375819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:10.375819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:44:10.375819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:44:10.375819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:44:10.375819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:44:10.375819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:44:10.375819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:44:10.375819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:10.375819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:44:10.375819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:44:10.375819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:44:10.375819 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:10.376466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:44:10.376466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:44:10.376466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:44:10.376466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:44:10.376466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:44:10.376466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:44:10.376466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:44:10.376466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:44:10.376466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:10.376466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:44:10.376466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:44:10.376466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:10.376466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:44:10.376815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:10.376815 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:44:10.376881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:10.376881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:44:10.376881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:44:10.376881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:44:10.376881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:44:10.376881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:44:10.376881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:44:10.376881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:44:10.376881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:44:10.376881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:44:10.376881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:44:10.376881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:44:10.376881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:44:10.376881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:44:10.376881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:44:10.376881 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:44:10.377344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:44:10.377344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:44:10.377344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:10.377344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:44:10.377344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:44:10.377344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:44:10.377344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:10.377344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:44:10.377344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:44:10.377344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:44:10.377344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:44:10.377344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:44:10.377344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:44:10.377344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:44:10.377344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:44:10.377344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:10.377344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:44:10.377344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:44:10.377344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:10.377943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:44:10.377943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:10.377943 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:44:10.380730 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:44:10.381113 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:44:10.381315 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:44:10.381379 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:44:10.381832 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:44:10.382137 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:10.382930 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:44:10.382930 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:10.382930 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:44:10.382930 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:44:10.382930 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:10.382930 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:44:10.382930 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:10.382930 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:44:10.383756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:10.383756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:44:10.383756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:44:10.383756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:44:10.383756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:10.383756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:44:10.383756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:44:10.383756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:44:10.383756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:10.383756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:44:10.383756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:44:10.383756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:44:10.383756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:44:10.383756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:44:10.383756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:44:10.384348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:44:10.384348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:44:10.384348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:10.384348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:44:10.384348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:44:10.384348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:10.384348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:44:10.384348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:10.384348 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:44:10.384652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:10.384652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:44:10.384652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:44:10.384652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:44:10.384652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:44:10.384652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:44:10.384652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:44:10.384652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:44:10.384652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:44:10.384652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:44:10.384652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:10.384652 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:44:10.385025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:44:10.385025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:44:10.385025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:10.385025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:44:10.385025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:44:10.385025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:44:10.385025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:44:10.385025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:44:10.385025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:44:10.385025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:44:10.385025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:44:10.385025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:10.385025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:44:10.385025 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:44:10.385398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:10.385398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:44:10.385398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:10.385398 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:44:10.385810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:10.385810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:44:10.385810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:44:10.385810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:44:10.385810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:44:10.385810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:44:10.385810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:44:10.385810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:44:10.385810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:44:10.385810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:44:10.385810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:44:10.385810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:44:10.385810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:44:10.385810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:44:10.385810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:44:10.385810 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:44:10.386279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:44:10.386279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:44:10.386279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:44:10.386279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:44:10.386279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:44:10.386279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:10.386279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:44:10.386279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:44:10.386279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:44:10.386279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:10.386279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:44:10.386279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:44:10.386279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:44:10.386279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:44:10.386279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:44:10.386279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:44:10.386279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:44:10.386279 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:44:10.386840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:10.386840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:44:10.386840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:44:10.386840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:10.386840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:44:10.386840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:10.386840 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:44:10.386840 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:44:10.386840 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:44:10.386840 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:44:10.387256 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:44:10.387301 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:44:10.387725 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:10.388567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:44:10.388567 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:10.388567 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:44:10.388567 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:44:10.388567 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:10.388567 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:44:10.388567 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:10.388567 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:44:10.389216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:10.389216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:44:10.389216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:44:10.389216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:44:10.389216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:10.389216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:44:10.389216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:44:10.389216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:44:10.389216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:10.389216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:44:10.389216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:44:10.389216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:44:10.389216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:44:10.389216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:44:10.389216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:44:10.389677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:44:10.389677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:44:10.389677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:10.389677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:44:10.389677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:44:10.389677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:10.389677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:44:10.389677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:10.389677 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:44:10.390129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:10.390129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:44:10.390129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:44:10.390129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:44:10.390129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:44:10.390129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:44:10.390129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:44:10.390129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:44:10.390129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:44:10.390129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:10.390129 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:44:10.390525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:44:10.390525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:44:10.390525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:10.390525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:44:10.390525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:44:10.390525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:44:10.390525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:44:10.390525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:44:10.390525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:44:10.390525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:44:10.390525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:44:10.390525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:10.390525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:44:10.390525 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:44:10.390877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:10.390877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:44:10.390877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:10.390877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:44:10.391561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:10.391561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:44:10.391561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:44:10.391561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:44:10.391561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:44:10.391561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:44:10.391561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:44:10.391561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:44:10.391561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:44:10.391561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:44:10.391561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:44:10.391561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:44:10.391561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:44:10.391561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:44:10.391561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:44:10.391561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:44:10.391561 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:44:10.393047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:44:10.393047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:44:10.393047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:44:10.393047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:10.393047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:44:10.393047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:44:10.393047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:44:10.393047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:10.393047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:44:10.393047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:44:10.393047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:44:10.393047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:44:10.393047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:44:10.393047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:44:10.393047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:44:10.393047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:44:10.393047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:10.393047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:44:10.393047 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:44:10.393567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:10.393567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:44:10.393567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:10.393567 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:44:10.393567 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:44:10.393567 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:44:10.393567 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:44:10.393567 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:44:10.393809 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:44:10.393809 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:44:10.393809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:10.393809 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:44:10.393809 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:10.393809 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:44:10.393809 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:44:10.393809 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:10.393809 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:44:10.393809 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:10.393809 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:44:10.394480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:10.394480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:44:10.394480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:44:10.394480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:44:10.394480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:44:10.394480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:44:10.394480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:44:10.394480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:10.394480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:44:10.394480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:44:10.394480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:44:10.394480 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:10.395080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:44:10.395080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:44:10.395080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:44:10.395080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:44:10.395080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:44:10.395080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:44:10.395080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:44:10.395080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:44:10.395080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:10.395080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:44:10.395080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:44:10.395080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:10.395080 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:44:10.395456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:10.395456 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:44:10.395565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:10.395565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:44:10.395565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:44:10.395565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:44:10.395565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:44:10.395565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:44:10.395565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:44:10.395565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:44:10.395565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:44:10.395565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:44:10.395565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:44:10.395565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:44:10.395565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:44:10.395565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:44:10.395565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:44:10.395565 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:44:10.396032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:44:10.396032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:44:10.396032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:10.396032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:44:10.396032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:44:10.396032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:44:10.396032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:10.396032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:44:10.396032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:44:10.396032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:44:10.396032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:44:10.396032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:44:10.396032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:44:10.396032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:44:10.396032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:44:10.396032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:10.396032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:44:10.396032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:44:10.396032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:10.396499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:44:10.396499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:10.396499 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:44:15.141866 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:44:15.143064 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:44:15.143064 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:44:15.143836 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:44:15.179444 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:44:15.179710 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:15.180372 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:44:15.180372 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:15.180372 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:44:15.180372 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:44:15.180372 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:15.180372 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:44:15.180372 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:15.180372 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:44:15.181447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:15.181447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:44:15.181447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:44:15.181447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:44:15.181447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:44:15.181447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:44:15.181447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:44:15.181447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:15.181447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:44:15.181447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:44:15.181447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:44:15.181447 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:15.182149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:44:15.182149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:44:15.182149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:44:15.182149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:44:15.182149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:44:15.182149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:44:15.182149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:44:15.182149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:44:15.182149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:15.182149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:44:15.182149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:44:15.182149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:15.182149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:44:15.182935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:15.182935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:44:15.182935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:15.182935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:44:15.182935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:44:15.182935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:44:15.182935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:44:15.182935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:44:15.182935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:44:15.182935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:44:15.182935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:44:15.182935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:44:15.182935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:44:15.182935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:44:15.182935 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:44:15.183813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:44:15.183813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:44:15.183813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:44:15.183813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:44:15.183813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:44:15.183813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:15.183813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:44:15.183813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:44:15.183813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:44:15.183813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:15.183813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:44:15.183813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:44:15.183813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:44:15.183813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:44:15.183813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:44:15.183813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:44:15.183813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:44:15.183813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:44:15.183813 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:15.184889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:44:15.184889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:44:15.184889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:15.184889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:44:15.184889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:15.184889 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:44:15.372830 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:44:15.373561 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:44:15.373639 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:44:15.374822 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:44:15.375113 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:44:15.375371 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:15.376263 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:44:15.376263 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:15.376263 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:44:15.376263 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:44:15.376263 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:15.376263 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:44:15.376263 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:15.376263 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:44:15.377908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:15.377908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:44:15.377908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:44:15.377908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:44:15.377908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:44:15.377908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:44:15.377908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:44:15.377908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:15.377908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:44:15.377908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:44:15.377908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:44:15.377908 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:15.378300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:44:15.378300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:44:15.378300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:44:15.378300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:44:15.378300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:44:15.378300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:44:15.378300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:44:15.378300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:44:15.378300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:15.378300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:44:15.378300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:44:15.378300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:15.378300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:44:15.378676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:15.378676 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:44:15.379558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:15.379558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:44:15.379558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:44:15.379558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:44:15.379558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:44:15.379558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:44:15.379558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:44:15.379558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:44:15.379558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:44:15.379558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:44:15.379558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:44:15.379558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:44:15.379558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:44:15.379558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:44:15.379558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:44:15.379558 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:44:15.380466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:44:15.380466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:44:15.380466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:15.380466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:44:15.380466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:44:15.380466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:44:15.380466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:15.380466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:44:15.380466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:44:15.380466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:44:15.380466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:44:15.380466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:44:15.380466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:44:15.380466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:44:15.380466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:44:15.380466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:15.380466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:44:15.380466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:44:15.380466 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:15.380962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:44:15.380962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:15.380962 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:44:15.381192 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:44:15.381810 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:44:15.381810 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:44:15.381810 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:44:15.382225 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:44:15.382543 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:15.383078 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:44:15.383078 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:15.383078 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:44:15.383078 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:44:15.383078 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:15.383078 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:44:15.383078 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:15.383078 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:44:15.383749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:15.383749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:44:15.383749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:44:15.383749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:44:15.383749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:15.383749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:44:15.383749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:44:15.383749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:44:15.383749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:15.383749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:44:15.383749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:44:15.383749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:44:15.383749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:44:15.383749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:44:15.383749 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:44:15.384471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:44:15.384471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:44:15.384471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:15.384471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:44:15.384471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:44:15.384471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:15.384471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:44:15.384471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:15.384471 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:44:15.384756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:15.384756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:44:15.384756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:44:15.384756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:44:15.384756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:44:15.384756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:44:15.384756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:44:15.384756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:44:15.384756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:44:15.384756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:44:15.384756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:15.384756 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:44:15.385108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:44:15.385108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:44:15.385108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:15.385108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:44:15.385108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:44:15.385108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:44:15.385108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:44:15.385108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:44:15.385108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:44:15.385108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:44:15.385108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:44:15.385108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:15.385108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:44:15.385108 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:44:15.385493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:15.385493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:44:15.385493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:15.385493 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:44:15.385602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:15.385602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:44:15.385602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:44:15.385602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:44:15.385602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:44:15.385602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:44:15.385602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:44:15.385602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:44:15.385602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:44:15.385602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:44:15.385602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:44:15.385602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:44:15.385602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:44:15.385602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:44:15.385602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:44:15.385602 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:44:15.386038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:44:15.386038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:44:15.386038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:44:15.386038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:44:15.386038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:44:15.386038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:15.386038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:44:15.386038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:44:15.386038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:44:15.386038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:15.386038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:44:15.386038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:44:15.386038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:44:15.386038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:44:15.386038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:44:15.386038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:44:15.386038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:44:15.386038 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:44:15.386536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:15.386536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:44:15.386536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:44:15.386536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:15.386536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:44:15.386536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:15.386536 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:44:15.386740 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:44:15.387017 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:44:15.387124 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:44:15.387247 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:44:15.387623 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:44:15.387846 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:15.388307 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:44:15.388307 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:15.388307 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:44:15.388307 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:44:15.388307 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:15.388307 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:44:15.388307 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:15.388307 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:44:15.389032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:15.389032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:44:15.389032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:44:15.389032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:44:15.389032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:15.389032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:44:15.389032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:44:15.389032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:44:15.389032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:15.389032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:44:15.389032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:44:15.389032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:44:15.389032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:44:15.389032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:44:15.389032 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:44:15.389453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:44:15.389453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:44:15.389453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:15.389453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:44:15.389453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:44:15.389453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:15.389453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:44:15.389453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:15.389453 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:44:15.389952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:15.389952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:44:15.389952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:44:15.389952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:44:15.389952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:44:15.389952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:44:15.389952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:44:15.389952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:44:15.389952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:44:15.389952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:15.389952 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:44:15.390259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:44:15.390259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:44:15.390259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:15.390259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:44:15.390259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:44:15.390259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:44:15.390259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:44:15.390259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:44:15.390259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:44:15.390259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:44:15.390259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:44:15.390259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:15.390259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:44:15.390259 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:44:15.390740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:15.390740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:44:15.390740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:15.390740 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:44:15.390997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:15.390997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:44:15.390997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:44:15.390997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:44:15.390997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:44:15.390997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:44:15.390997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:44:15.390997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:44:15.390997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:44:15.390997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:44:15.390997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:44:15.390997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:44:15.390997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:44:15.390997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:44:15.390997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:44:15.390997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:44:15.390997 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:44:15.391723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:44:15.391723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:44:15.391723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:44:15.391723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:15.391723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:44:15.391723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:44:15.391723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:44:15.391723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:15.391723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:44:15.391723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:44:15.391723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:44:15.391723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:44:15.391723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:44:15.391723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:44:15.391723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:44:15.391723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:44:15.391723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:15.391723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:44:15.391723 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:44:15.392523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:15.392523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:44:15.392523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:15.392523 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:44:15.392523 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:44:15.392750 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:44:15.392750 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:44:15.392855 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:44:15.393133 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:44:15.393344 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:44:15.393511 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:15.394216 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:44:15.394216 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:15.394216 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:44:15.394216 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:44:15.394216 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:15.394216 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:44:15.394216 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:15.394216 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:44:15.395031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:15.395031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:44:15.395031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:44:15.395031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:44:15.395031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:44:15.395031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:44:15.395031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:44:15.395031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:15.395031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:44:15.395031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:44:15.395031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:44:15.395031 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:15.395550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:44:15.395550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:44:15.395550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:44:15.395550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:44:15.395550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:44:15.395550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:44:15.395550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:44:15.395550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:44:15.395550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:15.395550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:44:15.395550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:44:15.395550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:15.395550 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:44:15.396060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:15.396060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:44:15.396060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:15.396060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:44:15.396060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:44:15.396060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:44:15.396060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:44:15.396060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:44:15.396060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:44:15.396060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:44:15.396060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:44:15.396060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:44:15.396060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:44:15.396060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:44:15.396060 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:44:15.396675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:44:15.396675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:44:15.396675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:44:15.396675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:44:15.396675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:44:15.396675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:15.396675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:44:15.396675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:44:15.396675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:44:15.396675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:15.396675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:44:15.396675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:44:15.396675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:44:15.396675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:44:15.396675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:44:15.396675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:44:15.396675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:44:15.396675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:44:15.396675 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:15.397393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:44:15.397393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:44:15.397393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:15.397393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:44:15.397393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:15.397393 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:44:20.142675 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:44:20.143698 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:44:20.143698 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:44:20.144925 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:44:20.181325 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:44:20.181579 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:20.182204 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:44:20.182204 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:20.182204 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:44:20.182204 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:44:20.182204 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:20.182204 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:44:20.182204 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:20.182204 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:44:20.183166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:20.183166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:44:20.183166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:44:20.183166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:44:20.183166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:44:20.183166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:44:20.183166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:44:20.183166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:20.183166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:44:20.183166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:44:20.183166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:44:20.183166 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:20.183844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:44:20.183844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:44:20.183844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:44:20.183844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:44:20.183844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:44:20.183844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:44:20.183844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:44:20.183844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:44:20.183844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:20.183844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:44:20.183844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:44:20.183844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:20.183844 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:44:20.184517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:20.184517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:44:20.184517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:20.184517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:44:20.184517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:44:20.184517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:44:20.184517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:44:20.184517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:44:20.184517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:44:20.184517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:44:20.184517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:44:20.184517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:44:20.184517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:44:20.184517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:44:20.184517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:44:20.185317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:44:20.185317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:44:20.185317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:44:20.185317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:44:20.185317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:44:20.185317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:20.185317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:44:20.185317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:44:20.185317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:44:20.185317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:20.185317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:44:20.185317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:44:20.185317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:44:20.185317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:44:20.185317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:44:20.185317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:44:20.185317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:44:20.185317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:44:20.185317 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:20.186300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:44:20.186300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:44:20.186300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:20.186300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:44:20.186300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:20.186300 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:44:20.373455 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:44:20.374099 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:44:20.374308 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:44:20.375430 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:44:20.375778 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:44:20.376065 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:20.377005 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:44:20.377005 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:20.377005 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:44:20.377005 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:44:20.377005 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:20.377005 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:44:20.377005 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:20.377005 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:44:20.378584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:20.378584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:44:20.378584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:44:20.378584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:44:20.378584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:44:20.378584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:44:20.378584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:44:20.378584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:20.378584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:44:20.378584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:44:20.378584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:44:20.378584 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:20.379518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:44:20.379518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:44:20.379518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:44:20.379518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:44:20.379518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:44:20.379518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:44:20.379518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:44:20.379518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:44:20.379518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:20.379518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:44:20.379518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:44:20.379518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:20.379518 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:44:20.380351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:20.380351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:44:20.380351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:20.380351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:44:20.380351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:44:20.380351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:44:20.380351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:44:20.380351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:44:20.380351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:44:20.380351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:44:20.380351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:44:20.380351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:44:20.380351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:44:20.380351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:44:20.380351 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:44:20.381758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:44:20.381758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:44:20.381758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:44:20.381758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:44:20.381758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:44:20.381758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:20.381758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:44:20.381758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:44:20.381758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:44:20.381758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:20.381758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:44:20.381758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:44:20.381758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:44:20.381758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:44:20.381758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:44:20.381758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:44:20.381758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:44:20.381758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:44:20.381758 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:20.383266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:44:20.383266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:44:20.383266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:20.383266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:44:20.383266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:20.383266 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:44:20.383266 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:44:20.383266 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:44:20.383266 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:44:20.383266 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:44:20.383852 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:44:20.383852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:20.383852 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:44:20.383852 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:20.383852 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:44:20.383852 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:44:20.383852 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:20.383852 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:44:20.383852 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:20.383852 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:44:20.384597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:20.384597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:44:20.384597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:44:20.384597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:44:20.384597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:20.384597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:44:20.384597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:44:20.384597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:44:20.384597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:20.384597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:44:20.384597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:44:20.384597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:44:20.384597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:44:20.384597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:44:20.384597 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:44:20.385134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:44:20.385134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:44:20.385134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:20.385134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:44:20.385134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:44:20.385134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:20.385134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:44:20.385134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:20.385134 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:44:20.385516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:20.385516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:44:20.385516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:44:20.385516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:44:20.385516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:44:20.385516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:44:20.385516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:44:20.385516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:44:20.385516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:44:20.385516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:44:20.385516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:20.385516 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:44:20.385912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:44:20.385912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:44:20.385912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:20.385912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:44:20.385912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:44:20.385912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:44:20.385912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:44:20.385912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:44:20.385912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:44:20.385912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:44:20.385912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:44:20.385912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:20.385912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:44:20.385912 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:44:20.386251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:20.386251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:44:20.386251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:20.386251 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:44:20.386651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:20.386651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:44:20.386651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:44:20.386651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:44:20.386651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:44:20.386651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:44:20.386651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:44:20.386651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:44:20.386651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:44:20.386651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:44:20.386651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:44:20.386651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:44:20.386651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:44:20.386651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:44:20.386651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:44:20.386651 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:44:20.388517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:44:20.388517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:44:20.388517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:44:20.388517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:44:20.388517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:44:20.388517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:20.388517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:44:20.388517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:44:20.388517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:44:20.388517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:20.388517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:44:20.388517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:44:20.388517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:44:20.388517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:44:20.388517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:44:20.388517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:44:20.388517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:44:20.388517 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:44:20.389612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:20.389612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:44:20.389612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:44:20.389612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:20.389612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:44:20.389612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:20.389612 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:44:20.389612 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:44:20.389612 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:44:20.389612 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:44:20.390400 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:44:20.390400 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:44:20.390400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:20.390400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:44:20.390400 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:20.390400 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:44:20.390400 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:44:20.390400 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:20.390400 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:44:20.390400 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:20.390400 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:44:20.390400 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:20.390857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:44:20.390857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:44:20.390857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:44:20.390857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:20.390857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:44:20.390857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:44:20.390857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:44:20.390857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:20.390857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:44:20.390857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:44:20.390857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:44:20.390857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:44:20.390857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:44:20.390857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:44:20.390857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:44:20.390857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:44:20.390857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:20.392074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:44:20.392074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:44:20.392074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:20.392074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:44:20.392074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:20.392074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:44:20.392074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:20.392074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:44:20.392074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:44:20.392074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:44:20.392074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:44:20.392074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:44:20.392074 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:44:20.392924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:44:20.392924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:44:20.392924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:20.392924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:44:20.392924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:44:20.392924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:44:20.392924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:20.392924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:44:20.392924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:44:20.392924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:44:20.392924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:44:20.392924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:44:20.392924 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:44:20.393344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:44:20.393344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:44:20.393344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:20.393344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:44:20.393344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:44:20.393344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:20.393344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:44:20.393344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:20.393344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:44:20.393344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:20.393344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:44:20.393344 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:44:20.393870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:44:20.393870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:44:20.393870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:44:20.393870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:44:20.393870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:44:20.393870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:44:20.393870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:44:20.393870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:44:20.393870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:44:20.393870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:44:20.393870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:44:20.393870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:44:20.393870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:44:20.393870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:44:20.393870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:44:20.393870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:44:20.393870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:44:20.393870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:20.393870 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:44:20.394487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:44:20.394487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:44:20.394487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:20.394487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:44:20.394487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:44:20.394487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:44:20.394487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:44:20.394487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:44:20.394487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:44:20.394487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:44:20.394487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:44:20.394487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:20.394487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:44:20.394487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:44:20.394487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:20.394487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:44:20.394487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:20.394487 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:44:20.394976 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:44:20.394976 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:44:20.394976 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:44:20.394976 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:44:20.394976 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:44:20.394976 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:44:20.395128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:20.395128 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:44:20.395128 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:20.395128 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:44:20.395128 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:44:20.395128 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:20.395128 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:44:20.395128 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:20.395128 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:44:20.395857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:20.395857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:44:20.395857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:44:20.395857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:44:20.395857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:44:20.395857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:44:20.395857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:44:20.395857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:20.395857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:44:20.395857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:44:20.395857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:44:20.395857 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:20.396104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:44:20.396104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:44:20.396104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:44:20.396104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:44:20.396104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:44:20.396104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:44:20.396104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:44:20.396104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:44:20.396104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:20.396104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:44:20.396104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:44:20.396104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:20.396104 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:44:20.396325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:20.396325 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:44:20.396816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:20.396816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:44:20.396816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:44:20.396816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:44:20.396816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:44:20.396816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:44:20.396816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:44:20.396816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:44:20.396816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:44:20.396816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:44:20.396816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:44:20.396816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:44:20.396816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:44:20.396816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:44:20.396816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:44:20.396816 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:44:20.397114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:44:20.397114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:44:20.397114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:20.397114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:44:20.397114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:44:20.397114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:44:20.397114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:20.397114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:44:20.397114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:44:20.397114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:44:20.397114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:44:20.397114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:44:20.397114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:44:20.397114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:44:20.397114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:44:20.397114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:20.397114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:44:20.397114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:44:20.397114 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:20.397468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:44:20.397468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:20.397468 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:44:25.143487 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:44:25.144387 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:44:25.144387 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:44:25.145058 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:44:25.150481 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG dbcounter [-] [105581] Writing DB stats neutron:SELECT=240 {{(pid=105581) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 14:44:25.179935 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:44:25.180102 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:25.180755 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:44:25.180755 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:25.180755 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:44:25.180755 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:44:25.180755 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:25.180755 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:44:25.180755 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:25.180755 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:44:25.181668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:25.181668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:44:25.181668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:44:25.181668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:44:25.181668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:44:25.181668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:44:25.181668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:44:25.181668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:25.181668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:44:25.181668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:44:25.181668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:44:25.181668 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:25.182042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:44:25.182042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:44:25.182042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:44:25.182042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:44:25.182042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:44:25.182042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:44:25.182042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:44:25.182042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:44:25.182042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:25.182042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:44:25.182042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:44:25.182042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:25.182042 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:44:25.182324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:25.182324 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:44:25.182684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:25.182684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:44:25.182684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:44:25.182684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:44:25.182684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:44:25.182684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:44:25.182684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:44:25.182684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:44:25.182684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:44:25.182684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:44:25.182684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:44:25.182684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:44:25.182684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:44:25.182684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:44:25.182684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:44:25.182684 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:44:25.183082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 14:44:25.183082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:44:25.183082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:25.183082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:44:25.183082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:44:25.183082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:44:25.183082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:25.183082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:44:25.183082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:44:25.183082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:44:25.183082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:44:25.183082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:44:25.183082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:44:25.183082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:44:25.183082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:44:25.183082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:25.183082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:44:25.183082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:44:25.183082 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:25.183515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:44:25.183515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:25.183515 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:44:25.374217 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:44:25.374749 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:44:25.374939 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:44:25.376103 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-5dcc5c62-3db6-43c1-9b38-e1c7de943f13, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:44:25.376485 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-09e042cf-39a9-42c8-b683-b29ffc42b049, routes=[], if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:44:25.376748 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:25.377661 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:44:25.377661 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:25.377661 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:44:25.377661 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:44:25.377661 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:25.377661 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:44:25.377661 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:25.377661 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:44:25.379616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:25.379616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:44:25.379616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:44:25.379616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:44:25.379616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:44:25.379616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:44:25.379616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:44:25.379616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:25.379616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:44:25.379616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:44:25.379616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:44:25.379616 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:25.380379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:44:25.380379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:44:25.380379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:44:25.380379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:44:25.380379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:44:25.380379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:44:25.380379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:44:25.380379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:44:25.380379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:25.380379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:44:25.380379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:44:25.380379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:25.380379 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:44:25.381116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:25.381116 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:44:25.381641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:25.381641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:44:25.381641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:44:25.381641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:44:25.381641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:44:25.381641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:44:25.381641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:44:25.381641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:44:25.381641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:44:25.381641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:44:25.381641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:44:25.381641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:44:25.381641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:44:25.381641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:44:25.381641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:44:25.381641 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:44:25.382218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 14:44:25.382218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:44:25.382218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:25.382218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:44:25.382218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:44:25.382218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:44:25.382218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:25.382218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:44:25.382218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:44:25.382218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:44:25.382218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:44:25.382218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:44:25.382218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:44:25.382218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:44:25.382218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:44:25.382218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:25.382218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:44:25.382218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:44:25.382218 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:25.382773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:44:25.382773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:25.382773 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:44:25.383120 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:44:25.383327 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:44:25.383452 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:44:25.383595 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105581) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 14:44:25.383957 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:44:25.384149 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:25.384709 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:44:25.384709 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:25.384709 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:44:25.384709 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:44:25.384709 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:25.384709 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:44:25.384709 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:25.384709 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:44:25.385462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:25.385462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:44:25.385462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:44:25.385462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:44:25.385462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:25.385462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:44:25.385462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:44:25.385462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:44:25.385462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:25.385462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:44:25.385462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:44:25.385462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:44:25.385462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:44:25.385462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:44:25.385462 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:44:25.385918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:44:25.385918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:44:25.385918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:25.385918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:44:25.385918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:44:25.385918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:25.385918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:44:25.385918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:25.385918 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:44:25.386354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:25.386354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:44:25.386354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:44:25.386354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:44:25.386354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:44:25.386354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:44:25.386354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:44:25.386354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 14:44:25.386354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:44:25.386354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:44:25.386354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:25.386354 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:44:25.386750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:44:25.386750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:44:25.386750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:25.386750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:44:25.386750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:44:25.386750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:44:25.386750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:44:25.386750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:44:25.386750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:44:25.386750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:44:25.386750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:44:25.386750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:25.386750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:44:25.386750 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:44:25.387298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:25.387298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:44:25.387298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:25.387298 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:44:25.387877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:25.387877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:44:25.387877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:44:25.387877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:44:25.387877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:44:25.387877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:44:25.387877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:44:25.387877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:44:25.387877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:44:25.387877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:44:25.387877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:44:25.387877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:44:25.387877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:44:25.387877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:44:25.387877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:44:25.387877 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:44:25.388350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 14:44:25.388350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 14:44:25.388350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 14:44:25.388350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:44:25.388350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:44:25.388350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:25.388350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:44:25.388350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:44:25.388350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:44:25.388350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:25.388350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:44:25.388350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:44:25.388350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:44:25.388350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:44:25.388350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:44:25.388350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:44:25.388350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:44:25.388350 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:44:25.388902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:25.388902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:44:25.388902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:44:25.388902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:25.388902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:44:25.388902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:25.388902 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:44:25.388902 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:44:25.388902 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:44:25.388902 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:44:25.389228 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:44:25.389228 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:44:25.389228 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:25.389705 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:44:25.389705 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:25.389705 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:44:25.389705 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:44:25.389705 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:25.389705 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:44:25.389705 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:25.389705 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:44:25.390417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:25.390417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 14:44:25.390417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:44:25.390417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:44:25.390417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:25.390417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:44:25.390417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:44:25.390417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 14:44:25.390417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:25.390417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 14:44:25.390417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:44:25.390417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 14:44:25.390417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:44:25.390417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 14:44:25.390417 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 14:44:25.391059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:44:25.391059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 14:44:25.391059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:25.391059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 14:44:25.391059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 14:44:25.391059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:25.391059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 14:44:25.391059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:25.391059 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 14:44:25.391375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:25.391375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:44:25.391375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:44:25.391375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:44:25.391375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:44:25.391375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:44:25.391375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:44:25.391375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:44:25.391375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:44:25.391375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:25.391375 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:44:25.391731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:44:25.391731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:44:25.391731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:25.391731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:44:25.391731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:44:25.391731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:44:25.391731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:44:25.391731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:44:25.391731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:44:25.391731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:44:25.391731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:44:25.391731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:25.391731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:44:25.391731 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:44:25.392238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:25.392238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:44:25.392238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:25.392238 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:44:25.392502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:25.392502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:44:25.392502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:44:25.392502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:44:25.392502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:44:25.392502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:44:25.392502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:44:25.392502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:44:25.392502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:44:25.392502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:44:25.392502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:44:25.392502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:44:25.392502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:44:25.392502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:44:25.392502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:44:25.392502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:44:25.392502 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 14:44:25.393355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 14:44:25.393355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 14:44:25.393355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 14:44:25.393355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:25.393355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:44:25.393355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:44:25.393355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:44:25.393355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:25.393355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:44:25.393355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:44:25.393355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:44:25.393355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:44:25.393355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:44:25.393355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:44:25.393355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:44:25.393355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:44:25.393355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:25.393355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:44:25.393355 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:44:25.393991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:25.393991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:44:25.393991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:25.393991 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics 
May 09 14:44:25.393991 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105581) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 14:44:25.393991 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 14:44:25.393991 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105581) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 14:44:25.393991 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105581) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 14:44:25.394299 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:44:25.394344 np0000173183 neutron-ovn-maintenance-worker[105581]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105581) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 14:44:25.394623 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:25.395541 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Traceback (most recent call last):
May 09 14:44:25.395541 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:25.395541 np0000173183 neutron-ovn-maintenance-worker[105581]:     txn.results.put(txn.do_commit())
May 09 14:44:25.395541 np0000173183 neutron-ovn-maintenance-worker[105581]:                     ^^^^^^^^^^^^^^^
May 09 14:44:25.395541 np0000173183 neutron-ovn-maintenance-worker[105581]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:25.395541 np0000173183 neutron-ovn-maintenance-worker[105581]:     raise RuntimeError(msg)
May 09 14:44:25.395541 np0000173183 neutron-ovn-maintenance-worker[105581]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:25.395541 np0000173183 neutron-ovn-maintenance-worker[105581]: 
May 09 14:44:25.397117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:25.397117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 14:44:25.397117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:44:25.397117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 14:44:25.397117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 14:44:25.397117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:44:25.397117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:44:25.397117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:25.397117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:44:25.397117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:44:25.397117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 14:44:25.397117 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:25.397585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 14:44:25.397585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:44:25.397585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 14:44:25.397585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:44:25.397585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 14:44:25.397585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 14:44:25.397585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:44:25.397585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 14:44:25.397585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:25.397585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 14:44:25.397585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 14:44:25.397585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:25.397585 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 14:44:25.398006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:25.398006 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 14:44:25.399027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics [None req-db27d7e9-dd74-4f85-852c-16db901cddd6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:25.399027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics Traceback (most recent call last):
May 09 14:44:25.399027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 14:44:25.399027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     work()
May 09 14:44:25.399027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 14:44:25.399027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 14:44:25.399027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:44:25.399027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 14:44:25.399027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 14:44:25.399027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 14:44:25.399027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 14:44:25.399027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 14:44:25.399027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 14:44:25.399027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 14:44:25.399027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 14:44:25.399027 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 14:44:25.399681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 14:44:25.399681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 14:44:25.399681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:25.399681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:44:25.399681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 14:44:25.399681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 14:44:25.399681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 14:44:25.399681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     next(self.gen)
May 09 14:44:25.399681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 14:44:25.399681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     with self.create_transaction(
May 09 14:44:25.399681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 14:44:25.399681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     self.result = self.commit()
May 09 14:44:25.399681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 14:44:25.399681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 14:44:25.399681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise result.ex
May 09 14:44:25.399681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 14:44:25.399681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 14:44:25.399681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 14:44:25.399681 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 14:44:25.400283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 14:44:25.400283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 14:44:25.400283 np0000173183 neutron-ovn-maintenance-worker[105581]: ERROR futurist.periodics